↖data source
My goal is to utilize the hadoop ecosystem to handle a large dataset and establish a pipeline for machine learning
munge :
- Aggregate columns using RDD transformations
- Create a column that indicates which of those column aggregations are outliers.
- Model data with Spark Machine Learning package
- Predict on test data
- Run this as is to use the toy data set example