A demo consumer of Spark-DS/logic-schema, a Scala library to create data specification and validation, help developing applications based on Apache Spark.
Spark-DS/logic-schema follows the principal of code as specification and configuration, it enables developers to
-
Create Apache Spark data schema;
-
Specify constraints;
-
Compare Schemas;
-
Validate data against schema.
Follow this document to install necessary software.
Open a terminal (Unix) or Command (Windows) window
Then follow these steps:
git clone https://github.com/Spark-DS/logic-schema-demo.git cd logic-schema-demo sbt test