MAGIC SLATE

An intelligent personal tutor that teaches you how to draw stuff...from basic shapes to the most beautiful sceneries. Submit your artwork to the automated grading system to see how you can improve!

Comment

Inspiration

Drawing is a learnable skill

I was always amazed by art and the skill involved. However, not having the innate talent of creating masterpieces, I was always lost on where to start. Lockdown came and I had the perfect opportunity to start. I had access to a ton of resources, but, there was no all-in-one solution.

And the bigger problem, I did not know if I was on the right path. I needed to know my progress, and basically how I could improve.

And thus I was motivated to fill in this gap of free resources on the internet.

What it does

Magic Slate is an application that, in short, teaches you how to draw. With an organized curriculum and clear video tutorials, all you have to do is get a pen and paper and start.

There are 7 lessons (not completed yet) that take you on a journey from a beginner to an expert. With exercises accompanying each lesson, you can test your progress.

And the best part, upload a picture of your artwork and get it graded automatically. Get recommendations so that you can learn from your mistakes.

How I built it

The android application is built using Android Studio (Java). The backend python scripts required for image processing and comparison are done using Python and Flask web framework.

Step 1: User can create accounts and login using the android application. There is a series of tutorials, drills and challenges available. After completing a tutorial, user can try out the drills and challeneges to check their progress. The objective image is provided, and user attempts to replicate it. The user can take a picture of the document and upload it on the application for grading.

Step 2: The image is then POSTed to the server backend using RetroFit API. Currently, the server is running on localhost, but can be migrated to a remote cloud-based server such as AWS or Google.

Step 3: In the backend, a python script runs to clean and process the image. The image is then compared against the ideal solution and graded based on the similarity. The steps involved are:

  1. Color Correction using Histogram Matching. This is to deal with varying contrasts, brightness and colors.
  2. Edge Detection and Contour detection using OpenCV. This is to find the region of interest (i.e. the drawing)
  3. Four Point Transform to adjust the perspective to get a standard 90o bird-eye view of the image.
  4. Local Thresholding to give it the 'black-and-white' paper effect.
  5. Structural Similarity, MSE and Key-Points Matching to find level of similarity between the two images.

Step 4: The server (in this case, the pyrhon script) return the final grade as a Response, which is then captured by the application and displayed to the user. For premium users, recommendations and corrections are also provided.

Challenges I ran into

Firstly, this was the first time I integrated an Android Application with Python Scripts on the backend. So, understanding the concept of REST APIs and Framework in a limited time was quite a challenge.

Secondly, a comparison of sketches and drawings is quite a difficult task because of all the details. And the addition of uncontrolled environments for images (referring to the user-clicked sketches) makes it even more challenging.

And finally, needless to say, Android Studio (and Java) is one of the most difficult tools to use...

Accomplishments that I'm proud of

I am quite proud of what I was able to accomplish in this whole project. From building the android application to integrating it with an image processing python script was quite an achievement for myself.

What's next for MAGIC SLATE

Well, it's nowhere near done. And perhaps, I would like to pursue this as a fully-fledged project. Regardless, some major areas of improvement are:

  1. The User Interface - I tried my best to make the UI quite clear, but it undoubtedly requires a lot of work.
  2. The image comparison system - The current system is not quite furnished. Some ideas that come to mind are:
    • Using a SIFT or SURF algorithm alongside the current system for image comparision.
    • Implementation of continuously-learning convolutional neural network that takes verified artwork from users to compare with the user-submitted drawing.
    • A more robust and accurate image cleaning technique.
  3. Backend Server - I would love to host the python scripts on a backend server (like AWS) so that this app could be functional at all times.
  4. Business Integration - A subscription service that allows users to get personalized recommendations, unlimited uploads, and more...
Share this project:

Updates