Doodlist

iOS interface

Inspiration

A lot of people once had childhood dreams of being world-class artists. Doodling is usually the first attempt as well as the last one. The app Doodlist can transform simple doodles into masterpiece drawings in just a couple of seconds. Doodlist not only broadens users' creative horizons, but also promotes art as a powerful form of expression. Most importantly, it brings out the best artist in every one of us.

What it does

Our project is an iOS app based on Mathematica and Wolfram Cloud. Users can take pictures of their doodles or upload images from their local photo library. The app automatically matches the doodle with a masterpiece of a similar style. Then the machine learning algorithm applies the most prominent features of the masterpiece to the doodle. The output image is a modified version of the doodle with world-class art style!

How we built it

We modified the current state-of-the-art Convolutional Neural Network: Deep Residual Network to implement a classifier of doodle styles and used a style-transfer function in Mathematica. Then we trained the classifier with doodle images collected from the internet by a python-based web crawler. In the end, we used Xcode to implement an iOS application that wraps the webpage from Wolfram Cloud.

Challenges we ran into

This is the first time that we have used swift to create an iOS app based on Wolfram Cloud.
It is hard to systematically categorize the styles of doodles. We need to collect a lot of doodles from the internet to train our classifier.
It is difficult to determine how similar the doodle's style is to that of the masterpiece.
Tuning the hyper-parameters of the neural network and deploy the functions on Wolfram Cloud.

Accomplishments that we're proud of

We have successfully created a functioning app and linked it to Wolfram Cloud.

What we learned

Machine learning is tricky. It is very difficult for machines to determine aesthetic styles of different drawings. Also, the result of style transferring is not always satisfactory.

What's next for Doodlist

Build a more sophisticated classifier that can determine multiple styles of doodles and link the doodles to a similar art genres

Built With

mathematica
swift

Submitted to

HackCMU 2017

Created by

I collected hundreds of doodle images using an open-source Python scraper .

Lavender Jiang
I worked on collecting and cleaning the image set for training the classifier. Idea generator. Main contributor to demo presentation.

Tianyi Zhu
Electrical & Computer Engineering at Carnegie Mellon University
I reconstructed the current (almost) state-of-the-art Convolutional Neural Network: Deep Residual Network in Mathematica and modified it to suit our needs. I also deployed the classifier and the style transfer function on Wolfram Cloud.

Matthew Chen
Xingsheng Wang