Inspiration
The Clarifai API is a powerful tool -- but it can do so much more than just dissect images. It can tell stories. They say a picture is worth a thousand words. Just how many of those words can Clarifai help us find?
What it does
Once Upon a Pixel allows users to text photos to its server. It then posts that picture to the main feed, where it is accompanied by an extensive caption written by famous authors.
How we built it
First things first, we collected a massive corpus of famous novels that were available for free online. We cleaned the text and iterated over ever word of every novel, logging the coordinates of every instance of every word in a massive hashmap and wrote the map out to JSON.
Then we set up a Flask server to accept incoming image texts from anyone who as our number. We used to Twilio API to power this feature.
We use the Clarifai API to look at each given image and provide tags -- letting us know what is in the picture. This is critical.
Then we select the most prevalent tags and retrieve their coordinates from our master hashmap of words. We randomly select a set of coordinates from the given list, and then go find the instance of that tag in the text. Then we extract the surrounding passages with regular expressions and stitch together the passages for all of the top tags.
This is the passage that we post to the main feed. And there you have it!
Challenges we ran into
We started the hackathon on a completely different project and did not pivot until late Saturday afternoon. The pressure to build a new application in the remaining time was immense, but we managed.
Accomplishments that we're proud of
Our team was coordinated and worked well together. We managed to survive a complete failure halfway through the hackathon and successfully pivot to a full web application.
What we learned
We all learned tons of new technology, and a huge amount about teamwork!
What's next for Once Upon a Pixel
Expand the literature base!!! This will make much more varied stories. Also expand tag searches to synonyms! Oh and maybe make it available to the public...
Log in or sign up for Devpost to join the conversation.