Sebastian Crowell

• 

  Using annotations to draw boxes around many flowers by hand

• 

  Favicon for the site

Inspiration

The project stemmed from my enjoyment in poetry but my shyness in sharing that poetry. I always had a little tinge of regret about not giving the girl I liked in high school any of the poems I wrote and I figured this was sort of a fun and integral part of valentines that people shouldn't miss out on. I started with that premise and remembered a machine learning assignment I did to identify state parks based on the arches in it and I went from there.

What it does

It is a machine learning data set with handpicked labels (3000 flowers labeled over 6 hours) from IBM Watson and tensor flow in google cloud fed into a JavaScript React web app that harnesses a live feed from the integrated camera in the device running it. The live camera is used to give a prediction off of the model in real time and when the user is ready, they can choose to make a poem using the current flower or two with the highest confidence with a simple rhyme using Flower is Color, Flower is Color, Non-rhyme, Rhyme with 2nd Flower is Color.

How we built it

I started with the ml first to see if I would even have time to get a set trained and modeled, everything went smooth so I flipped a coin on if I would do a web app or android app, got web so I grabbed react and started from there. Got the model and everything hooked up quickly off of a couple of well documented sites and started getting things tidy but the app still needs about 20 more hours of coding and 40-50 hours of training really.

Challenges we ran into

I started with python but ran into an issue where the model was going to use my gpu over about 300 hours which wasn't feasible and it would still be not very accurate. I was indecisive about how I wanted the scheme to work and was flip-flopping between making a color picked for the canvas object in JavaScript but didn't like it and then I was wondering about scraping rhyming words and I had to find a way to test everything because the training set was sort of sparse being hand picked and labeled.

Accomplishments that we're proud of

The app works like I wanted at least.

What we learned

I got a lot of hands on experience with the IBM suite and was quite impressed with how it handled things.

What's next for Sebastian Crowell

I want to retrain the data, I want to get the scraping up to full, I want to do some branch predicting for the poems and I want to host it on firebase.

Built With

• cheerio
• javascript
• python
• react
• tensorflow