MAISpeare

To sleep, or not to sleep / That is the question~
Poem generation
About page
Who we are page

💡 Inspiration

”A picture is worth a thousand words"...but which words are they? That’s what our team at MAISpeare set out to discover on this 24h journey through many a venv and epoch…

Everyone has been there: rendered speechless by surprise, stumped for inspiration with a deadline looming, or the classic “it’s….ah, how do I describe it?” From students, writers, to the visually impaired, converting visual stimuli to written art is a challenge that everyone faces in their lifetime. Using machine learning, MAISpeare is here to solve such situations with the help of Google Vision and our own LSTM neural network to write poems for any image input!

Stop being rendered speechless, start rendering our templates!

✍️ What it does

Takes user image input (.jpeg or .png)
Leverages Google Vision API to label all objects recognized from the image
Passes labeled objects to our Long Short-Term Memory neural network, trained on over 20,000 lines of poetry from the Poems Dataset (NLP).
Using the poem prompt, our model then outputs a poem of 30 words, split into stanzas as a unique free verse poem about the image

⚙️ How we built it

Web App front end: vanilla Javascript, HTML5/CSS
Web App Back end: Flask, Python
Infrastructure: Google Vision API, LSTM using Tensorflow Keras

📉 Challenges we ran into

As with all hackathons, "the course of true functionality never did run smooth". Some issues we encountered include, but are not limited to: Google Colab refusing to work for our free account any further; Alex’s file organization; too much training data and too little training data at the same time

📈Accomplishments that we're proud of

Completing a functional remotely-hosted web app within 24h
Achieving mostly grammatical generation of English sentences (it's called Artistic License, guys)
Learning more about poetry, text generation, and Flask

🧠 What we learned

With limited experience in Flask and varying levels of experience with machine learning, MAIS Hacks 2021 was a great learning opportunity for the whole team. It was the first time implementing the Google Vision API for some, first deep dive into Flask for others, and a brand new perspective on poetry and text generation for everyone in our most ambitious (and fun!) hack yet! We are grateful for the marvellous experience of reading badly generated poetry at 4am.

🔜What's next for MAISpeare

In the future, we'd love to continue training our model on larger datasets to improve its accuracy! With more time we'd also like to incorporate the option to upload saved images as input; implement a parser to better exploit information about sentence-internal structure; count syllables to fit into different meters of poetry; incorporate rhyming and other poem structures; and maybe even training our model on datasets in different languages!

"I would not could not on a boat, read the mess of code I wrote" - Alex