Illustrated

Illustrated is a simple web app that converts normal books into picture books to empower visual learning.

Comment

Inspiration

Illustrated was created after looking at research regarding visual learning. Studies have shown visual learning has a big impact on how well and how fast people learn. Illustrated was created to help better promote visual learning, making it a tool accessible to everyone.

What it does

Illustrated is a simple web app that takes normal books or texts and adds images to them. Users can choose to download the newly generated book as a PDF.

How we built it

Illustrated uses Next.JS and tailwindcss stylingon the frontend and takes in either a text file or text from the text box. This information is sent to the server where the text is broken down into sentences. A sentence ranking algorithm sorts the sentences by importance. The top 1/4 most important sentences are selected and uses. The selected sentences are fed into a server running stable diffusion where images are generated. These images are returned to the user where it can be rendered. The new book can then be downloaded as a PDF using the jsPDF library.

Challenges we ran into

  1. Finding a way to upload large machine learning data files to github (solution was to use gzip to compress the data files)
  2. Hosting machine learning models that require a gpu (solution was to run flask on a google colab notebook and utilize their gpu) ## Accomplishments that we're proud of
  3. Implementing a sentence ranking machine learning algorithm
  4. Implementing Stable Diffusion ai image generation.
  5. Converting html to PDF ## What we learned
  6. Gained a better understanding of machine learning tools
  7. We learned how to plan ahead and organize our workflow
  8. As a team we developed stronger bonds as friends
  9. We learned how to quickly respond to difficult situations and logically think them through. ## What's next for Illustration
  10. Improve the PDF download functionality
  11. Add PDF text recognition as an input for text
  12. Add camera text recognition as an input for text
  13. Add video function where text is converted to a video

Built With

Share this project:

Updates