Recipe Architech

Homescreen
Generated Recipe
Saved Recipes
Another Generated Recipe

Inspiration

Groceries are expensive and learning new recipes all the time can be very overwhelming and time consuming. As struggling engineering students, we are always looking for ways to save money :^)

What it does

Recipe Detect scans grocery flyers for the cheapest deals. Then, a list of recipes are generated based on the available deals, and existing items in the user's pantry. Users can then select a recipe and cook it, or even just save it to their list of favorite recipes!

How we built it

Our language of choice was Python 🐍

Frontend

Heroku - We hosted our frontend at the url http://recipearchi.tech/ (get it :D)
Taipy - Used to generate and build the frontend

Backend

Flask - Easy interfacing with all the python libraries we used
OpenCV - Used to crop individual items from each grocery flyer based on color thresholds
Google Cloud Vision API - Used OCR to read each grocery listing and identify text and prices
Cohere - LLM used to interpret the relevant information in each section of text read by the OCR; also used to generate recipes based on available ingredients
MongoDB - Used to store and manage existing pantry items
CockroachDB/SQLAlchemy - Used to store and manage favourited recipes

Challenges we ran into

We all aimed to try new technologies in this project! Taipy was a lot of fun to work with, but the fact that it's a newer framework made it so that there were very few resources to fall back on when we were encountering hard to debug issues. For OpenCV, we had a hard time finding what threshold to crop at to limit each color. We spent a lot of time tweaking the settings to make sure everything was cropped well. The OCR kept detecting extra text or grouping them together in weird ways. We had to play around a lot with the labels to get it working effectively! We added a lot of messages to our LLM to help it give us ideal answers! That was a lot of fun to play around but it was also pretty time consuming. We had fun generating weird recipes like toilet plunger cheetos soup 😋 I was also kinda annoyed by Heroku :((( DNS is hard

Accomplishments that we're proud of

Our app is in a very usable state, which is something that we're happy with. 🥳 Having an app fully live up to your vision is hard but we were able to come pretty close in this hackathon. Our time management was also something that ended up going very well. All of us had times where we had to give up sleep to finish up a project, but we wanted to make sure that we worked smarter instead of harder and wanted to keep ourselves free from becoming zombies. To do this, we meticulously planned the tasks we all needed to do and made sure that we were all constantly getting something done, which worked out very well in the end! We also challenged ourselves to use good coding practices in our repo and code!

What we learned

In terms of frontend, we learned lots about how to use Taipy! We feel like it'll be a great tool for building quick frontends for Python apps in the future and hope it gets more users and widespread recognition. For backend, we learned about using SQLAlchemy and Flask with CockroachDB. Our usecase was simple, but we think it was worth it since we were able to learn a lot! Similarly, it was fun trying out MongoDB as well and comparing the two systems. We used LLMs a lot throughout this project and we thought it would be a great opportunity to try and learn all about Cohere which had a neat prize and even cooler product 😎. It was honestly great to play around with the LLMs and we thought of so many other ways to use them - we will definitely be using them again!!! OpenCV is the de facto library for modifying images and since we needed its functionality we chose to go for it and it felt extremely satisfying. Image processing is super cool!!!

What's next for Recipe Architect

We truly want to revolutionize the cooking industry and so we want to add more features as well as overhaul some parts of the current app. The current Taipy frontend worked well for the purposes of a hackathon because of its ease of development, but we'll want to switch to something that has more flexibility in the future, like React or Svelte. Additionally, the way that we parse the flyers and extract the text has room for error and so we'll want something that works a lot more consistently. We're also considering training our own OCR model to improve flyer recognition because we've been having some trouble with that too.

Other random ideas we had :)))