Culinary Vision

Inspiration

We were inspired by the environmental issue of food wastage and wanted to come up with an easy, efficient solution for individuals to help contribute to the solution for this problem.

What it does

The site takes the image of food items and uses a computer vision model to detect what ingredients are present in a bounding box format, and passes this list of ingredients into the spoonacular API which gives the user three options for potential recipes to cook from the available ingredients.

How we built it

We used YOLOV8 for the object detector, we ensembled it with other previous YOLO models, and combined the predictions with spoonacular API. We decided on simple web upload user flow, with a animated carosell to display the three recipe options.

Challenges we ran into

One of the main challenges we ran into was integrating the YOLO model with the website, which led us to use Flask. Additionally, none of us had worked with deployment, which was a new skill we had to acquire during the project. We also tried to train a model using MMdetection on a larger data set of approximately 35,000 samples but due to training times being 20+ hours, we chose another option.

Accomplishments that we're proud of

We are proud of the individual elements of the project, especially the initial concept. We managed to combine three different aspects of object detection, understanding the workings of NLP to generate recipes, and putting it all through on a website.