Roast Generator

Guiding user's face position to centre using a face outline
Example roast of a male, with no beard, who looks young

Inspiration

Our inspiration for the Roast Generator project came from the desire to blend humour and technology in a unique and engaging way. We wanted to create an interactive and fun experience for users while exploring the capabilities of computer vision and React. The idea of allowing users to take a photo of themselves and then receive "roasts" based on their facial features was born out of the belief that technology can not only be educational and functional but also entertaining.

What it does

Roast Generator is a React-based web application that leverages computer vision to analyze a user's facial features and generate humorous and light-hearted "roasts." Users can simply take a photo of themselves using their device's camera, and our algorithm does the rest. It identifies key facial features, such as the shape of the user's nose, the size of their ears, or the expression on their face, and combines this information with a Large Language Model to generate witty and humorous personalized roasts. The roasts are then displayed on the screen, providing users with a good-natured laugh at their own expense.

How we built it

We built Roast Generator using a combination of technologies and tools, including:

React: We used React to create the user interface and manage the application's components, ensuring a smooth and responsive user experience.

Computer Vision Face Recognition Model: After scanning the image of a user's face, we ran the image through our model to output an array of 0s and 1s each corresponding to a facial features and whether the user has them. We used transfer learning principles by using a base model of resnet50 and then trained a dataset of face images with their associated features (source: https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)

The images were easy to preprocess since they all had identical dimensions and the eyes of the faces were placed in the same pixel location every time. Once we trained this model, we exported it and connected it to our backend API call once a user takes a picture of themselves.

OpenAI API: Using the facial features collected from the computer vision model output, it sent the results to the OpenAI API and processed through its LLM to generate a roast.

Frontend Design: We invested time in designing an intuitive and user-friendly interface that makes it easy for users to take photos and view their roasts.

Challenges we ran into

Developing Roast Generator presented us with several notable challenges, particularly in the areas of deployment and training our machine learning model within a limited timeframe.

Deployment Challenges: We could not successfully deploy in time.

Training Model with a Large Dataset: Training our machine learning model to generate witty roasts based on facial features required a substantial dataset. Tweaking the model parameters until we got a minimal error was difficult and required a lot of trial and error.

Accomplishments that we're proud of

Throughout the development of Roast Generator, we achieved several significant accomplishments:

Facial Feature Analysis: Although the model is not highly accurate, we successfully implemented a facial feature analysis system that can identify certain key facial features in user photos. Considering this was our first time working with images in deep learning, we are very proud!

Machine Learning Integration: Our machine learning model has been trained to generate personalized and humorous roasts, making the user experience truly unique.

User-Friendly Interface: We designed an intuitive and visually appealing user interface that ensures an enjoyable user experience from start to finish.

What we learned

Roast Generator was an incredible learning experience for our team. We gained insights into various technologies and domains, including computer vision, machine learning, web development, and user interface design. Some of our key takeaways include:

The power of computer vision in analyzing and interpreting visual data.

The challenges and complexities of training machine learning models for specific tasks.

The importance of user experience and interface design in making a project engaging and accessible.

What's next for Roast Generator

While Roast Generator is already a unique and entertaining project, we have exciting plans for its future:

We plan to work on the model by training on an external GPU to handle more data. This way, it can provide accurate facial features for each to make the roasts funnier and honest!