BreedReveal

Inspiration

My family had a hard time first taking care of our dog, and we didn't have enough information on the dog's breed since it was adopted and heavily mixed. To prevent others from going through this experience, especially in the wake of COVID-19, I created this python program to help people take care of their dogs.

What BreedReveal does

Given a photo of a dog, it predicts the dog's breed and gives additional information on how take care of it, with the additional information being generated by OpenAI's Assistant API

How BreedReveal works

We used pytorch and transfer learning to create an optimized neural network for image recognition. After several training repetitions, I got it to a point where it could reliably predict dog breeds. Afterwards, I used the Assistant API from OpenAI to create a client thread, which when fed messages to it, would run an AI assistant that can provide users with basic info on how take care of the dog

Challenges we ran into

Downloading the Assistant API and working with it, as well as processing the various training errors and image scaling errors for PyTorch, were both rather significant problems. Overfitting was also a problem that I was worried about, which is when a model is trained too much on the training data and as a result is only optimized for that data (it performs poorly in testing data).

Accomplishments that we're proud of

The model surprisingly got 70% after several repetitions of training, which I am happily surprised by. Not only did it get such a high accuracy relatively soon, but I can also easily improve it by training it more.

What we learned

I learned a lot more about PyTorch and the Assistant API. Although I previously had some experience with PyTorch, this hackathon really led me to program by myself and become thoroughly familiar with the Python library. It was my first time using Assistant API, and it was a rather challenging but fulfilling experience.

What's next for BreedReveal

I first want to move the code to a website (HTML, CSS, JS), since python isn't as easily accessible to everyone as a URL. I also want the user to have the option to input videos, as videos generally have more data which can lead to the model making more accurate predictions.