Manuela

Inspiration

Sometimes, its hard to find someone you trust to talk to. Manuela offers a solution for people who feel like they have no one to talk to and will keep your secrets safe (because she won't remember).

What it does

Manuela is an easily accessible buddy there for you, when it can be hard to find others. Maybe there is no one around to talk to or you find it difficult to talk to another person, Manuela finds solutions using OpenAI to help resolve your mood problems.

How we built it

We used a ton of external libraries to build Manuela. We created an initial list of core features and broke them down further into easily digestible stories. After we built the features, we started stitching the bits and pieces together. We started with detecting emotion from speech and then moved onto detecting emotion based on a facial image of a person.

Challenges we ran into

Among the numerous problems we ran into, we had trouble with the compatibility of different members' laptops. Setting up a working coding environment between everyone's laptops was a burden as some members had different python setups or anaconda (a special version of python) which took away precious development time. Additionally, the Github action checks we created were such a headache at the start for all of us, but turned out for the better in the end as we kept our code in good and running condition, particularly for our main branch. The checks also helped keep our coding style similar to make reviewing easier (cutting out random changes that are really only for formatting). With our limited skillset, we spent a long time debugging and putting features together that we were unfamiliar with. Lastly, we had issues dividing up the work as we were not sure where to start for some of them and some small tasks turned out to be bigger tasks.

Accomplishments that we're proud of

• OpenAI based responses
• Tensorflow to recognize emotions based on camera feed
• Dedicated emotion recognition for text
• Linking emotion reading from text and face together

What we learned

As the hours went on, we improved with git and Github conventions. We had a member who was not so familiar with git or github, but we all stepped in as a team to help fill the gaps and improve our knowledge on git and github as a whole. Additionally a majority of the python libraries, were unfamiliar to us. Luckily, we were able to find tutorials that helped us understand the library enough to implement our core functions. Working out dependencies was also a challenge in of itself, but in the end, we were able to rise up to the challenge and conquer it.

What's next for Manuela

We hope to bring Manuela outside of a computer and into the real world. We would like her to be more than just a simple python app on a computer. We envision a stuffed animal, like a teddy bear that mimics movements of a comforting person in your worst moments and a cheerleader ready to listen all your best stories and moments of your life.

Built With

• deepface
• openai
• python
• tts