Zen Agent

Inspiration

Stress, anxiety, and other emotions can be detrimental to people's health and well-being. We were inspired to harness technology to create a project that strives to identify any early signs of negative emotions to improve people's mindfulness.

What it does

Zen Agent is an AI Agent that estimates stress and mindfulness based on facial indicators. It is trained on an ML model to provide assistance such as:

Providing notifications during virtual meetings to check your emotions
Suggesting a quick relaxation exercise to reduce stress and anxiety
Other useful tips and information derived from your facial patterns

How we built it

System diagram

Zen Agent was developed using the following tools and technologies:

Next.js Frontend
Python Backend
LangGraph Agent
WebRTC and WebSocket connection

Challenges we ran into

Verifying that the agent works properly with a recording (video) file.
Connecting the backend and frontend components to get the project to deploy successfully.
Finding the balance between too few and too many factors for the model to consider.
Getting the dashboard to display data visualizations based on the video.

Accomplishments that we're proud of

Developing a modern and simple User Interface for easy use and navigation.
Creating a trained ML model that can accurately determine emotions based on facial cues.
The intricacies involved in successfully fusing the frontend and backend of applications.
Integrating agentic AI into our project to provide recommendations based on the data analyzed.

What we learned

Being able to map facial expressions to quantifiable measurements that our model can examine.
How to best train ML models to prevent model overfitting and account for a broader population of data.
The challenges encountered in processing live video in applications for a smooth, responsive user experience.
How agentic AI can be utilized in innovative ways, such as a tool to promote better well-being and mindfulness.

What's next for Zen Agent

Developing mobile apps on iOS and Android to expand access to the Agent to more users.
Creating a system where users can provide feedback and suggestions to improve the application.
Potential use of Hume's API to use a model trained on millions of datapoints for more accurate analysis.
Expand the scope of the application to include more emotions for the model to provide a wider range of recommendations.