Empa | Devpost

Inspiration

The inspiration behind Empa stems from the growing challenges of virtual communication, where the nuances of non-verbal cues often get lost. As we transition into a more digital world, understanding and conveying emotions become paramount, and as such, we created a tool to support users in decoding emotions in a personalized and insightful manner.

Empa was conceived with three main purposes, each addressing crucial aspects of our evolving digital landscape:

Social Interaction Aid

Empa serves as a valuable aid for individuals navigating a myriad of social situations by effectively interpreting and responding to others' emotions with confidence and empathy.

Communication Disorders Support Tool

For those facing communication disorders, the challenges of expressing and understanding emotions can be especially daunting. Empa steps in as a supportive tool, offering real-time insights into the emotional context, empowering users to communicate more effectively.

Emotional Education

In an era where emotional intelligence plays a pivotal role in personal and professional success, Empa acts as an educational companion. It not only analyzes emotions but also provides personalized insights, contributing to the user's emotional growth and communication skills.

What it does

Radar chart showing emotion metrics
Transcribed audio to text, and recommended phrases for the emotion detected/expressed
Intuitive GUI and seamless frontend design

How we built it

Backend: Flask Frontend: Vanilla React

Landmarking

Data Collection:
We started by downloading the FER2013 dataset, which contains facial expression images. These images were used as input for our landmarking process.
Landmarking Process Using a Jupyter notebook (landmarking.ipynb), we employed Mediapipe to landmark 463 facial points on each image. The resulting data was then written to a CSV file (fer2013_landmarks_nopathsfixed.csv).

Model Training

Data Preparation:
x represents the emotion label for each image.
y represents the facial coordinates (landmarks) per image.
Model Architecture:
We designed and trained a custom model using TensorFlow and Sci-kit Learn. The model takes facial landmark coordinates as input and predicts the corresponding emotion label.
Training Process:
Input: Landmarked data from the CSV file (fer2013_landmarks_nopathsfixed.csv).
Output: Trained model capable of predicting emotions based on facial landmarks.
The training process involved optimizing the model's parameters to achieve accurate predictions.