ProdWatch

Inspiration

Have you ever found yourself struggling to maintain focus and well-being during long work or study sessions? We sure have. The inspiration for ProdWatch stems from our desire to create a supportive and adaptive tool that enhances productivity.

What it does

This web app tracks your facial expressions and surroundings to keep you motivated, focused, and healthy. By giving you encouraging messages when you're unhappy, telling you to take breaks when you're tired, or reminding you to get back to work when you're distracted, ProdWatch is a dynamic and adaptable tool that is perfect for anyone who wants to up their productivity. After your study time, we will give you a focus report for you to help you find your best study time in the future.

How we built it

We trained and deployed a CNN in TensorFlow using a Kaggle dataset to identify facial features and expressions (such as fatigue/tiredness). We used an OpenCV pre-trained model for the eyetracker feature, which would identify whether the user was focused on the screen or was looking elsewhere. The web app was built using Flask, a lightweight and versatile web framework for Python. Flask allowed us to quickly develop a dynamic and interactive user interface, seamlessly integrating various components of our project, such as the video streaming / face tracking, as well as adding customizability, such as adjusting webcam tracking area, and allowing user to take screenshots.

Challenges we ran into

Issues included building an accurate CNN that would be able to detect facial expressions and emotions with high accuracy. The dataset we used had many ambiguous cases, for which we had to adapt our model.

None of our team members had much experience in front-end development, so we all had to learn the basics of building web apps and JavaScript. We also did not have any experience running Python files on the web, so we also had to learn the basics of Flask. Likewise, UI and UX design was a challenge, as none of us had training or much experience in this area as well, even though UI/UX was to be a crucial part of our project to work effectively.

Due to the nature of the private video recordings that our app would manage, it was a challenge to make sure that our private video recordings were secure and not accessible to others (outside of the user).

Accomplishments that we're proud of

Self-building and adapting our CNN architecture from about 50% accuracy to 88% accuracy. Hosting the data locally keeps sensitive data (such as recordings of users' activity) private.

What we learned

We learned skills in Machine Learning engineering, specifically how to deploy our own convolutional neural network model into an active web app, that could be easily used by students and workers alike.

We also learned about Flask and browser extensions in Chrome.

What's next for ProdWatch

More features, including audio recognition (hearing the user speak) and detecting tones, pauses, and other aspects of speech will be added; these patterns can be put together to give tips to the user regarding public speaking, especially in situations such as virtual interviews or speeches. Make the CNN model more robust by obtaining more accurate training data.

Another aspect that can be worked on is to aggregate the data of whether the user is focused or not, and the user can view their past history to track their improvement in concentration.