Inspiration
Every morning, Jane starts her day by doomscrolling on her phone, but she often feels unproductive and tired afterwards. She struggles to connect these feelings to her extended social media use and doesn't feel the urge to stop. Jane isn’t alone in these experiences. Did you know that more than 60% of social media users feel overstimulated, with 45% feeling tired? Overstimulation causes mental fatigue, impacting daily lives and reducing productivity.
But what solutions exist?
Social media detoxes are nearly impossible in this digital world. We need an approach that doesn't just block access to apps but educates and empowers users to manage their own consumption. Unlike competitors like Cold Turkey, which block phone use entirely, we want users to understand and control their usage, leading to a more sustainable and fulfilling relationship with social media. Our approach connects users to something tangible, which is how tired they feel after using their phone to help them reduce unfulfilling usage instead of blocking it. Phones and social media usage aren't necessarily bad if that usage fulfils their users, it's the unfulfilling usage we want to reduce.
One issue users encounter when trying to quit social media is that their goal to be more productive is not tangible and easy to ignore. Our approach connects users to something more concrete: how tired they feel after using their phone. This helps them reduce unfulfilling usage instead of simply blocking it.
What it does
Introducing Unclutter. Our full-stack PWA measures your cognitive load from social media use. It monitors users’ social media usage by processing the visual and auditory content users are viewing, and calculates an overall “cognitive load index” which measures how overwhelmed the users are.
This image demonstrates this concept simply:
This video showcases the flow of Unclutter.
How we built it
After researching the market for existing products, we experimented with our design by creating iterative mockup designs on Figma to ensure that our application was accessible yet user friendly.
The frontend Progressive Web App was created using React and Typescript to provide a user interface, and we used JavaScript with Express to create a backend service that will call a post request to extract the brightness, contrast, temperature, loudness, and bpm of our video.
We converted the video to frames for easier processing, then iterated through each pixel to convert it to RGB values. We calculated the average value for each frame and then determined the overall average for the video. Each measurement involved slightly different calculations, utilizing various formulas and libraries.
We utilized 'essentia.js' to process audio data into a usable format, which then allowed the extraction of loudness and BPM. Loudness was derived using the root mean square function, while BPM was extracted through RhythmExtractor, which estimates the BPM of an audio, adjusting based on deviations from a baseline.
Challenges we ran into
A huge problem was calculating the individual cognitive load index score. This consisted of four smaller subproblems:
Figuring out the optimal way to obtain the video/image such that the user's privacy is respected.
Processing the video/image into quantifiable data points (e.g. what number represents contrast in a way that gives us information whether if it's eye-relieving contrast, eye-straining contrast or overstimulating contrast).
Calculating a useful number and aggregate from these quantifiable data points (how could we derive such a formula).
How do we provide users with actionable tips that are specific enough where they will benefit them, but also general enough so that the open AI API we used to process this won't provide any startling output. We want to protect and help our users at the same time.
Another problem was deployment. We used many libraries which caused issues with memory (additional problem was storing video files on the deployed web service).
This is our current architecture:
Accomplishments that we're proud of
Overall, we are proud of our ability to create a high-quality, user-friendly, and effective application that allows people to understand and control their usage, leading to a more sustainable and fulfilling relationship with social media.
We have created a high fidelity Figma prototype to demonstrate the design and vision of the application. We used ReactJS and TypeScript for the frontend, allowing us to build a robust and scalable frontend for a seamless user experience. Our use of Unovis allows us to create visually appealing and informative graphs. We focused on creating a visually appealing and intuitive user interface, which reflects our commitment to delivering a high-quality product. We used shadcnUI for styling to ensure consistency and professionalism in the user interface. Our use of express allows us to create a fast and efficient backend that can communicate with the frontend seamlessly.
We successfully transformed video into frames and audio into workable vectors. By applying various formulas and libraries, we demonstrated our data processing expertise. This process also enhanced our knowledge of color analysis and audio processing.
Our application was deployed (frontend and backend) on onrender.
- Here is the backend link: https://unclutter.onrender.com
- Here is the frontend link: https://hagsonrender.com
What we learned
We learned the importance of staying informed about overstimulation resulting from prolonged social media use and how overstimulation can impact users. By researching current accessibility guidelines for video and image content, we gained insights into the characteristics that cause overstimulation and the reasons why they are overstimulating. This helped us to identify key areas of focus when considering how to calculate the overall cognitive load index and to develop a robust and effective system that provides actionable insights to help manage side effects from over-stimulation.
Additionally, we learned about the value of providing consumers with information and resources that can help them make informed purchasing decisions. By creating an app that makes it easy for consumers to stay informed about their social media usage, we are empowering them to take control of their own well-being. This can help to build trust and loyalty among consumers, and also promote greater transparency and accountability among social media companies.
What's next for Unclutter - minimise distraction, maximise life.
Moving forward, we will continue to build upon our existing work by refining and improving our calculations for the overall cognitive load index score. We could work on collecting additional data not only from user uploaded videos/screenshots but also while monitoring social media usage across the app, gathering insights from user interactions and content consumption, to improve the accuracy and effectiveness of our calculations.
Another possibility that we'd like to explore is broadening the scope of our calculations to account for additional metrics. We could also explore ways to integrate our app with other platforms or services to create a more seamless user experience.
Finally, we could also work with developing partnerships and collaborations with other organisations or stakeholders who share our goals and objectives. By working together with other groups, we could leverage their expertise, resources, and networks to achieve our goals more efficiently and effectively.
We'd also love to create an additional native mobile app as we currently only have PWAs, which can be installed by users on both their desktops and mobiles, and also be viewed on the web.
We were mostly doing post-processing as we couldn't code up the real-time processing version. This is our current architecture/flow:
However we'd like to change it to this (real-time processing and some more ML custom models to assist with calculating the index):
Built With
- color-temperature
- essentia.js
- express.js
- extract-colors
- multer
- node-wav
- openai
- shadcnui
- sharp
- typescript
- unovis
- vite
Log in or sign up for Devpost to join the conversation.