SweetFriend

• 

  Dashboard

👬 Team

Dennis Miczek ([email protected])

Hemanth Kapa ([email protected])

Shahir Ahmed ([email protected])

Ethan Wong ([email protected])

🩺 Tracks

• Best Healthcare Hack
• Best Use of Computer Vision
• Super Fast AI Track
• Best Next-Gen LLM-Powered Application
• Best Use of MongoDB Atlas
• Best Beginner Hack

🧠 Inspiration

Our project was inspired by a personal connection to diabetes. One of our team members lives with type-1 diabetes and experiences firsthand the challenges of managing it, especially right after diagnosis. The complexities of carb counting, predicting fluctuations from food and physical activity, and determining the right insulin doses were overwhelming.

For many diabetics, especially those newly diagnosed, these daily tasks can feel like a risky guessing game. Not having an idea of estimates and misjudging carb intake or insulin needs can lead to serious health consequences - too little insulin may cause hyperglycaemia, while too much can result in hypoglycaemia. Add in the variable effects of physical activity on blood sugar levels, and the management becomes even trickier. This gap in easy-to-use, accurate tools for diabetes management became clear to us. We saw an opportunity to create something that could bridge this knowledge gap and empower diabetics to make more informed decisions about their health. Our goal is to develop a solution that simplifies these complex calculations and predictions, potentially improving quality of life for people managing diabetes.

⚕️What it does

Our webapp integrates with Dexcom continuous glucose monitors to display real-time blood sugar data in an easy-to-read graph. Users can add custom events to this timeline, marking meals and physical activities with distinct icons. This visual representation helps users understand how specific events impact their glucose levels over time.

We've incorporated a food recognition feature where users can upload meal photos. Our vision model analyzes these images, breaking down the carbohydrate content to assist with insulin dosing decisions. To enhance diabetes management, we've implemented an AI-powered suggestion system. This feature provides personalized tips on glucose control based on the user's data. Additionally, our AI chatbot allows users to discuss thier glucose trends over weeks, offering insights on potential improvements and identifying which events have the most significant impact on their levels. A crucial component of our system is the notification and alert feature. It sends timely communications about glucose levels and gives an AI suggestion, ensuring users stay informed and can take prompt action when needed.

⚒️ How we built it

We built Sweet Friend using a combination of modern web technologies and AI services to provide a seamless and powerful experience for users managing diabetes.

Our React frontend allows users to interact with their glucose data through an intuitive, responsive interface, displaying real-time graphs of glucose levels and easy-to-use logging features for meals and exercises.

The Flask backend manages our API, handling data from Dexcom, user inputs, and AI-powered insights, while MongoDB stores user data, including glucose logs, meal entries, and exercise records.

To integrate with continuous glucose monitors, we used the Dexcom API to pull glucose data in real time. For meal logging, Tune Studio provides vision model inference through GPT-4o, enabling users to upload meal photos and receive accurate carb estimations. The Cerebras AI platform allows for ultra-fast chatbot responses, delivering personalized advice based on user data. We also integrated Twilio to send timely notifications, keeping users informed about critical health metrics.

🚧 Challenges we ran into

The most difficult part of our project was bringing together all the APIs and ensuring seamless integration. For example, to bring AI-generated advice to users, we needed to first gather data from the Dexcom API, saving it to MongoDB for repeated use, before formatting it and sending it to the Cerebras API, all while maintaining conversation history and memory on the front-end.

For recognizing carbohydrate content from images, we needed to give the model some space to think and break down what it was seeing to make better estimates. This, and the requirement for structured outputs to add to log, required some work with the Tune Studio API. Once it was integrated with the frontend, it was the most rewarding feature to test.

🏆 Accomplishments that we're proud of

We are proud of integrating all of the APIs, frontend, and backend by the end of the hackathon. We know that our product will help diabetes patients by streamlining the diabetes management process. We completed all major features we were planning on implementing, including meal recognition, AI-based insights, and comprehensive food/activity logging.

🤓 What we learned

We learned how to use React and Flask to its fullest potential, utilizing libraries like chart.js, react-markdown, and date-fns. We learned how to integrate the frontend with backend, which required cooperation between all developers on our team. In cooperating on this project, we also learned a lot about the git workflow and how to resolve merge conflicts. Finally, implementation of this project required a lot of work with APIs and interaction between the backend and the MongoDB database.

🔮 What's next for SweetFriend

Looking ahead we want to expand SweetFriend's capabilities and reach. Our future ideas include developing a mobile app to provide on-the-go access and real-time monitoring. We plan to broaden our compatibility beyond Dexcom, integrating with wide variety of glucose sensors to make our tool accessible for more users. A key focus will be implementing a remote patient monitoring system, allowing healthcare providers to track patients' glucose trends and influential activities more effectively. Additionally, we're working on an AI-powered feature to generate personalized diet plans. This system will take into account user preferences and historical glucose data to suggest meals that help maintain stable blood sugar levels.

Built With

• cerebras
• flask
• python
• tune
• vercel