Caloryx

Inspiration

Tracking calories and macros accurately is harder than it should be. Most existing tools rely on manual input, rough guesses for portion sizes, or barcode scanning that fails for homemade meals. As students balancing fitness goals, health, and busy schedules, we wanted a faster and more intuitive way to understand what we’re eating without adding friction to our day. That gap in usability and accessibility inspired us to build Caloryx.

What it does

Caloryx is an AI-powered nutrition assistant that analyzes images of food and returns estimated calorie and macronutrient information. Users simply upload a photo of their meal, and Caloryx processes the image to identify food items and generate a structured nutritional breakdown. The goal is to make food tracking quicker, simpler, and more approachable than traditional logging methods.

How we built it

We built Caloryx using a modular, API-driven architecture. The backend is powered by FastAPI, which handles image uploads, request validation, and clean JSON responses. We integrated a vision-language model to interpret food images and estimate nutritional values based on visual cues and contextual understanding. Image uploads are processed using multipart handling, and the system is designed to be extensible so future features like user profiles, meal history, and personalization can be added without major refactoring.

Challenges we ran into

One of the biggest challenges was dealing with ambiguity in food recognition. Meals often contain multiple ingredients, overlapping foods, or variations in portion size that are difficult to interpret from a single image. Translating visual recognition into realistic calorie and macro estimates also required careful prompt design and structured outputs. Additionally, ensuring backend stability when handling image uploads and parsing AI responses consistently took several iterations to get right.

Accomplishments that we're proud of

We’re proud of building a complete end-to-end AI pipeline that goes from image upload to meaningful nutritional analysis within the hackathon timeframe. We designed a clean and scalable backend API that can support future expansion and front-end integration. Achieving fast response times with structured, usable outputs while maintaining system reliability was a major milestone for the team.

What we learned

Through this project, we learned that vision-language models are extremely powerful, but the quality of results depends heavily on prompt structure and output validation. We also learned the importance of designing APIs early to allow rapid iteration. Most importantly, we realized that even seemingly simple consumer problems hide complex technical challenges, and shipping a functional MVP requires strong prioritization and focus.

What's next for Caloryx

Next, we plan to add user profiles with personalized calorie and macro targets, along with meal history and weekly nutrition summaries. We also want to integrate fitness and health goals such as cutting, bulking, or maintenance, and introduce a conversational AI coach for nutrition guidance. A mobile-first frontend is a key priority to make Caloryx practical for everyday use.