main
Lens Studio Run
NutriLens Webapp

NutriLens AR: AI-Powered Nutrition at a Glance

Inspiration

Over 93% of people worldwide have little awareness of what they're truly putting into their bodies—and how it impacts their health. Existing tracking apps are tedious, manual, and often ignored.
We asked: What if tracking nutrition was effortless?

NutriLens AR was born from that question—a hands-free, real-time nutritional assistant powered by Snap Spectacles, WebSockets, AI, and a live web feed.

What NutriLens Does

NutriLens AR transforms how people interact with food. Just say:

"I am eating"

…and the lens instantly starts analyzing the scene, identifying food items in front of you, and returning:

🔢 Calories
🥩 Protein
🥑 Fat
🍬 Carbs & Sugar
🍊 Vitamins & Minerals

The results are:

👓 Displayed via augmented reality inside Snap Spectacles
🗣️ Read aloud via built-in text-to-speech
🌐 Sent in real-time to a web-based dashboard

The companion app & website:

Store your meal history
Display a nutritional breakdown
Offer personalized insights based on your BMI, allergies, or fitness goals

How We Built It

Lens Studio + Snap Spectacles: AR lens interface, voice input, and hand tracking
RemoteServiceModule + Heroku WebSocket Server: Real-time messaging and data flow
GPT-4 + OpenAI APIs: Smart nutrition understanding from images and text
MongoDB Atlas: Stores historical food logs and nutrient data
** Web App**: Displays user history, trends, and suggestions

WebSocket Magic

Our Heroku-hosted WebSocket server acts as the glue:

Snap Spectacles send images + voice-triggered prompts
AI processes the input
Server responds with personalized health info
Data is stored live to MongoDB
Web dashboard and iOS app show the log in real-time

Challenges

Connecting Spectacles to Heroku via WebSocket
Maintaining low-latency streaming and consistent TTS feedback
Handling AR + AI + Real-time Data simultaneously
Dealing with Heroku idle timeouts and MongoDB connection retries

Accomplishments

Fully functional real-time calorie and macro nutrient scanner
Built a seamless web + AR + AI pipeline
Enabled live food logging + data sync to MongoDB and web dashboard
Scalable backend architecture with low compute footprint

What We Learned

Deep Spectacles integration with RemoteServiceModule
Efficient WebSocket architecture for bi-directional AR communication
How to manage async AI calls and real-time feedback
Deployment best practices on Heroku + MongoDB Atlas

What’s Next

💬 Add GPT-powered conversational voice support
🧠 Train with verified nutritional databases
📷 Add photo-based food logging for mobile users
🏥 Partner with our own fitness apps
🌍 Launch a your food analysis and calorie count with food recognition across cultures