MarketMuse

Hero section
Upload
Receipt details display
User Action Plan

Inspiration

Staring at a shoebox of receipts at tax time, I realized: expense tracking fails because it's tedious. 60% of Americans struggle to manage finances, and the #1 reason is friction. Existing apps force manual entry or risky bank linking.

What it does

📸 Snap & Track Take a photo of any receipt AI extracts merchant, date, items, total in <5 seconds Auto-categorizes expenses (Food, Transport, Shopping, etc.)

🤖 Smart Insights "You spend 40% more on weekends" "On track to overspend $150 on dining this month" "Cut coffee by $30/month to hit your savings goal"

💬 Conversational Natural language: "I spent $50 on gas" Voice input supported Ask questions: "How much did I spend on groceries?"

🔒 Privacy-First No mandatory bank account linking Your data stays yours Local-first storage

How we built it

Stack:

Frontend: Next.js 14, TypeScript, Tailwind CSS, Framer Motion
AI: Anthropic Claude Sonnet 4.5 (Vision + Chat APIs)
Backend: Next.js API Routes, Edge Functions
Deployment: Vercel

Architecture: User uploads receipt → Claude Vision API → Structured JSON → Auto-categorization → Smart insights → Dashboard

Result: 95%+ accuracy across receipt formats

Mobile-First Design

Camera access with fallback to file upload
Touch-optimized UI (44px min tap targets)
Progressive Web App capabilities

Challenges we ran into

Receipt Variability Thermal paper, poor lighting, different layouts—real-world receipts are messy. Solution: Detailed prompts with examples. Added "If confidence < 0.8, return null" which improved reliability from 60% → 95%.
Real-Time Performance Vision API takes 3-5 seconds. Users expect instant feedback. Solution: Optimistic UI updates, engaging loading states, skeleton screens. Feels instant even when processing.
Structured Output First prompts returned narrative text. Needed reliable JSON. Solution: Explicit schema in prompt + examples + confidence scoring. Breakthrough: requesting null for uncertain fields instead of guessing.
Mobile Camera Integration iOS Safari, Android Chrome, desktop browsers—all handle camera differently. Solution: Progressive enhancement. File upload as fallback, optimize for camera capture.

Accomplishments that we're proud of

95%+ OCR accuracy on real-world receipts
<5 second processing from photo to expense
Zero-config - works without bank linking
Production-ready with proper error handling
Premium UI/UX with smooth 60fps animations
Complete MVP in 36 hours
Validated with 20+ real receipts during development

What we learned

Technical Skills:

AI-assisted development with Claude Code - Learned to effectively collaborate with AI for rapid prototyping, from UI generation to API integration
API integration patterns - Deepened understanding of working with modern AI APIs (Claude Vision, authentication flows)
Full-stack authentication - First time implementing proper user auth with Clerk, managing sessions and protected routes Database design & management - Gained hands-on experience with Supabase for structured data storage and real-time operations
Prompt engineering - Discovered that prompt structure is as critical as code architecture—learned to craft prompts that return reliable, structured JSON

Product & Process:

Power of mentorship - Cal Hacks sponsors and mentors were invaluable in identifying potential edge cases and refining our approach to receipt parsing accuracy
Iterative refinement - Through mentor feedback, we caught issues with date parsing and currency handling that would have broken the demo
Privacy is a feature - Learned that in fintech, being transparent about data practices isn't just good ethics—it's a competitive advantage
Ship fast, polish later - Better to have 5 working features than 15 half-baked ones
Real data reveals reality - Testing with actual receipts from our wallets exposed edge cases synthetic test data never would