SJ SNAP

Building Better San Jose, One Snap at a Time

Comment

Inspiration

San Jose residents often face confusing, time-consuming processes when trying to report public issues like trash, broken sidewalks, or graffiti. Existing systems are not mobile-friendly, require multiple steps, and allow unverified submissions, leading to slow responses.

We wanted to create a faster, smarter, mobile-first solution — using AI to verify reports and make civic participation accessible to everyone.

What It Does

SJ.Snap is a mobile-first AI web app where residents can:

  • Speak or type a quick description of a public issue.
  • Take or upload a photo.
  • Have AI verify that the description and image match.
  • AI will rewrite the description in more informative way, and generate exact spot (eg. overflowing trash-can in front of "San Jose" Sign, near black lamp post).
  • Instantly submit a verified report (with location) to the city.

It improves trust, speed, and accuracy in public issue reporting — helping make San Jose cleaner and safer.

How We Built It

  • Frontend:

    Built using Next.js (React framework) with Tailwind CSS for styling.

    Upload or take picture, store it in Supabase, type or voice input using Web Speech API (best supported on Google Chrome), and location fetching uses the OpenStreetMap API + Mapbox for map display.

  • Backend:

    Hosted on Vercel Serverless Functions.

    AI verification uses Hugging Face APIs — GEMMA-3 for image captioning and image-text matching.

  • Reporting:

    We used EmailJS to send professional email reports (including description, photo, and location) directly to the appropriate city service department (now is our email for demo) and the user (if they logged in).

  • Hosting:

    Everything is deployed on Vercel, ensuring fast load times and mobile optimization.

Challenges We Ran Into

  • Image and Text Mismatch:

    Fine-tuning the LLM verification to be strict enough without frustrating users was difficult.

  • Mobile Photo Capture:

    Integrating photo uploads smoothly across different devices (iOS vs Android) required careful testing using Supabase.

  • Instant Feedback:

    Implementing a user-friendly retry system (if verification fails) without losing user-entered information was tricky.

  • Voice-to-Text Accuracy:

    Background noise during voice input sometimes reduced description quality, so fallback typing needed to be seamless.

Accomplishments That We're Proud Of

  • Successfully built a fully mobile-first, AI-verified reporting system — no app download required.
  • Integrated voice, photo, and location into one fast and easy flow.
  • Reduced the chance of false reports through automatic verification.
  • Created a clean, professional HTML email report format ready for real-world city use.

What We Learned

  • AI + Civic Tech:

    How large language models (LLMs) can be used practically to verify public submissions.

  • Accessibility Matters:

    Making a tool work smoothly on all phones, even with low bandwidth, makes a huge difference for real users.

  • User Flow Importance:

    Designing fast retry loops and instant feedback dramatically improves user trust and engagement.

What's Next for SJ.Snap

  • Adding multi-language support to serve San Jose’s diverse community better.
  • Identify Duplicate reports to reduce resource usage
  • Building a reward system to encourage active citizen reporting (badges, civic points).
  • Integrating urgency tagging via AI to help prioritize serious issues.
  • Directly syncing with San Jose’s 311 system through open APIs for real-time issue tracking and updates.

Built With

Share this project:

Updates