VoiceReach

Inspiration

Outreach workers shouldn't have to choose between helping someone and completing mandatory report logs. In San Francisco, field teams juggle dozens of interactions daily—only to spend hours transcribing them after the fact. With VoiceReach, we wanted to cut the clipboard out of the equation. But this wasn’t just about convenience—we set out to build a data ecosystem, one that empowers not only street workers but also public health officials, analysts, and nonprofits to make informed decisions in the fight against homelessness.

What it does

VoiceReach is a voice-first mobile transcription platform designed for street outreach teams. Workers can record spoken observations on the go. Our backend processes the audio through OpenAI Whisper for transcription and GPT-4o for structured field extraction—age, gender, height, behavioral observations, and more. These entries are tagged with location data, optionally include photos (with verbal consent), and can be searched in real-time via advanced filters.

VoiceReach is not just an app—it’s a platform. We built companion tools to close the data loop:

A data analytics dashboard for decision-makers to track trends, team performance, and safety hotspots (built using the amazing service and the hackathon sponsor Lovable!)
A live 3D map that overlays recorded interactions with real-time crime data and homeless shelter locations to support outreach planning and risk assessment (Also sketched out using Lovable)

Our system supports statistical research, cross-agency coordination, and operational monitoring—all grounded in the voice of the field.

Core Features

Real-time voice transcription and AI-powered field categorization
Manual data entry for noisy or sensitive scenarios
Consent-based photo capture with full audit trail
GPS tagging of encounters and address resolution
Danger scoring and override for worker safety
Live search and advanced filtering via Supabase queries
Duplicate detection using LLM-powered confidence scoring
Structured CSV exports for internal and external reporting
Fully functional mock mode for offline testing and demos

How we built it

Frontend: React Native + Expo for mobile UI
Backend: FastAPI server with OpenAI + Supabase integration
AI: Whisper for transcription, GPT-4o for data extraction
Database & Storage: Supabase PostgreSQL + Supabase Storage
Visualization Tools: Deck.gl + Mapbox for 3D map overlay
Dashboard: Lovable - React + Chart.js for insights into outreach patterns, encounter trends, and danger hotspots

Challenges we ran into

AI Field Inference: GPT-4o doesn’t always play nice with structured formats, so we built custom logic to normalize, validate, and fallback gracefully.
Photo Consent: Tracking who took the photo, when, and where—without introducing friction—required thoughtful UI/UX design.
Data Privacy: We built in legal warnings and hardcoded compliance rules to ensure sensitive categories like criminal status or mental health couldn’t be added by mistake.
Visualization Accuracy: Matching outreach points to crime data in a performant, visual way involved tuning spatial queries and coordinate normalization.

Accomplishments that we're proud of

Built a working transcription-to-search pipeline in under 30 hours
Developed a usable frontend with real-time AI feedback and mobile-friendly design
Created a data platform that could realistically be adopted by city agencies
Successfully layered encounter data on a 3D city map for spatial analysis

What we learned

How to design AI systems that support, but don’t overwhelm, real-world workflows
The importance of flexible data validation when working with natural language input
How mobile-first UX can elevate field reporting without requiring training
That civic tech needs to be both human-centered and analytics-capable to create systemic change

What's next for VoiceReach

Pilot deployments with SF-based outreach nonprofits
iOS release with full authentication and privacy controls
Dashboard v2 with predictive analytics, filtering by team, and encounter trends over time
Full offline support with encrypted local storage and background sync
Role-based access and data sharing protocols for partner agencies
Helping non-technical nonprofit and government officials use Lovable to build their own data analysis tools on our Data Ecosystem!