Inspiration
(Anonymous member): During my time working as an assistant in a hospital, I witnessed firsthand a heartbreaking reality that healthcare professionals rarely talk about: the invisible patients.
I remember anonymous , a 67-year-old Spanish-speaking grandmother who sat confused in the waiting room, clutching a prescription she couldn't read. I saw Mr. anonymous, who nodded politely at his doctor's complex medical explanations, too embarrassed to admit he didn't understand a single word. I watched Sarah, a deaf patient, struggle to communicate her symptoms through a hastily scribbled note.
These weren't isolated incidents. Every single day, I saw patients—immigrants, elderly individuals, people with disabilities, non-native English speakers being left behind by a healthcare system that wasn't built for them. They would miss critical medication instructions because they were too complex, nod along to diagnoses they didn't comprehend, leave appointments more confused than when they arrived, and delay seeking care because communication felt impossible.
The breaking point came when an elderly patient had a stroke in our waiting room. The early warning signs were there, but the language barrier prevented timely intervention. That moment changed everything for me. I realized that in healthcare, communication isn't just about convenience—it's literally life or death.
CuraSyn+ was born from a simple but powerful question: What if technology could bridge the gap between medical expertise and patient understanding?
What it does
CuraSyn+ is a comprehensive healthcare platform designed to democratize medical care by breaking down communication barriers. It's not just an app—it's a lifeline for underserved communities.
Core Features
Intelligent Symptom Checker
- AI-powered symptom analysis using Google Gemini
- Multi-step questionnaire that adapts to user responses
- Provides preliminary insights and urgency levels
- Recommends appropriate specialists based on symptoms
Real-Time Disease Detection
- Seamlessly integrated facial asymmetry detection using OpenCV
- Runs in the background during symptom checks
- Immediate emergency routing if stroke indicators detected
- Could save lives by catching strokes in the critical "golden hour"
Doctor-Patient Communication Bridge (Transcribe)
- Real-time speech recognition captures the doctor's explanations
- Two modes: Simplify (converts medical jargon to plain language) or Translate (converts to patient's native language)
- Powered by Google Gemini for accurate medical context
- Text-to-speech output (Eleven Labs + browser fallback) ensures patients hear AND see the information
- Supports Spanish, Chinese, Hindi, Arabic, French, and Portuguese
Smart Doctor Matching
- AI-powered matching algorithm considers symptoms, location, and medical history
- Match the doctors/healthcare centers using recommendation systems based on symptoms reported
- Displays match scores for transparency
- Integrated hospital finder with Google Maps
- Real-time availability and appointment scheduling
Accessible Education Hub
- Post-appointment educational content tailored to diagnosis
- Multilingual support (6+ languages)
- Accessibility features: font size controls, high contrast mode, text-to-speech
- Both video and article formats for different learning preferences
- Curated content from authoritative medical sources (CDC, AHA, Mayo Clinic)
- Recommended by doctors
- Explanatory videos sent by doctor
Comprehensive Medical Records
- Detailed appointment summaries with doctor's video explanations
- Prescription management with clear instructions
- Lab results visualization
- Medication tracking with reminders
24/7 AI Health Assistant
- Context-aware chatbot powered by Google Gemini
- Understands patient's medical history
- Provides medication information and health guidance
- Available anytime, anywhere
How we built it
Architecture
Frontend (React.js + TypeScript)
├── Symptom Reporting Flow
├── Stroke Assessment (OpenCV integration)
├── Doctor Matching & Hospital Finder
├── Transcribe Page (Speech-to-Text-to-Speech)
├── Education Hub (Multilingual & Accessible)
├── Appointment Management
└── AI Chatbot Interface
Backend (Python Flask)
├── Authentication (Google OAuth)
├── Symptom Analysis API (Gemini integration)
├── Face Detection Service (OpenCV Haar Cascades)
├── Emergency Response System
├── Appointment Scheduling
└── Medical Records Management
Database (MongoDB Atlas)
├── User profiles with medical history
├── Symptom reports and assessments
├── Appointment records
├── Stroke incident tracking
└── Medication schedules
Technology Stack
AI/ML Layer:
- Google Gemini API: Natural language processing for symptom analysis, text simplification, and translation
- OpenCV (Haar Cascades): Facial asymmetry detection for stroke screening
- Web Speech API: Real-time speech recognition for doctor-patient communication
- Eleven Labs API: High-quality text-to-speech for natural voice output
Frontend:
- React.js 19 with TypeScript for type-safe medical data handling
- React Router for seamless navigation
- Axios for API communication
- Custom CSS with Harvard Crimson theme for professional medical UI
- Responsive design ensuring accessibility on all devices
Backend:
- Flask for rapid API development
- Flask-CORS for secure cross-origin requests
- Google OAuth 2.0 for secure authentication
- MongoDB PyMongo for database operations
- OpenCV-Python for computer vision tasks
Infrastructure:
- MongoDB Atlas for cloud database hosting
- Google Cloud Platform for OAuth and Maps API
- Environment-based configuration for security
Development Process
- Research Phase: Interviewed healthcare workers and patients to understand pain points
- Design Phase: Created user flows prioritizing accessibility and simplicity
- Core Development: Built symptom checker and doctor matching first
- ML Integration: Implemented stroke detection using OpenCV after TensorFlow.js proved unreliable
- Accessibility Focus: Developed Transcribe page and multilingual Education Hub
- Testing & Iteration: Extensive testing with real medical scenarios
- Polish: Refined UI/UX for professional medical environment
Challenges we ran into
1. ML Model Reliability Crisis
Problem: Initially implemented facial asymmetry detection using TensorFlow.js and MediaPipe FaceMesh. It worked perfectly in ideal conditions but completely failed in real-world scenarios—poor lighting, different skin tones, camera angles.
Solution: Pivoted to backend-based OpenCV Haar Cascades. This industry-standard approach provided consistent face detection across diverse conditions, real-time processing with visual feedback (bounding boxes), and higher accuracy for medical-grade assessments.
Learning: Sometimes the "cooler" technology isn't the right technology. Reliability over novelty in healthcare.
2. The API Quota Nightmare
Problem: Hit Google Gemini's free tier limit (250 requests/day) during testing. Eleven Labs quota exhausted even faster (5 credits).
Solution: Implemented graceful fallbacks (browser speech synthesis when Eleven Labs fails), optimized API calls using useCallback to prevent unnecessary requests, added request caching where appropriate, and learned to manage API quotas strategically.
Learning: Always plan for rate limits and have fallback strategies in production healthcare apps.
3. OAuth Configuration Hell
Problem: Google OAuth kept throwing 405 METHOD NOT ALLOWED and origin_mismatch errors. Frontend and backend were miscommunicating.
Solution: Properly configured authorized origins (localhost:3000 AND localhost:5000), fixed API base URL inconsistencies (REACT_APP_API_BASE_URL vs REACT_APP_API_URL), added proxy configuration in package.json, and set strict_slashes = False in Flask to prevent redirect issues.
Learning: Authentication is never as simple as the tutorials make it seem. Documentation is your best friend.
4. Real-Time Speech Recognition Complexity
Problem: Web Speech API would stop listening after silence, restart loops caused infinite re-renders, and interim results cluttered the UI.
Solution: Implemented continuous listening with proper onend handlers, used useCallback to prevent dependency hell, filtered out interim transcripts (only processing final results), and added comprehensive error handling for "no-speech" events.
Learning: Real-time features require careful state management and lifecycle handling.
5. Accessibility vs. Aesthetics Balance
Problem: Making the Education Hub both beautiful AND accessible (WCAG 2.1 Level AA compliant) was challenging.
Solution: Implemented dynamic font sizing without breaking layouts, created high-contrast mode that maintains brand identity, added keyboard navigation for all interactive elements, integrated text-to-speech without cluttering the UI, and tested with screen readers.
Learning: Accessibility shouldn't be an afterthought—it should drive design decisions.
6. Medical Data Complexity
Problem: Healthcare data is messy—varying symptom formats, complex medical histories, appointment types, emergency vs. routine flows.
Solution: Used MongoDB's flexible schema to accommodate diverse data structures, created clear data models with TypeScript interfaces, implemented proper error handling for missing/incomplete data, and added data validation at both frontend and backend.
Learning: Healthcare applications require robust data architecture from day one.
Accomplishments that we're proud of
- Life-Saving Potential: Built a real-time stroke detection system that could genuinely save lives by catching strokes in the critical "golden hour"
- Breaking Communication Barriers: Created a platform that serves patients who speak 6+ languages and have varying abilities
- Full-Stack Healthcare Solution: Integrated AI, computer vision, speech recognition, and real-time translation into one cohesive platform
- Accessibility First: Achieved WCAG 2.1 Level AA compliance while maintaining a beautiful, professional interface
- Real Problem, Real Solution: Built something that addresses actual pain points we witnessed in healthcare settings
- Technical Complexity: Successfully integrated 8+ APIs and services (Gemini, OpenCV, Web Speech API, Eleven Labs, Google OAuth, Google Maps, MongoDB Atlas) into a seamless user experience
What we learned
Technical Skills
- Advanced React patterns: useCallback, useEffect dependency management, performance optimization
- Computer vision: OpenCV for medical applications, real-time video processing
- AI integration: Prompt engineering for medical contexts, handling API limitations
- Accessibility: WCAG compliance, screen reader compatibility, multilingual support
- Full-stack architecture: Designing scalable healthcare APIs, secure authentication flows
Healthcare Insights
- Communication is medicine: Clear explanations improve patient outcomes as much as prescriptions
- One size doesn't fit all: Different patients need different communication styles (visual, audio, text)
- Trust through transparency: Showing AI confidence scores and match percentages builds patient trust
- Emergency detection matters: Early stroke detection can mean the difference between recovery and disability
Human-Centered Design
- Empathy drives innovation: The best features came from real patient stories, not technical possibilities
- Simplicity is sophistication: Complex medical systems need simple interfaces
- Accessibility is universal: Features built for disabled users benefit everyone
- Cultural sensitivity: Language translation isn't enough—cultural context matters in healthcare
What's next for CuraSyn+
Immediate Goals
- HIPAA Compliance: Implement end-to-end encryption, audit logs, and compliance frameworks
- Telemedicine Integration: Add video consultation capabilities directly in the platform
- Wearable Device Integration: Connect with Apple Health, Fitbit for continuous monitoring
- Expanded Language Support: Add 20+ languages including regional dialects
- Offline Mode: Critical features available without internet for rural areas
Long-Term Vision
- AI Diagnostic Assistant: Train custom models on medical datasets for more accurate preliminary diagnoses
- Prescription Auto-Fill: Integration with pharmacies for seamless medication delivery
- Family Account Management: Caregivers managing elderly or disabled family members
- Insurance Integration: Direct claims processing and coverage verification
- Community Health Programs: Partner with hospitals to provide free access to underserved communities
Impact Goals
- Reduce hospital readmission rates by improving post-appointment understanding
- Decrease emergency room visits through better symptom triage
- Improve medication adherence with clear, multilingual instructions
- Increase healthcare access for non-English speakers and disabled individuals
- Save lives through early stroke detection and emergency routing
The Bigger Picture
CuraSyn+ isn't just a hackathon project it's a movement toward healthcare equity. Every year, thousands of patients suffer preventable complications because they didn't understand their treatment plan. Countless others avoid seeking care because the system feels inaccessible.
We believe healthcare should be:
- Universal: Accessible regardless of language, disability, or technical literacy
- Transparent: Patients deserve to understand their own health
- Proactive: Technology should detect problems before they become crises
- Compassionate: Medical care should feel human, not institutional
CuraSyn+ is our answer to Maria, Mr. Chen, Sarah, and millions like them who deserve better. It's proof that with the right technology and the right heart, we can build a healthcare system that truly leaves no one behind.
Log in or sign up for Devpost to join the conversation.