Co MD

Picture this: a doctor rushing across a crowded hospital corridor to their next appointment, already running five minutes late. They access the EHR and pull up 20+ different tabs: lab reports, notes, medications, all stacked across years of care. They have <1 minute to skim everything before the patient comes in. Which detail do they prioritize? Do they miss a critical lab value? Is there time to scan the latest treatment guidelines?

This is the daily reality for many doctors. They are expected to absorb oceans of information on impossible timelines, while burnout rates in medicine soar to all-time highs. In fact, studies show physicians spend two hours in the EHR for every one hour with patients, contributing to a professional exhaustion crisis that costs the U.S. healthcare system an estimated $4.6 billion annually in burnout-related turnover and reduced productivity.

That’s why we built Co MD, an answer to one of the most human problems in medicine: the crushing imbalance between data and time.What it does

Co MD is a voice-first AI agent that acts like a copilot for doctors. Instead of racing through 20+ EHR tabs, doctors can simply ask Co MD for a summary, and in seconds, it delivers a concise, clinically aware voice briefing of everything that matters: patient history, medications, labs, and even the latest treatment guidelines.

This transforms what was once an anxiety-filled scramble into a calm 60-second preparation ritual that saves time, reduces risk, and restores focus.

The impact goes far deeper: Financial: Better physician utilization = less time wasted in documentation, and reduced overhead from burnout-driven turnover Doctors’ Wellbeing: Reducing cognitive overload faced by doctors = reduce stress, reduce error rates, and help them focus on what drew them into medicine in the first place - healing patients. Quality of Care: Patients benefit directly from more present, better-briefed doctors. Missed information decreases, diagnosis quality improves, and health outcomes trend upward. Systemic Impact: Over the long term, reducing physician burnout and increasing efficiency improves retention, lowers staffing crisis risks, and translates into real savings across a healthcare system already stretched thin. In short, Co MD is more than a convenience, it’s a clinical force multiplier that helps doctors reclaim time, patients receive more attentive care, and health systems improve utilization without adding more strain.How we built it

We designed Co MD with the following components: Data integration: We used FHIR APIs to pull structured patient data from open-source EHR systems, and parsed unstructured clinician notes using an LLM. Contextual summarization: A large language model (LLM) was fine-tuned and prompted to produce concise summaries based on large volumes of EHR and patient data. Voice interface: Leveraged text-to-speech APIs to deliver natural-sounding, contextual summaries. Frontend demo: A React interface where doctors can interact with to get instant patient summaries. Challenges we ran into Balancing medical accuracy vs. brevity: Summarization models could accidentally omit critical clinical details or over-simplify. Doctors need speed and clarity, so designing summaries that were brief but still informational with the right level of context was a constant balancing act. Accomplishments that we're proud of Built and deployed a fully-functional voice agent within the hackathon timeline. Successfully merged EHR dummy data with other data sources to produce contextually relevant summaries. Created a simple, intuitive interface that could realistically integrate into a doctor’s workflow. What we learned Doctors don’t need more data, they need less friction. The problem is less about data gaps and more about human-centered design. Summarizing medical notes requires domain-specific prompting and guardrails to prevent omissions or hallucinations. Leveraging each other’s strengths and having clear designations of work scope to make the best use of our time, checking in frequently to make sure our progress is aligned What's next for Co MD

These are the features in the pipeline: Conversational agent: Go beyond a one-off interaction to enable a conversational user experience for doctors to ask follow-up questions based on the initial summary Calendar / Schedule integration: Ability to access doctor’s appointment schedules to automatically generate voice summaries based on the next appointment Post-appointment follow-ups: Enable doctors to record voice notes after each patient appointments to set action items and reminders User personalization: Allow each doctor to set summary preferences (e.g., focus on labs vs. guidelines), tailoring output to their workflow To scale Co MD: Conduct clinical validation: Collaborate with physicians to refine summary formats and voice delivery styles. Compliance: Support fully HIPAA-compliant infrastructure Pilots and partnerships: Conduct pilots with physicians and explore partnerships with EHR vendors.

Built With

• elevenlabs
• fastapi
• frameworks
• language
• openai
• phenoml
• platform
• python
• react
• render
• typescript