🧠 Inspiration
Understanding medical prescriptions, lab reports, and doctor’s notes can be overwhelming for patients—especially when these documents are filled with technical jargon and unclear data. One of our team members has a family member with a chronic condition and saw firsthand how hard it was to interpret lab results or medication changes. That’s when we thought—why not use LLMs to make this simpler?
We wanted to create something that would bridge the gap between doctors and patients, empowering everyday users to understand their health information clearly, without needing a medical degree.
💡 What it does
WhiteCoatAI takes any medical document—whether it’s a prescription, lab report, or test summary—and transforms it into a simplified, visual, and interactive health overview.
Key Features:
- 📄 Document Upload: PDF, DOCX, or image-based reports
- 🧠 LLM-powered Summary: Understand complex prescriptions in plain English
- 📚 Glossary Generator: Quick definitions for tricky medical terms
- 📊 Test Result Visualizer: Charts comparing your values to normal reference ranges
- 💬 Chatbot Interface (optional): Ask questions like “Is my cholesterol high?”
🛠️ How we built it
We built the front-end using Streamlit to deliver a clean and user-friendly interface, and powered the backend entirely with the Gemini API to process and interpret medical documents with ease and accuracy.
🛠️ Tech Stack Overview
- 🔍 Document Parsing: Handled PDF, DOCX, and image files through Streamlit's file uploader for smooth and intuitive input handling.
- 🤖 LLM Processing: Used the Gemini API to extract, analyze, and simplify medical content from uploaded documents.
- 📊 Insight Display: Visualized extracted values and summaries directly within the Streamlit interface for easy user interpretation.
- 💬 Chatbot Interaction: Enabled interactive question-answering using Gemini-powered follow-ups with contextual awareness.
We worked in a collaborative repo, separating concerns between front-end, parsing, and LLM logic.
🚧 Challenges we ran into
- 🧾 Parsing varied file formats and extracting clean text from images was trickier than expected—OCR output needed heavy cleanup.
- 🧠 Medical terms often require context, and some LLMs hallucinated definitions, so we had to refine our prompt engineering.
- ⚙️ Managing multiple pipelines (upload → parse → analyze → visualize) while keeping the UI fast and responsive was a balancing act.
- 🔄 Some teammates were new to LangChain, so chaining prompts with memory and retrieval took a bit of learning.
🏆 Accomplishments that we're proud of
- Built an end-to-end working prototype with multi-format file support, LLM integration, and chart visualizations—within the hackathon timeframe!
- Designed a UI that’s actually usable for non-tech-savvy users
- Created a tool with real-world impact potential—especially for underserved or older patients
📚 What we learned
- 🧠 How to fine-tune and engineer prompts to work effectively with LLMs
- ⚙️ How to combine different AI tools into a cohesive pipeline (OCR, parsing, NLP, visualization)
- 🤝 The importance of cross-functional collaboration—splitting frontend/backend tasks really boosted our speed
- 🩺 Gained deeper understanding of health tech data and real-world constraints
🔮 What's next for WhiteCoatAI
We’re excited to take WhiteCoatAI beyond the demo. Our roadmap includes:
- ✅ Support for scanned handwritten prescriptions (better OCR + filtering)
- 🌐 Multilingual support for non-English speaking patients
- 🧠 AI-generated follow-up questions (e.g., “Should I ask my doctor about X?”)
- 📱 A mobile version for patients to upload their reports from anywhere
- 🔒 HIPAA-compliant storage for real-world use cases
Health information should be accessible, not anxiety-inducing—and we believe WhiteCoatAI is a step in that direction.
Log in or sign up for Devpost to join the conversation.