Inspiration
In aviation, black boxes are indispensable for understanding what happens during critical events. We realized the medical field lacks an equivalent system that can transparently capture and analyze procedural data. Inspired by this gap, we set out to build a “black box” for medical and dental procedures, one that enhances accountability, supports quality assurance, and provides objective medico-legal documentation.
What it does
MedBox AI uses computer vision to automatically record, segment, and analyze procedure videos in real time. It detects key surgical or dental steps, identifies tool usage, and flags deviations from standard protocols. The output is a secure, timestamped record that can serve clinicians, insurers, and patients as a reliable medico-legal reference.
How we built it
We integrated:
- OpenCV Computer vision models (for step segmentation and tool detection)
- Chroma for storing and retrieval of procedure and history data
- A Next.js + Shadcn/UI dashboard for clinicians to review annotated procedures and audit logs.
Our backend manages inference and report generation, while the front-end visualizes the timeline of detected events and deviations.
Challenges we ran into
- Obtaining clean, annotated datasets for surgical and dental tool detection.
- Designing a user interface intuitive enough for clinicians but detailed enough for legal review.
Accomplishments that we're proud of
- Built a fully working prototype that segments and labels procedural steps.
- Designed a dashboard that makes procedural review visual, structured, and auditable.
- Established a framework that could support medico-legal compliance and standardization in future deployments.
What we learned
- The complexity of medical workflows requires both domain knowledge and robust ML design.
- Effective medico-legal documentation is as much about data integrity as it is about AI accuracy.
What's next for MedBox AI
- Expand training datasets across surgical and dental specialties.
- Integrate with hospital EMR/EHR systems for automatic report linking.
- Partner with hospitals and insurers to pilot real-world trials focused on medico-legal use cases.
Log in or sign up for Devpost to join the conversation.