EzMR

EzMR provides EMS teams with real-time, HIPAA- and HIE-compliant access to critical patient data, enabling life-saving decisions under immense pressure.

Comment

Inspiration

An EMT arrives to find an unconscious patient. The heart monitor flatlines, oxygen saturation drops. Is the patient diabetic? Allergic to a life-saving drug? Is their pacemaker safe to defibrillate? In emergencies where every second counts, paramedics are too often forced to act without critical context.

State-by-state Health Information Exchange (HIE) regulations and HIPAA compliance barriers limit Emergency Medical Services (EMS) access to medical records. Hospitals already hold this life-saving information, but regulatory complexity keeps it out of the hands of first responders when it matters most. Our solution aimed to address this - we wanted to create a way for EMS to access life-saving information about the patient without compromising their personal data.

What it does

Our product acts as a compliance filter, enabling EMS crews to access all patient information permitted under state-specific HIE regulations and HIPAA guidelines. We store medical records from commonly used EMR systems in a cleaner format, allowing seamless integration between emergency systems and patient data.

Feature Overview

Postgres-driven Secure Patient Record handling: Allows Doctors to view compilations of prior EMR data (including patient data, visualizations, medical history, and insurance information) in a Next.js and React-powered visualization.

Cloudinary-driven Secure File Upload: Also allows for secure upload and viewing of DICOM, XRAY, and Catscans, built using custom file parsing algorithms and Cloudinary cloud storage.

AI-Powered Clinical Search: Quickly surface critical patient details from records, making clinical consultations a lot more focused on patient interactions and not data.

Medical Diagram Visualization: Generated medical diagrams using React Three Fiber 3D modeling to highlight conditions and emergencies so that emergency responders can process them quickly.

AI-Powered Clinical Notes Handling: Uses CedarOS for high-quality, detailed Voice-to-Text recognition, and uses Google Gemini with the Maestra Agent Framework in order to help sort, process, and categorize unsorted notes.

GPS Prescription Assistance: Uses location data to help doctors determine relevant pharmacies and medication to prescribe to patients.

Secure Authentication and Permissions: Uses NextAuth for secure authentication and fine-grained management of patient data permissions, in perfect compliance with HIPAA.

How we built it

Full-Stack Architecture

EZMR is a (mostly) comprehensive Electronic Medical Records system built as a multi-tenant SaaS platform using Next.js with React and TypeScript. The frontend uses Tailwind CSS with a custom design system featuring geometric clip-path shapes, monospace typography, and seamless light/dark mode switching via next-themes. The UI leverages Radix UI components for accessibility and Framer Motion for smooth animations.

AI & 3D Medical Visualization

The system integrates Google Gemini 2.0 Flash for advanced medical data parsing, configured with optimized parameters to extract structured data from unstructured medical text including demographics, vitals, medications, allergies, and care plans. The 3D visualization engine uses Three.js with React Three Fiber and React Three Drei to render interactive anatomical models, combining transparent skeleton overlays with clickable block-model body parts. The system includes an animated blood cell background with theme-aware colors and parallax effects.

Database & Authentication

PostgreSQL serves as the primary database with Prisma ORM managing 25+ models covering the complete medical workflow (Patient, Allergies, Medications, Problems, ClinicalNotes, etc.). The schema supports multi-tenancy, role-based access control, and comprehensive audit logging. Authentication uses NextAuth.js with Resend for magic link authentication, a Prisma adaptor for session management, and includes break-glass emergency access with audit logging.

File Processing & Medical Imaging

Cloudinary integration via next-cloudinary handles medical document uploads, while DICOM support uses a custom DICOM parsing engine for parsing medical imaging files with advanced viewer features, including window/level adjustment, zoom controls, and multi-slice navigation. JSZip handles compressed medical files, and OCR capabilities extract text from medical documents and ID cards.

Real-time Data & API Architecture

The system implements real-time patient assessment with debounced API calls, optimistic UI updates, and comprehensive error handling. The REST API includes 15+ endpoints covering patient management, AI analysis, file processing, and administrative functions, all built with Next.js API routes and proper TypeScript typing. As a whole, we use Next.js, React, TypeScript, Tailwind CSS, Three.js, Prisma, PostgreSQL, NextAuth.js, Google Gemini AI, Cloudinary, DICOM parser, Radix UI, Framer Motion, and comprehensive medical terminology mapping with 800+ medical terms for injury analysis and body part identification.

Challenges we ran into

AI Notes: We wanted to make this as reliable as possible, supporting accurate transcription, even with background noise. We struggled to get the model to output formatted data that we could programmatically process, but with Maestra’s Structured Data Schemas, we could enforce a specific output format for the model. It also required a multi-step pipeline and several iterations of the system prompt to

3D Model of Human Body: We wanted to visually show doctors at a glance what areas of a patient’s body have had prior issues. At first, we thought we could do that with a simple diagram but we realized we needed a 3D model so doctors could see it from all angles. We tried modelling a person in React Three Fiber as a series of rectangular prisms but this alone didn’t provide enough visual clarity, while our second approach of detailed skeletal models didn’t have nearly the level of responsiveness and clarity that the prisms did. Eventually, we settled on a hybrid approach, and it worked fantastically.

Security: One of our biggest hurdles was data sharing with EMS under HIPAA’s minimum necessary rule. We first tried a blanket filter to hide “non-essential” fields, but it broke workflows we hadn’t considered. We then pivoted to flexibility, building fine-grained access controls so hospital admins can decide exactly which fields each role can view, and we redesigned the dashboard to render any combination of fields cleanly. Every data read is also audit-logged to create a clear paper trail.

Accomplishments that we're proud of

On a similar note to the major problems we faced, we’re really, really proud of three things that we managed to accomplish over the course of this hackathon.

Visual Clarity. We think that through all of our work, we’ve managed to create a UI that both provides very good functionality and visual impressiveness. The UI looks sleek and works fast, which is everything we could have ever asked for from it. The models, which we painstakingly made ourselves through react software, turned out really good, and our planned UI designs and color schemes ended up working out well.

AI Integration. We’re really proud of how effectively we used AI throughout this project. We think we’ve put AI to a good purpose: making tools that Doctors and EMS use that could help save lives far more efficiently, and making the lives of society's medical professionals easier. Our use of the Gemini API felt efficient, super functional, and very, very accurate to what we wanted it to do. Overall, despite all our problems and issues with it, we got it working very well in the end.

Authentication. As mentioned, probably one of the biggest issues with EMR and EMS is HIPAA, and we think that our authentication implementation with NextAuth has allowed us to create a very secure storage, authentication, and permissions system that maintains HIPAA policies while still creating a much more efficient and effective EMR managing system.

What's next for EzMR!

Technological expansion to support massive datasets, and integration across multiple healthcare providers

While there are many potential avenues to grow EzMR, we believe that our data would impact the most lives if integrated with healthcare firms. However, this requires the technological expansion of our tools to support massive datasets, an action that would generalize the use of this technology to include a wider range of healthcare professionals.

Corporate partnerships with healthcare firms to implement the technology

Tying into our previous point, the large-scale use of EzMR by healthcare firms would only be possible through corporate sponsorships. Doing this would allow us to reach even larger audiences, therefore allowing our tools to streamline data access in countless medical settings.

Global access, technology simplification for 3rd world implementation

We hope that the benefits of EzMR can spread to third-world countries, enabling healthcare providers everywhere to have access to the type of data that could potentially save their lives. Such global access would further aid more patients in these critical moments.

Skeleton Model Reference

All animation files for the skeleton generation were taken from https://anatomytool.org/open3dmodel, a Open Source and Free 3D Model of Human Anatomy. Created by Anatomists at Leiden University Medical Center, Leiden University Medical Center (Leiden, Netherlands) and a number of other highly reputable institutions in Europe.

Built With

Share this project:

Updates