MedSketch

Inspiration

MedSketch was inspired by the challenges patients face in understanding complex medical procedures. Many individuals experience anxiety and confusion when facing medical interventions due to complicated terminology and unclear explanations. We recognized that personalized, visual explanations that adapt to a patient's age, background, and needs could significantly improve comprehension and reduce anxiety.

One of our team members had surgery last year. During his initial consultation, the surgeon was struggling to visually describe what would be taking place during the surgery. The surgeon then proceeded to spend 5 to 10 minutes looking on google images for relevant images that would help him better explain the process. This wasted more time than necessary (doctors already consistently run late as it is), and there was still some confusion even after the explanation. MedSketch would have streamlined this process and given the surgeon the necessary tools to better explain the process, and allowed the patient to fully visualize and understand the upcoming procedure.

What it does

MedSketch is a patient-centered platform that transforms complex medical procedures into personalized step-by-step explanations with accompanying medical illustrations. The application:

• Takes patient information (age, sex, race) into account to customize explanations
  
• Breaks down medical procedures into clear, digestible steps
  
• Generates age-appropriate medical illustrations for each step
  
• Provides detailed descriptions that combine patient-friendly language with necessary medical terminology
  
• Delivers consistent, high-quality medical information in an accessible format
  

How we built it

MedSketch was built using a modern tech stack:

• Backend: FastAPI framework for high-performance API endpoints
  
• Database: MongoDB for flexible document storage of patient data, procedures, and steps
  
• AI Integration: OpenAI's GPT-4 for generating clear medical explanations and GPT-Image for creating accurate medical illustrations
  
• Cloud Storage: Cloudinary for managing and serving medical illustrations
  
• Architecture: Task-based engine for asynchronous processing of procedure generation requests
  
• Patient Personalization: Algorithms that adjust language complexity and visual style based on patient demographics

Challenges we ran into

• Creating medically accurate yet accessible content required carefully balancing simplicity with precision
  
• Ensuring illustrations were detailed enough for adults while remaining appropriate for younger patients
  
• Managing the asynchronous task pipeline to provide real-time updates on procedure generation progress
  
• Implementing content moderation to ensure all generated content remained safe and appropriate
  
• Optimizing the system to handle complex medical topics while maintaining reasonable response times
  

Accomplishments that we're proud of

• Developing an age-adaptive system that tailors both text and visuals based on patient characteristics
  
• Creating a robust moderation system that ensures all content is educational and appropriate
  
• Building an efficient asynchronous task engine that handles complex generation jobs
  
• Implementing a database architecture that efficiently stores and retrieves procedures with their associated steps
  
• Achieving high-quality medical illustrations that enhance understanding without causing anxiety

What we learned

• The importance of personalization in medical communication
  
• How to effectively prompt AI systems to generate consistent, structured medical content
  
• Techniques for balancing technical accuracy with accessibility in health education
  
• Strategies for implementing content moderation for sensitive medical topics
  
• Methods for optimizing complex AI-powered generation pipelines for production use

What's next for MedSketch

• Expanding the platform to support a wider range of medical procedures and conditions
  
• Adding interactive elements to the illustrations to enhance engagement and understanding
  
• Implementing user feedback mechanisms to continuously improve content quality
  
• Developing integration options for healthcare providers to seamlessly incorporate MedSketch into their patient education workflows
  
• Supporting multiple languages to make medical information accessible to non-English speakers
  
• Creating a mobile application for on-the-go access to personalized medical explanations

Built With

• cloudinary
• fastapi
• mongodb
• nextjs
• openai
• python
• react
• tailwind