Lyka

Enabling a culture first driven AI multi modal system with vector databases.

Comment

TL;DR — (August 2024 Project)

What it is: A culturally-aware AI-assisted telemedicine interface that augments doctor–patient communication by encoding cultural context into medical recommendations without replacing professional care.

Core Technical Idea

The system introduces a cultural-context recommendation layer on top of telemedicine workflows using multimodal inputs + vector-based reasoning.

Instead of treating culture as free-text notes (which get lost in translation), it is:

  • Structured
  • Weighted
  • Auditable
  • Adaptively learned

System Architecture (Pipeline)

1. Data Ingestion

  • Multimodal inputs:

    • Images (e.g., affected body area, medication)
    • Audio → text (via Whisper for translation)
    • User-entered logs (symptoms, behaviors)
    • Cultural self-identifiers (traditions, remedies, exercises)

2. Context Recognition

  • NLP + embedding models convert inputs into vectors
  • Cultural expressions are embedded alongside medical data
  • Avoids brittle 1-to-1 translation by preserving semantic meaning

3. Adaptive Weighted Inputs

  • Each data source gets a dynamic weight, e.g.:

    • Clinical risk > cultural comfort (when necessary)
    • Cultural remedy > generic advice (when safe)

  • Weights adjust over time using feedback loops

4. Recommendation Engine

  • Vector database similarity search:

    • Finds relevant cultural remedies, exercises, or warnings
    • Cross-referenced against medical safety constraints

  • Outputs dual-track recommendations:

    • Culturally comforting actions
    • Clinically validated guidance

5. Feedback Loop

  • Patient + provider feedback updates weights

  • Helps distinguish:

    • Helpful traditions
    • Neutral practices
    • Harmful misinformation

Key Technical Contributions

  • Vectorized culture modeling (instead of text-only notes)
  • Multimodal fusion (image + audio + text)
  • Adaptive weighting system for trust + safety
  • Explainable recommendations for clinicians
  • Compliance-aware infrastructure (Indian data residency)

Tech Stack Choices

  • Vector DB: Pinecone → Xcalar (India-based servers)
  • Cloud: AWS (India regions)
  • Speech / Translation: Whisper (fine-tuned)
  • Vision: Image recognition for symptoms + meds
  • Compliance: Designed for Indian healthcare + HIPAA-style constraints

Importance of Culture as a Normative Weight

  • Culture is treated as structured signal, not noise

  • Avoids overfitting to stereotypes by:

    • User-specific profiling
    • Feedback-driven weight correction

  • Bridges human trust gaps without automating diagnosis

One-Liner

A multimodal, vector-based recommendation system that encodes cultural context as weighted signal layers in telemedicine, improving trust, interpretability, and safety without replacing clinicians.

Built With

Share this project:

Updates