Inspiration
Telemedicine Hub doctors need accurate and complete patient clinical history to improve evidence-based diagnosis and decrease the need for repeated history taking. Our solution creates an accessible and efficient method for patients and medical professionals to communicate despite language barriers.
What it does
Auxilium facilitates efficient and effective communication between patients and medical professionals through summarizing and suggesting diagnosis, thus reducing time spent during in-person consultations.
During Call: Patients give a verbal description of their current symptoms, after which Auxilium asks follow-up questions. At the end, Auxilium verifies the summary with the patient to ensure accuracy of content.
After Call: Based on transcribed call, Auxilium summarizes the conversation and matches possible diagnosis from its database.
Doctor Diagnosis: Doctors are able to view a summary (and original transcript) of patient’s symptoms. Alongside suggested diagnosis, Auxilium also predicts triage of each case, allowing urgent cases to get immediate help.
Aftercare: Alongside the email/SMS of diagnosis, doctors can also prompt patients to book follow-up appointments depending on the ailment.
How we built it
We created our patient analyzation algorithm first by importing in a csv from kaggle which contains list of symptoms and a list of diagnonses associated with the symptoms. We then took these symptoms and AI generated a list of one word synonyms to the symptoms which can be associated with them or possibly said by our patient to mean the same thing. Also to increase the flexibility of our algorithm, we also lookup all the words in the word family to avoid not diagnosing because of a different tense of a word is used. We cross compare the symptoms that are associated with the patient and give possible diagnoses.
C A L L P L EA SE DI A N A P U T T H HI SI S I N N N N N NNN
The follow up questions are generated by Gemini AI by including the patient’s symptoms in the prompt. We then reprompt the user and re-process the transcript of their answer to both follow-up questions.
After the call is complete, the call will create a pdf which will be sent to a doctor for ease of viewing. The website would then store each patient’s pdf as well as whether or not their appointment is resolved or not.
Challenges we ran into
Challenges we ran into are fine tuning the responses and synonyms of the symptoms in order to generate this bot which makes an accurate description of symptoms and possible diagnoses’.
Accomplishments that we're proud of
Some accomplishments we are proud of is our python algorithm to analyze the transcription of our patient’s dialogue with Auxilium. This algorithm is capable of not only detecting keywords which correspond with symptoms, also synonyms which have been generated by AI and different tenses of every word (i.e. fatigue and fatigued). Additionally we are proud of porting our application to phone calls via Twilio which utilizes various flask endpoints to run the call function. The conclusion of the call will lead to the generation of a summary pdf sent to your healthcare provider.
What we learned
Some of our team members came into the Hackathon without much experience with python or react, but after working towards creating a full stack for the doctors who utilize our app, have more strongly developed their skills in both. Additionally we have used new technologies we haven’t before such as Twilio, NLKT and Speech Recognition.
What's next for Auxilium
Multilingual: Increasing number of languages, especially niche and minority languages, can ensure that more people get the medical attention they require.
Video/Image Integration: Incoporating real time image and video analysis and recognition allows for more accurate identification of symptoms.
Real World Application Example: In countries such as Africa, monkeypox is prevalent but healthcare services may be scarce due to language, financial and geographical barriers. Combined with video analytics and AI-assisted detection of lymph nodes, diagnosis can be carried out faster and more conveniently, allowing more patients to get the medical attention they require.
Log in or sign up for Devpost to join the conversation.