DocScribe

Inspiration

The main goal and motivation behind our project is to increase the time that the doctor can spend with the patient. Doctors have to do a lot of busy work when it comes to reading test reports and describing the test results. Our Idea is to create a web app that does all of that busy work for the doctor.

What it does

Our web app takes in test reports, such as MRI scans. It generates summaries and notes about those test reports so that the doctor can read the notes and get a quick summary of the test results instead of going through that tedious task of reading and summarizing themselves. This can be done for a multitude of test results. For now, we have implemented blood, urine, and liver tests, but the project can be expanded to accommodate any number of tests. It takes in a PDF as input and gives text and a PDF download link as output.

How we built it

For the MRI scans specifically, we built a machine-learning model trained on a large dataset of MRI scans. The ML model uses MRI scan images as input and describes the patient's predicament and diagnosis. This can save the doctor an immense amount of time going through the photos themselves. Our ML model is over 99% accurate. We also tried to do ML for the ECG test but we failed at that. The other tests we implemented were read from and written using generative AI. We used custom prompt input, readings from the input pdf, and the GPT API to generate a summary report on medical test results.

Challenges we ran into

The biggest challenge we ran into was finding suitable datasets for our ML models. Finding large datasets with exactly the columns we wanted was basically impossible, so we had to constantly work with the data to make it compatible with our project.

Accomplishments that we're proud of

The ML model with over 99% accuracy for MRI brain scans is something that we are proud of.

What we learned

We learned how to set up and train machine-learning models, we learned exactly how to get a generative AI output from your program and we learned how to use AWS to store all of our data.