EMS-Speech

Inspiration

As an active part of the EMS community, we noticed that writing Patient Care Reports (PCRs) is one of the most stressful parts of the EMS process. Given that a majority are handwritten or typed, we determined that the process can be automated with a speech assistant. According to EMS World, Paramedics have an average career span of five years. Caused by long hours in addition to the physical and emotional demands, many succumb to burnout which decreases productivity, social engagement and physical health. Adding to this burden, tasks outside the scope of first response contribute to cumulative stress adding to EMT dissatisfaction. Our project strives to reduce this stress by deploying an AI assistant that can facilitate writing PCRs giving EMTs one less thing to worry about.

What it does

The app listens for a structured verbal input from a user and parse through the message. It will try to determine relevant information regarding that input, and autocomplete a given EMS form in real time.

How I built it

This app was build in Flask. We utilized a speech recognition library that leveraged Google's Natural Language processor to understand command inputs we gave our app. Those commands in turn were relayed to Wit.ai, another natural language processor to determine the context and intent of commands given to our application. Integration of Google's API and Wit.ai was done in Python.

Challenges I Ran Into

Integrating the Wit.ai and Google Speech Recognition APIs into a high-performance speech parser proved difficult. Because Google's speech-to-text API requires sending and receiving information via the internet, there was inconsistency in the speed at which our application runs. In addition, the audio quality of our available microphones compounded the issue inaccurate speech transcriptions, whose error would propagate through the Wit.ai API and result in inaccurate readings.

While analyzing and parsing user inputs, we realized that Wit.ai was difficult to train for our application. It was a blessing and a curse that Wit.ai takes very little training data: no training data was readily available and Wit.ai only requires some example sentences. We had to apply heavy constraints on the structure of sentences fed through the application; admittedly, this is far from the flexible speech-to-form application that we were hoping for.

We also had initially planned on using a full stack development suite with React as the front-end interface and Flask as the back-end. However, React was difficult to set up with flask and did not update synchronously with the flask app. As a result, we ended up scrapping much of our React application in favor for a simpler flask application.

Accomplishments that I'm proud of

We were able to build a working application within a little more than 24 hours. We interfaced multiple APIs in--what we believe--is a unique way to solve a relevant medical issue. Certain fields of a PCR form now can be completed quickly and with ease given that medical personnel can articulate those fields.

What I learned

Given the differing background of our team, each of learned different valuable skills. Some of us learned how to use Git efficiently, make post requests, train a learning model, use Flask, and work with various APIs. Overall, we learned how to manage application development and form stronger bonds as a team.

What's next for EMS-Speech

There is much to be refined in our EMS-Speech application. Developing a better natural language processing model for text parsing would greatly improve usability allow more flexible, flowing inputs that are comfortable for medical personnel to use. In addition, making EMS-Speech usable in the field during dispatch will further improve ease-of-use and the speed at which EMS services can record events. Another strong avenue for development involves integrating this application with existing electronic medical record systems, giving EMS services a streamlined way to input and submit information to necessary persons and agencies.