Speech Assist

Bringing back the meaning to everyday speakers' presentations.

Comment

Inspiration

Find yourself falling asleep in class once again? Not even because you're not interested but because you just can't seem to stay connected with the lecturer? As students of engineering, computer science, and statistics, we have been through hundreds of unnervingly boring lectures and presentations by peers and teachers alike. Sometimes, it's us being the boring ones. The worst part is that even if the audience is interested in the subject matter, if the presenter isn't able to convey their ideas, often an issue because of unanimated facial tendencies, then all effectiveness is lost. Our idea stemmed from this common issue and we decided we wanted to help all people with their ability to connect with an audience.

Public speaking is the backbone of success whether it be in politics or a technical field. Conveying the subject of the speech in an interesting and engaging manner is key. We created Speech Assist to provide teachers, students, presenters in general a way to practice their public speaking through an application that provides instant feedback. The app is helpful education wise for improving public speaking.

What it does

SpeechAssist allows the user to record themselves practicing a speech. The video they create is then run through computer vision, speech recognition, and natural language processing algorithms to output organized results indicating where the speaker's text and facial expressions "matched" and where they did not.

How we built it

Used Microsoft Azure's Cognitive services to analyze speaker's facial expression along with the speech text's sentiment and provided analysis for how well the speech correlated with the speaker's facial presentation. The facial expression was tracked with OpenCV and the Face API; the text sentiment was analyzed by first converting speech to text and then using the Text Analytics API. We used Django to build our website along with a Python 3 backend.

Challenges we ran into

Incorporating a video recording through HTML and JavaScript presented a challenge. Also, combining our backend in Python with frontend through Django took a long time.

Accomplishments that we're proud of

The website works! We were able to successfully create a good user interface with recording ability, and were able to process the data to provide graphs so that the speaker knows how to improve.

What we learned

We learned more about how to use Django, and how to create an application that processes both audio and video.

What's next for Speech Assist

We want to expand speech assist to be more specific for certain audiences. Perhaps for students it will give them a grade on their oral presentation skills. Maybe for professors it will describe how intelligible all parts of their lesson were. It could assess the amount of eye contact maintained during the speech or incorporate body language into its analysis.

Share this project:

Updates