SpeechDruid

Inspiration

A love for Dungeons and Dragons inspired based on the need for a tool to aid the dungeon master's story telling.

What it does

It converts speech to text using a javascript Speech Recognition API. Then, using the compromise natural language processing (NLP) API, we parsed meaning from the recorded words. Finally, we placed the words we had in our image library onto the webpage, creating a scene. On the side, the words previously recorded scroll by in real time.

How we built it

It gets served through a node express server, after being rendered using handlebars. On the client side it starts by listening to the user's speech. Then it converts that speech to text and passing it the NLP part. In the NLP part, we classify words into their parts of speech. After that, we compare the words with our dictionary. If we find a match, we then display it on the screen.

Challenges we ran into

We were unable to use the pixijs library as we originally wanted to, as the learning curve was too high for our limited time, and none of our team had experience with it before.

Accomplishments that we're proud of

It generates two scenes with different artifacts, a cave and a forest.

What we learned

The English language is very complex!

What's next for SpeechDruid

Hopefully we can add more scenes and items to the library. Being able to recognize the same object with synonyms of a word would be an interesting challenge as well. Dynamically placing elements and altering their color based on the scene around them.