LionScribe!

Inspiration

Recorded lectures can be a very helpful resoruce. BUt it can be frustrating if you are only confused about a single topic and have to parse through a 50 minute lecture. Therefore we decided to use Watson to parse the lecture and pull key information at the correct time.

What it does

It parses through a video lecture for relevant keyword information and indexes those topics based on when they were stated in the video. This can be very beneficial for students as time is a very valuable resource and this application can same much time. It can also reinforce the topics for the students.

How we built it

We built a webapp with a flask backend and a frontend in html/css/javascript. We use a youtube to audio conversion to get the audio file that we then pass to Watson. The watson api queries are done through asynchronous calls on the backend. We then transform the JSON given to us by Watson into a condensed format and send the transcription to the frontend so we can do lightning fast jumps in timestamp for our video player whenever the user types in a keyword that they want.

Challenges we ran into

We started with the asp.net framework but we had trouble calling python based audio transcriber. We then had to rewrite our whole backend in flask, which we weren't very familiar with. It was also very challenging to get the audio of youtube videos because youtube makes it challenging to get any information out of their videos.

Accomplishments that we're proud of

The core of the application, which time indexes the keywords of a lecture and queries the transcription for the timestamps of a word works incredibly well and is very fast and lightweight.

What we learned

We learned to pay careful attention to language compatibility when building an application because that would have saved us a lot of time. We also learned the importance of agreeing on a framework and design for our product is paramount to success because if we didn't stop all working on small components and agree on the bigger picture then we wouldn't have been able to put anything together.

What's next for LionScribe

Getting students and teachers to adopt the system is really important. The whole purpose of the application is built around students using lecture videos that are uploaded by teachers and without adoption the platform itself is not very strong.