Trivia On-Demand

Note: audio did not attach to my YouTube video, explainer audio here: https://drive.google.com/file/d/1D9nBQA7Wx2QZdGtUMp-FkcOpCp5HMUkE/view?usp=sharing

Inspiration

I'm really into videogames and Twitch streaming, but noticed that during less exciting parts of the stream, viewers churn. Simultaneously, my cofounder, who is the youngest black filmmaker to be featured in the New York Times for his work on showcasing overlooked black success stories, was learning about how AI video tools were starting to impact the industry.

What it does

Trivia On-Demand is a feature that turns live video content (in this case, a Twitch stream) into trivia questions that award users points. The top 3 point-earning users can cash in their points on the 1st of each month for cool prizes like a Sony Playstation 5, Bose QuietComfort Headphones, and a Roku SmartTV. Anyone who wasn't top 3 gets their points rolled over and counted toward the next month's 1st.

How we built it

Since April 7th, Clark and I have been working on a product called moviechat that enhances your isolated Netflix (and other) streaming experience into a social and competitive trivia game. However, we had not explored any AI tools and had primarily been trying to build trivia manually. This doesn't scale.

During the hackathon, we explored tools like Whisper, OpenAI completion model, and Roboflow to turn live video into trivia content. We also had to modify previously built UI components to properly receive and display the new data that the APIs we were learning about were returning.

We relied on an open-source Whisper shell script to transcribe audio, Roboflow to tag video content, and took those components to incrementally turn live video into prompt inputs for OpenAI's completion API.

Challenges we ran into

Clark is in NYC producing films with Ken Burns right now, so this ended up being a solo hack. A lot of the issues came from trying to standardize OpenAI API outputs, with the solution that I wanted to use (guardrails ai) not being available in JavaScript, which is what Chrome Extension development requires.

Accomplishments that we're proud of

Accurate video/audio context tagging and transcription as well as extremely reliable trivia data object output using AI tools. Also successfully getting in contact with one of my personal role models, Broxah, and getting him onboard to actually use our product.

What we learned

We saw a lot of other teams working on cutting-edge technology. Unfortunately, neither of us are really skilled enough as technical founders, with the bulk of our engineering education coming from ChatGPT as a coding tutor in the past two months since I was part of a mass layoff at a previously high-growth tech startup. We learned that we had to rely on our creativity and niche insights far more than pure engineering talent, but it ended up making for a really cool use case as a result.

What's next for Trivia On-Demand

Working with our first pilot volunteer, Broxah, former Fnatic jungler and League of Legends World Championship Finalist. Broxah has 458k followers on Twitch and usually averages 20 to 30k concurrent viewers. If that goes well, we have the chance to work with Broxah's management, link, and their entire portfolio of streamers who comprise some of the biggest names in e-sports and Twitch content creation with a collective 17 million+ subscribers.

Built With

• chrome
• firebase
• javascript
• openai
• roboflow
• web
• whisper