Inspiration
Calling maintenance, making appointments, managing your schedule, creating new events. Elite lawyers and politicians have someone to do these things for them. But these tasks are aggravating for the average joe, too. Why not have your own personal AI secretary that you can direct at will?
What it does
PepperAssistant allows users to connect via Google to our web application and chat with PepperAssistant, allowing users to create new events in their calendar, make call-based appointments at doctor's offices, call maintenance with requests, and more!
Our built-in chat system allows users to accomplish all of these things simply by telling PepperAssistant what to do, and where to go. All the work Pepper does herself!
How we built it
Our application runs on a Next.js frontend and a FastApi backend hosted on Vercel and DigitalOcean respectively. The Docker container hosting our backend software uses Deepgram API, Twilio API, and Groq Chat using Llama3-8b-8192 to autonomously issue and interact in live phone calls, translating STT, parsing the text under the specified agenda, and TTS to speak back to the person on the other end of the line, all in real-time.
Challenges we ran into
We ran into significant challenges getting the voice pipeline to work, manipulating audio and fine tuning silence thresholds to create an optimal agent that can speak on the user's behalf.
Accomplishments that we're proud of
The accomplishment that we are most proud of is automating the process of scheduling appointments totally autonomousl.
What we learned
We learned a lot about Google Auth, how to use APIs, and building applications from scratch.
What's next for PepperAssistant
PepperAssistant wants to continue adding features to the AI secretary, like automatically adding appointments to the calendar, and preparing meeting notes ahead of time.
Log in or sign up for Devpost to join the conversation.