Inspiration
We’re all too familiar with the awkward situation of taking notes during someone’s interview. As the interviewer, it’s hard to focus on what the candidate is actually saying, and as the candidate, it feels disrespectful and intimidating to know notes are being taken on you. Both parties end up disconnected.
That’s why we created AudioNote, the all in one solution for text to speech notes.
What it does
To use AudioNote, you simply click the record button and forget about it. While you’re actively talking in the conversation, AudioNote automatically performs speech to text translation in real time. After the conversation is over, the recording stops on its own when it detects you are no longer speaking.
We also added the summarize feature. Using the trending OpenAI API, we use natural language processing models to summarize the conversation to bullet points.
Once the recording ends and we click the summarize button, we’re given a succinct description of what happened in the conversation. We can download the raw text or the summarized text using the download buttons in each box, and we can even save the file on the cloud for later access.
How we built it
AudioNote was built using a React.js frontend and a Node.js and express backend. Our files are stored using Firebase, and our summarization is done through making an API call to OpenAI.
Challenges we ran into
We found it difficult to use the OpenAI API as it's relatively new and not too well documented. We also tried making our own designs for the first time, which ended up being a lot harder than we expected.
Accomplishments that we're proud of
We're proud of the fact that we created a fully functioning application within the deadline. While there were obviously features we would've loved to add, we were able to meet most of our goals set within the timeframe. We found at this hackathon that being flexible was extremely important and while difficult at times, it helped us out in the long run. Being adaptable was one of our strongest points and helped us to continue making progress throughout the entire hackathon.
What we learned
We learned that while tools may sound exciting, development with them may not always be easy. This hackathon challenged us in utilizing small amounts of information and piece it together more than usual. Overall, this was a great learning experience to prepare us for the world of software development, as working in a team setting allowed us to build and learn from each other.
What's next for AudioNote
We would like to improve the speech recognition model to distinguish between multiple users speaking at once. This would be a huge step in production as separating text between others in conversation is significantly better than the current model of one centralized block.
Log in or sign up for Devpost to join the conversation.