TalkToText

Frontend user interface
Email sent by the application

Inspiration

The audio transcription service was developed with the aim of providing valuable insights from lengthy audio content, catering to individuals with ADHD or limited time. This service not only delivers accurate transcriptions but also includes summaries, key points, sentiment analysis, and action items, making information accessible and manageable. It serves as a valuable resource for gaining meaningful insights quickly, benefiting individuals with tight schedules or attention-related challenges like ADHD, as well as those who are deaf or hard of hearing.

What it does

Our audio transcriber/summarizer, powered by advanced AI technology, offers the following comprehensive functionalities:

Transcribes Audio: It employs Whisper AI to convert spoken words from audio recordings into accurate written text. This not only makes content accessible in a readable format but also includes sentiment analysis, providing valuable insights into the emotional tone of the audio.
Summarizes Audio: Utilizing Chat-GPT 3.5, it condenses lengthy audio content into concise, easy-to-digest summaries, saving time and effort. These summaries also include key points and action items for quick reference, making it an invaluable tool for efficient information consumption.
User Validation: To ensure accurate transcription and summarization, our system employs a user validation process. A validation email is sent to the provided email address, ensuring that the correct email is inputted. Once the email is validated, the transcription process begins.
Versatile Use: This tool is suitable for transcribing and summarizing various types of audio content, such as lectures, meetings, interviews, and more. Its versatility enhances the accessibility of diverse spoken material, catering to a wide range of user needs.
Multilingual Support: It facilitates transcription and summarization in multiple languages, accommodating a global user base and a wide variety of content types.

Once transcribed and summarized, the results are conveniently sent via email for easy access and sharing, providing users with valuable insights and accessible content.

How we built it

Our app was developed using industry-standard methods and tools. We used Flask in Python for the backend, Microsoft Azure Web App for Containers for deployment, and Vercel for the frontend with React. For our development environment, we relied on IntelliJ as our IDE.

For file storage, we leveraged MongoDB GridFS to securely hold user files. When a user is validated, we retrieve their file from MongoDB and proceed with transcription using WhisperGPT. Once transcribed, we send the resulting content through SendGrid for email delivery.

Additionally, we implemented Continuous Integration and Continuous Deployment (CI/CD) for both the backend and frontend, ensuring seamless updates and continuous enhancement of the app. This streamlined approach led to a dependable and user-friendly application.

Challenges we ran into

During the development of our hackathon project, we encountered some significant challenges, particularly when it came to deploying our backend. Our initial choice of AWS proved problematic as it did not offer an SSL certificate, which is essential for securing web applications and user data. We explored PythonAnywhere as an alternative, but it lacked support for multithreading, a critical component of our project. Ultimately, we found the solution we needed with Azure, which not only supported multithreading but also provided a straightforward process for installing essential libraries like ffmpeg and ffprobe. We successfully deployed our app on Azure using Docker and Azure's web app for containers service, ultimately overcoming this deployment hurdle.

Accomplishments that we're proud of

We're proud of the seamless communication between our Azure-backed backend and Vercel-hosted frontend, ensuring a reliable user experience. Using Azure for deployment and MongoDB GridFS for file storage, we've created a secure and efficient application that emphasizes our commitment to quality and scalability.

What we learned

Our journey involved integrating the ChatGPT Whisper AI API into our Flask backend, revealing the potential of AI to enhance user experiences. Additionally, working with cloud services like AWS and Azure for deployment and backend tasks expanded our skill set for future cloud-based projects.

On a personal level, we honed several important skills. We sharpened our problem-solving abilities, as we encountered and successfully resolved challenges during the integration process. Our teamwork skills were refined as we collaborated to find solutions and make our application work seamlessly. Adaptability became a valuable asset as we navigated through unexpected technical complexities.

What's next for TalkToText

Looking ahead, our app has exciting plans for further enhancements. We are actively working on implementing real-time translation and transcription capabilities into our existing transcribing service. This feature will enable users to experience seamless, on-the-fly translation and transcription, making our app even more versatile and user-friendly. These planned updates align with our commitment to continually improve and expand the functionality of our app, ensuring it remains a valuable tool for a diverse range of users and content types. Stay tuned for these exciting developments in the near future!

Programming Language

Our app was primarily built using two programming languages: Python and JavaScript. We utilized Flask, a Python web framework, for the backend development, while the frontend was developed using React, a popular JavaScript library. This combination of Python and JavaScript allowed us to create a robust and interactive application that meets the needs of our users.

Demo Video Example

Original Audio Source: Link to Audio

Email Output Presented In The Video:

Summary: In a controlled experiment, data was collected and analyzed to examine the relationship between two factors. The results showed statistical similarity, suggesting a connection. Participants were selected from a diverse pool and ethical guidelines were followed. The study concluded that further research is needed to validate the initial hypothesis. Limitations included sample size and potential bias, which should be addressed in future investigations. Overall, the study aimed to explore the variables systematically and highlighted the importance of future research.

Key Points:

The experiment was conducted in a controlled environment with specific variables.

Data was collected, analyzed, and presented in a concise report.

The results indicated a statistical similarity between the two factors under investigation.

Participants were selected from a diverse pool of candidates, and informed consent was obtained.

Ethical guidelines were followed throughout the study.

The methodology employed rigorous testing procedures and strict adherence to the research protocol.

The conclusion drawn from the findings is that further research is warranted to validate the initial hypothesis.

The limitations of the study include sample size and potential bias.

These limitations should be addressed in future investigations.

Action Items: There are no specific action items or tasks mentioned in the text.

Sentiment Analysis: The sentiment of the text is generally positive.

Explanation:

The language used throughout the text is neutral and objective, focusing on the methodology and procedures of the experiment.

Words like "controlled," "specific," "concise," "rigorous," and "adherence" indicate a professional and systematic approach, which contributes to a positive sentiment.

The mention of "diverse pool of candidates" and "ethical guidelines" suggests a responsible and inclusive approach, further enhancing the positive sentiment.

The conclusion that "further research is warranted" indicates a desire for improvement and advancement, rather than a negative outcome.

The mention of limitations and potential bias shows an awareness of the study's shortcomings, but it is presented in a neutral manner without any negative connotations.

The final sentence seems to be unrelated to the rest of the text and does not contribute to the sentiment analysis.

Full Transcript: The experiment involved a controlled environment with specific variables. Data was collected, analyzed, and presented in a concise report. The results indicated a statistical similarity between the two factors under investigation. Participants were selected accordingly in a diverse pool of candidates. Informed consent was obtained, and ethical guidelines were followed throughout the study. The methodology employed rigorous testing procedures and strict adherence to the research protocol. The conclusion drawn from the findings... further research is warranted in order to validate the initial hypothesis. The limitations of the study include sample size and potential bias, which should be addressed in further future investigations. In summary, the study aimed to examine the relationship between the variables in question using a systematic approach that provided and highlighted the importance of future research.