Inspiration
Our journey embarked from a personal space, as one of our team members navigated the nuances of ADHD and its impact on learning. Observing their struggle and the lack of tailored educational resources sparked a drive within us to create a solution. We aimed to bridge the attention gap in educational videos, which led to the inception of AttentionTune, envisioning a world where learning is accessible and tailored to the unique needs of every individual, irrespective of their attention span.
What it does
AttentionTune seamlessly monitors the learner's attention in real-time as they engage with educational videos. Upon detecting distraction, it prompts the user with questions related to the video content, ensuring they remain engaged and comprehend the material. At the end of the session, a succinct summary of key points is provided to reinforce retention.
How we built it
Computer Vision:
- Eye Tracking: Implemented using OpenCV to monitor users' eye movements and attention patterns.
Web Technologies:
- React: Developed the user interface (UI) for seamless interaction and user experience.
Audio Processing:
- FFMPEG: Utilized for extracting audio from educational videos, enabling further analysis and processing.
Speech-to-Text Transcription:
- OpenAI Whisper: Converted audio content into text, providing a basis for further analysis and interaction.
Natural Language Processing (NLP) and AI:
- OpenAI GPT (Generative Pre-trained Transformer):
- Segmentation: Analyzed transcriptions and segmented them into logical sections for better organization.
- Question Generation: Generated multiple-choice questions (MCQs) based on segmented text, enhancing interactivity.
- Text Summarization: Summarized textual content, providing users with concise key points for reinforcement.
- Answer Verification: Verified user-provided answers to MCQs, ensuring accurate comprehension assessment.
Challenges we ran into
Frontend Development Learning Curve:
- Our team had a strong foundation in backend development, but frontend was a new territory. This required us to quickly ramp up on frontend technologies, design principles, and user experience considerations to ensure our solution was not only functional but also user-friendly.
Real-Time Attention Tracking Intricacies:
- Navigating the complexities of real-time attention tracking proved to be a significant challenge. Ensuring that the system promptly and accurately responds to distractions demanded a deep dive into optimizing our Computer Vision algorithms and ensuring seamless integration with the frontend for real-time feedback.
Accomplishments that we're proud of
Project Completion Amid Challenges:
- Despite facing various hurdles, we successfully navigated through them and completed the project, showcasing our team's resilience and problem-solving capabilities.
Unwavering Dedication:
- Our team demonstrated unwavering dedication by working tirelessly for two consecutive days. We made sure every aspect of the project was thoroughly fine-tuned and rigorously tested during this time.
Making a Difference:
- Having the opportunity to address the learning challenges faced by individuals with ADHD is something we hold dear. The potential impact of our project on enhancing the learning experience for this group is a meaningful accomplishment that goes beyond just technical achievement.
What we learned
The journey imparted valuable insights into the limitless potential of AI in addressing educational challenges. It also deepened our understanding of the specific learning needs of individuals with ADHD, highlighting the importance of inclusive and adaptive educational solutions.
What's next for AttentionTune
Integration of GPT-4 Vision:
- As an evolution to our current Computer Vision technology, we aim to explore the integration of GPT-4 Vision. This advanced model promises a fusion of natural language processing with visual recognition, potentially allowing for more nuanced attention tracking.
Personalized Learning Algorithms:
- We envision incorporating intelligent algorithms, possibly leveraging Reinforcement Learning (RL), to create a more personalized learning experience. By utilizing RL, the system could dynamically adapt to each individual's learning patterns and engagement levels.
Log in or sign up for Devpost to join the conversation.