SmartScribe

Landing Page
Lectures List Page
Summary Page
Flashcard Page

Inspiration

As students ourselves, we've experienced the struggle of keeping up with lectures and study materials. The breaking point came during finals week when we found ourself desperately trying to watch an entire term's worth of lectures at 2x speed. We realized that learning shouldn't be this stressful or inefficient.

The traditional approach to studying lectures – watching hours of content, taking disorganized notes, and hoping to remember key concepts – simply doesn't align with how our brains actually learn and retain information. This gap between educational content delivery and effective learning methods inspired us to create SmartScribe.

We wanted to build something that transforms passive lecture watching into active learning, making education more accessible, efficient, and personalized for everyone.

What it does

SmartScribe is an AI-powered study companion that transforms lecture videos and transcripts into comprehensive, interactive study materials:

Topic Summaries & Transcripts: Automatically generates key concepts and creates concise, structured summaries of lectures along with related transcripts for each concept
Flashcards: Creates flashcards focused on key concepts to support active recall learning
Knowledge Graph Visualization: Maps connections between concepts visually to enhance understanding of relationships
Multi-modal Learning: Supports different learning styles with visual, text, and audio materials
Text To Speech: Converts text summaries to speech for auditory learners or on-the-go studying Students simply upload their lecture video or transcript, and within minutes, SmartScribe processes the content and generates all these study materials automatically.
PDF Export: Allows students to download all generated study materials as a structured PDF for offline studying

How we built it

Frontend: NextJS, Tailwind CSS, Cosmograph
Backend: Express
Relational Database: PostgreSQL, Prisma
Vector Database: ChromaDB
AI & Processing:
- OpenAI's text-embedding-3-small model for vector embeddings
- OpenAI's models for generating summaries and flashcards
- OpenAI Whisper for transcribing video to text
Text-to-Speech: Speech Synthesis API
PDF functionality: js-PDF

Challenges we ran into

API Limitations: We encountered numerous issues with OpenAI timing out or being unable to process large files
UI/UX: Creating an interface that abstracts the underlying complexity of our system while remaining intuitive
Idea Consensus: Initially, team members had different visions for the project. We had to balance ambition with practicality given our time constraints, ultimately converging on SmartScribe as the most valuable and feasible idea
API Contract Management: Maintaining consistent communication between frontend and backend required consistent documentation to ensure both sides were in sync
Error Handling: Implementing fallbacks and error handling in the frontend proved challenging due to the multiple potential failure points in our AI-driven pipeline
Learning Curve: Several team members had to quickly learn new technologies like vector embeddings and Prisma ORM to implement key features effectively
Data Processing: Optimizing the processing of large lecture files to create acceptable input sizes for language models required preprocessing techniques

Accomplishments that we're proud of

Large File Processing: While still working around OpenAI's limitations, we successfully enabled processing of lectures up to 2 hours long—covering the most common lecture durations
Frontend Implementation: We created a user interface that closely matched our original UI/UX vision from Figma
Team Communication: We established effective collaboration through Discord for sharing code, resources, and environment secrets
Feature Completion: We managed to implement most of our planned features and make them functional within the hackathon's tight time constraints
Semantic Understanding: Our system successfully identifies and connects related concepts even when presented using different terminology across lectures
Accessibility: We created a tool that makes quality education more accessible to different learning styles and students with various constraints.

What's next for SmartScribe

Looking ahead, our vision for SmartScribe centers on making education more accessible and personalized for diverse learning needs. We plan to deepen our focus on adapting to different learning styles by expanding our multi-modal offerings and introducing more customizable study paths.

Accessibility remains at the core of our mission, and we're working to ensure SmartScribe can serve students with various disabilities, language backgrounds, and educational contexts. This includes developing support for multiple languages, implementing screen reader compatibility, and creating interfaces for users with limited internet connectivity.

We also recognize the collaborative nature of effective learning. Future iterations will introduce features that enable students to share insights, compare notes, and study together within the platform. By creating an ecosystem where knowledge can be collectively enhanced, we hope to foster communities of learners who support each other's educational journeys.

As educational technology evolves, so too will SmartScribe. We're committed to continuously refining our AI understanding of academic content, expanding integration with learning management systems, and adapting to the changing landscape of education. Our ultimate goal is to create a tool that doesn't just help students pass exams, but genuinely enhances understanding and retention of knowledge for lifelong learning.