About the Project – SHORTR

🎬 Inspiration

We were inspired by the explosion of short-form video content on platforms like TikTok, Instagram Reels, and YouTube Shorts. Long interviews, podcasts, and webinars often contain hidden gems — but finding and editing them into engaging reels is slow and tedious. We wanted to automate this process: turn hours of footage into viral-ready reels in minutes.

🛠️ How We Built It

• Transcript Parsing: We built a pipeline that ingests raw transcripts (JSON with start, end, and text fields).
• Keyword Mining: Using LLMs and repetition analysis, we automatically extract trending or hook-worthy keywords.
• Segment Selection: We cluster transcript chunks and apply heuristics (hook phrases, question marks, emotional intensity, pacing) to identify the best 20–35 second highlights.
• Video Processing: With ffmpeg, we cut and merge segments frame-accurately, then reframe them into 9:16 vertical format for Shorts/Reels.

• Sponsor Tools:

  • LlamaIndex — indexing and semantic clustering of transcript segments.
  • Anthropic/Minimax — LLM-driven rewriting of captions and hooks.
  • TigerData — storing transcripts and selected clips for querying.
  • Apify — scraping trending TikTok hashtags to guide keyword boosting.

📚 What We Learned

• How to combine heuristics with LLMs for better segment ranking (pure LLMs were too verbose, heuristics alone were too rigid).
• Efficient video slicing and concatenation pipelines with ffmpeg.
• Designing evaluation loops: using HoneyHive-style logging to compare automatic picks with human judgment.
• The importance of temporal dispersion (spreading highlights across the video rather than clustering them all at the start).

🚧 Challenges We Faced

• Audio extraction on macOS: librosa failed without ffmpeg backends; we had to add fallback modes (--no-audio).
• Transcript alignment: merging overlapping or near-adjacent timestamps cleanly.
• Reel length tuning: balancing the sweet spot of 20–35s while avoiding cuts mid-sentence.
• Keyword quality: filtering out generic words (“today”, “video”, “episode”) while surfacing specific viral terms.

✨ Outcome

With SHORTR, we can go from a 60-minute podcast → 3 viral-ready reels in under 2 minutes. This empowers creators to repurpose long-form content, reach wider audiences, and ride on trending keywords without spending hours in manual editing.

Built With

• apify
• brightdata
• fastapi
• gladia
• minimax
• python
• tigerdata