TourFlow Live

TourFlow Live captures the tour as it happens, streaming data through Confluent the moment connectivity allows. AI reconstructs the timeline instantly, turning a jungle trek into a live digital feed.

Comment

TourFlow Live

The hands-free AI scribe for guides. We stream offline tour data through Confluent, using Gemini to generate a structured real-time story feed and permanent gallery—hands-free.

Inspiration

Living in Bocas del Toro, Panama, my partner and I spend our days guiding travelers—I lead hands-on chocolate workshops at Finca Montezuma, and they lead nature tours through the rainforest.

We see the same challenge every day: guests want to capture everything, but they often end up with a messy camera roll and no context. In my workshops, their hands are covered in sticky chocolate; on nature walks, they might snap 50 blurry photos trying to get one good shot of a sloth.

We realized the problem wasn't taking photos—it was curating them. We built TourFlow Live to turn memory-making into a collaboration. Guests can snap away freely, and I can focus on storytelling. The magic happens in the background: our AI "Director" listens to the tour and watches the stream of photos coming in from everyone. It automatically identifies the best shots—whether it’s my close-up of a fermentation box or a guest’s perfect angle of the group—and syncs them to the narrative instantly. It lets us co-create a perfect, shared gallery without breaking the flow of the experience.

How we built it

TourFlow Live is a sophisticated real-time pipeline built for speed and context-aware intelligence:

  • The Blueprint (Context Engine): Before the tour, the guide defines a "Tour Blueprint" (stored in a compacted Kafka topic). This acts as the source of truth, telling the AI exactly where the group is (e.g., "Stop 1: Cacao Nursery"). This allows the AI to ground its storytelling in facts rather than hallucinations.
  • The Brain: We used Google Gemini 3.0 Flash for multi-modal analysis. It consumes the "Blueprint" context, "listens" to the guide’s real-time audio, and "looks" at guest photos. It then fuses these three inputs to write engaging, factually accurate captions.
  • The Nervous System: Confluent Cloud (Apache Kafka) handles the data stream. We utilized Kafka Tiered Storage with infinite retention, turning Kafka into our primary "Source of Truth" database.
  • The Interface: A responsive Next.js frontend provides a vertical "Live Feed" and a "Gallery Mode" for masonry-grid exploration, connected via Server-Sent Events (SSE).

Challenges we ran into

  • Context synchronization: The biggest challenge was ensuring the AI knew which stop the group was currently at. We solved this by having the backend join the live media stream with the "Blueprint" topic, so every photo sent to Gemini is already tagged with "Stop: Cacao Fermentation Area."
  • Kafka as a Database: Moving away from a traditional SQL database to using Kafka's offset-seeking for our "Gallery Replay" was a paradigm shift that required careful management of high watermarks.
  • Deployment Logistics: Coordinating environment variables across GitHub Actions, Google Cloud Run, and Confluent was complex, but automating it via CI/CD proved invaluable.

Accomplishments that we're proud of

  • Infinite Replay: Guests can return to their unique tour link anytime—days, months, or years later—to see the full story exactly as it happened. By replaying the event stream from Kafka, we recreate the "Live" feeling on demand.
  • Hallucination-Free Storytelling: By anchoring Gemini 3.0 Flash with our "Tour Blueprint," we achieved extremely high accuracy in our captions.
  • Zero-Database Architecture: We proved that for modern event-driven apps, Kafka’s tiered storage can fully replace a traditional database for event history.

What we learned

  • Structured Prompting: We learned that feeding an LLM a structured "State of the World" (from our Blueprint) alongside raw unstructured data (audio/video) drastically improves output quality.
  • Event-Driven Thinking: We deepened our understanding of Kafka offsets and how to build frontends that can reconstruct their state from an event stream.

What's next for TourFlow Live

  • Visual Planner UI: Building a drag-and-drop interface for guides to easily create and edit their "Tour Blueprints" before they head out.
  • Interactive Maps: Using the Blueprint's GPS coordinates to render a live map showing the group's progress.
  • Guest Interaction: Allowing remote followers to "react" or ask questions that the guide can see in real-time.

Built With

Share this project:

Updates

posted an update —

I changed the backend of Tourflow Live due to Cloud Run costs (caused by Kafka). The solution: 1. Remove Kafka: I replaced the expensive Confluent Kafka event bus with a local, in-memory EventEmitter. This runs entirely within the server process—zero external cost.

  1. Merge Services: Instead of running separate server.js and ai_processor.js processes, I combined them. The server will handle uploads and immediately trigger the AI logic internally.
  2. Simplify Storage: I'm using simple in-memory storage for the session history. Note: This means history will reset if the server restarts.

Log in or sign up for Devpost to join the conversation.