xBeat

logo
3d

Inspiration

xBeat was born from a vision to democratize professional DJing by combining cutting-edge AI with intuitive voice control. Just kidding, we are just a bunch of wannabe-DJs who need AI to create sick transitions for us.

What it does

xBeat is a voice-controlled AI DJ assistant that revolutionizes how DJs mix music. Users can speak naturally to Grok AI—saying things like "create a smooth transition to deck B" or "boost the bass on deck A"—and the system translates these commands into precise audio operations. The platform features a professional dual-deck mixing system with real-time 3D visualizations, automated transitions, track analysis, and smart recommendations. Everything runs in the browser with no hardware required.

How we built it

Core Audio Engine - We built a custom music engine from scratch using the Web Audio API, implementing dual-deck playback, real-time audio analysis, BPM detection, and a complete audio processing pipeline. This engine handles all low-level audio operations including buffer management, gain control, EQ filtering, and effects processing.
AI Integration - We integrated Grok AI through the Vercel AI SDK, creating Next.js API routes for voice command processing, track analysis, transition planning, preset generation, and track recommendations. We developed sophisticated prompts that enable the AI to understand DJ terminology, musical context, and current mixer state.
Voice Control - We implemented browser-based speech recognition for voice input and integrated xAI's text-to-speech API for AI responses. The system processes natural language commands, maintains conversation context, and translates user intent into precise audio operations.
Visualization - Using Three.js and React Three Fiber, we created real-time 3D visualizations that analyze audio frequency data and render immersive visual feedback. Multiple visualization modes (Cymatic, Particles, Tunnel, Waveform) provide different aesthetic experiences synchronized with the music.
Polish & Integration - We refined the UI using shadcn/ui components, integrated Vercel Blob for cloud audio storage, optimized performance for real-time operations, and added features like track library management and automated transition execution.

Challenges we ran into

Creating a professional-grade dual-deck audio engine with the Web Audio API required deep understanding of audio graphs, buffer management, and real-time processing. Implementing features like BPM detection, multi-band EQ, and effects processing while maintaining sub-50ms latency was particularly challenging. We had to carefully manage audio node connections and optimize buffer sizes to prevent audio glitches.

Also, rendering real-time 3D visualizations while processing audio and running AI inference simultaneously required careful optimization. We had to balance visual quality with performance, implement efficient frequency analysis algorithms, and optimize React re-renders to maintain smooth 60fps performance, especially on lower-end devices.

Accomplishments that we're proud of

We're proud to have built what we believe is one of the first fully voice-controlled DJ systems powered by AI. The ability to control professional mixing through natural conversation is a significant achievement that makes DJing more accessible for beginners.

What we learned

We learned a lot about web audio APIs and 3D graphics programming.

What's next for xBeat

We would like to add real-time collaboration features, allowing multiple DJs to work together on mixes remotely, with shared control and synchronized playback. Additionally, the ability to streaming mixes directly to platforms like X, Twitch, YouTube, or SoundCloud with synchronized visualizations would be awesome.