Ayre | Devpost

Voice Command Panel
Two Hand Demo
One Hand Demo

Inspiration

While discussing the joy instrumental music brings to people, our team realized that learning and playing instruments isn’t equally accessible to everyone. Traditional lessons and instruments can be expensive, and many digital music tools assume the user can physically interact with a keyboard, mouse, or instrument in conventional ways.

Our idea originated from the broader theme of access — what happens when someone wants to create music but physically cannot?

That question inspired Ayre: a webcam-based, accessible virtual instrument that removes physical and financial barriers to musical expression. We wanted to reimagine music creation through accessibility-first design rather than treating accessibility as an afterthought.

What it does

Ayre is a webcam-powered virtual instrument that allows users to create music without traditional hardware. Using computer vision and gesture detection, users can trigger instrument sounds and synth controls through single or double hand movement — or even voice alone.

Challenges we ran into

One of our biggest challenges was balancing responsiveness with accessibility. The gesture detection needed to feel immediate to be musically satisfying, but overly sensitive detection amplifies tremors and unintended movement. Designing a smoothing system that reduced jitter required careful tuning.

Another challenge was rethinking traditional music interfaces. Most digital instruments are designed around two-handed input and fine motor control. Rebuilding the control scheme to work one-handed — while still feeling expressive and intuitive — forced us to rethink spatial layouts and interaction design from the ground up.

On the same note, voice-only mode introduced its own obstacles. We had to design commands that were simple, distinct, and reliable while avoiding false triggers during performance.

Accomplishments that we're proud of

Tremor smoothing — an adjustable stabilization slider that reduces jitter caused by hand tremors or conditions like Parkinson's or muscle weakness.

One-hand mode — redesigned control scheme so the full instrument is playable with a single hand, while maintaining the full functionality that the two hand mode entails.

Voice-only mode — trigger instruments and change synth presets via voice commands using the Web Speech API, for users who cannot use hand gestures at all.

Customizable finger assignment — reassign which finger controls which instrument. Accommodates non-standard hand anatomy or personal preference.

No sophisticated hardware required - the product overcomes the income barrier because it runs on any modern laptop or desktop with a webcam and microphone.

What we learned

We learned that designing with accessibility in mind is no simple feat, which is why it is likely often overlooked. Incorporating accessibility reshapes everything: interface layout, latency tolerance, gesture sensitivity, input mapping, and even how you define “success” for a feature.

We also learned that solving accessibility challenges requires balancing competing needs. For example, reducing tremor-induced jitter improves usability for some users, but too much smoothing can reduce musical responsiveness.

Another key lesson was that assumptions about “normal” interaction patterns are deeply embedded in software design. Most music tools assume two hands, precise finger control, and stable movement. By questioning those assumptions, we were able to be more creative with our product and discovered entirely new interaction models — like customizable finger assignment and voice-only control — that actually expand creative possibilities for everyone, not just users with disabilities.

Finally, we learned that inclusive design leads to better design overall. Features like tremor smoothing, sensitivity calibration, and flexible input modes don’t just serve edge cases — they make the product more robust, expressive, and user-centered for all musicians.

What's next for Ayre

Fixing the voice to instrumental audio ratio, so it is well balanced without the user having to consciously reduce the volume.
Add more instrumental sounds
Live transcription
Adding the option for voice commands to overlay vocals on the created instrumental lyrics