Inspiration
Our teammate, Daniel, has a diploma in Piano - he’s performed at the Athens Concert Hall!
He teaches dozens of students how to play Piano and has found that it's very hard for aspiring self-taught musicians to infer the dynamics in beginner pieces - the pianoforte, the crescendo and so on. Students with learning difficulties, in particular Dyslexia and Dyspraxia, really struggle with the text annotations of dynamics on scores!
What it does
You upload a picture of the sheet music you are struggling with. We process the image, track the measures and apply our heuristics (learned through years of Daniel’s piano) to infer the dynamics for certain beats. We show the sheet music, annotated with the dynamics and render a virtual keyboard for users to apply their learnings!
How we built it
We process the image and pass the notes, durations and musical context inferred from the piece to a GPT model. We use tailored system prompts and generate a set of possible dynamics, which we cross-reference against our rules and then overlay on the sheet music.
Challenges we ran into
The biggest challenge we faced was finding a way to effectively process the sheet music, we experimented with different sheet formats such as musicxml, musescore etc. We settled on using musicxml as its easy to add in our generated dynamics at their appropriate measures and bars.
Accomplishments that we’re proud of
Creating a platform to enhance the learning journey of self taught musicians, saving money that would have to be spent on hiring professional piano teachers to infer the dynamics and supporting those with learning difficulties maximise their learning journey.
What we learnt
We learnt about the challenges in creating an effective and widely applicable heuristic to determine the dynamics of a complex piece of music.
What’s next
We believe this would be a great idea to incorporate for all instruments, integrate into music teaching apps, add expressivity markings and further even make an algorithm for the fingering technique.
Log in or sign up for Devpost to join the conversation.