Humasong

Inspiration

One of our teammates, Michelle, always had a dream to be able to input notes into composing software by singing, which is one of the reasons that brought her to CMU.

What it does

This project aims to actualize Michelle's dream, by turning voice singing into electronic notes universal to music composing software.

How I built it

Making use of the Processing library in Java (and p5.js), we record and analyze audio and interpolate the frequency patterns to map the audio input to a .mid file, which encodes. The libraries are integrated into an HTML5 responsive website using p5.js so that it would be available for every music composers.

Challenges I ran into

Recognition of human voice is particularly challenging because human voices contain a combination of frequencies and confuse ordinary computer algorithms.

Accomplishments that I'm proud of

Cosine similarity has been widely used in statistical language models as a robust measure of similarity between vectors. In this project, we implemented a similarity measurement including cosine distance modulated by a Gaussian curve to measure the similarity between vector-encoded pronunciations. We also exhibited great teamwork and division of labor during a limited time, that resulted in an impressive outcome.

What I learned

Trust yourself and never give up on what you think you can do. Work closely with as a team and embrace different ideas.

What's next for Humasong

More robust audio recognition algorithms, e.g., RNN models and Attention models can be implemented to improve the continuity of the generated signals. The web application can be deployed as a hybrid-app that runs locally on Android and IOS devices.