Gesturefy

Inspiration

Our inspiration for this project was our combined passion for music. All of us love listening to music, and we know how much it is used on a daily, we wanted to make something that would make the process easier for others.

What it does

 Our project utilizes a hand-recognition system to sense gestures that work as commands to make the Spotify experience more efficient.

How we built it

We first created, using media pipe, a hand-recognition system that we would manually code to be able to sense certain hand signals and categorize them. After categorizing these symbols, we stored them as commands in the Spotify API. These commands would allow the user to manipulate Spotify (such as skipping, increasing volume, etc.) in a far more efficient sense, without even having to be on the app.

Challenges we ran into

One big challenge we ran into was storing gestures without them being overridden by other gestures. We found that certain hand signals were too common, and with lots of trial and error, we were able to specify and fix all the problems. We also had many minor challenges with implementing the code into the Spotify API, as it would run differently at times and differences between MacOS and WindowsOS would cause problems.

Accomplishments that we're proud of

A big accomplishment we had was when we finally were able to store each gesture with minimal to no error. This was big as we had to go through lots of debugging and properly understand different ways to go about a solution.

What we learned

We learned primarily how to utilize information from different sources to better understand the task we had at hand quickly. As our team has never worked with libraries like mediapipe or the Spotify API, there was a big learning curve we were able to tackle effectively.

What's next for Gesturefy

We wish to expand Gesturefy into different devices apart from just laptops. We also mainly want to implement Gesturefy into a proper application that can be sent to many, allowing them to access Gesturefy on any device with no error.