Freestyle Buddy

Inspiration

Rap music is something that brings me and my friends closer together. One of their favorite pastimes is to spend their time "freestyling", or improvising rap verses over assorted instrumentals. Freestyling isn't just a source of entertainment - it's even scientifically proven to boost your mood. For my project, I wanted to create an easy way for them to hone their skills by using Meta's MusicGen library as well as custom LLMs trained using the Intel Developer Cloud to generate rap beats and lyrics respectively.

What it does

My vision for Freestyle Buddy is a simple application that allows users to generate customized rap instrumentals based on text input. The model returns short excerpts of instrumentals using a MusicGen model that are tailored to the input the users specifies. Additionally, LLMs are used to generate rap lyrics based on some input. This way, the user can practice rapping over different beats with inspiration for lyrics being provided as they do so.

How I built it

The MusicGen model inference was ran using the Intel Developer Cloud in order to make use of their hardware acceleration capabilities for AI to lower the model latency. The application backend was managed using Django, and I created the frontend using Next.js.

Challenges I ran into

A major challenge I ran into was converting the model pre-trained by Facebook into a format compatible with the Intel Developer Cloud. I also struggled with fine-tuning the LLM that generated rap lyrics. Wrestling with these issues left very little time to develop the application itself.

Accomplishments that I'm proud of (and what I learned)

I'm proud I was able to learn how to use the MusicGen model to produce music based on a prompt. It was fascinating to see the capabilities of music generation using AI. I'm also proud to have worked alone to develop an MVP for my project idea, even if I ran out of time to see it come to fruition. Lastly, the experience I got developing my project using the Intel Developer Cloud provided me valuable experience using the cloud to acceleration AI applications.

What's next for Freestyle Buddy

Next steps for Freestyle Buddy include lowering the time it takes to generate lyrics and instrumentals, improving the user interface, and deploying the application and getting more feedback. It will be important to use cloud resources to run inference on the music generation model, due to the vast amount of resources required. I hope to develop the app into something any aspiring rapper, or someone who just wants to have fun, can use to rap what's on their mind.

Built With

• django
• intel-developer-cloud
• musicgen
• nextjs
• openai