your tempo

saving a part of yourself into music. a delightful combination of machine learning, web dev, discord bots, and most importantly music :D

Comment 2

Inspiration

Hm.... saving? What do we save, what is Important to be saved? Money, trees, bees? Well, when we first heard of the theme, we first thought about more materialistic things we could save; however, the more we thought into it, we thought that the most important thing ever to save is yourself. Similar to how you save a piece of yourself in an elementary school yearbook, "your tempo" hopes to save a part of yourself- your texts- into a one-of-a-kind song.

What it does

Well, for the user, it's pretty simple. You type in "-tempo", and then you see a message telling you type in "<3saving." Right afterwards, you are given the link to the song that has YOU saved into it. The link leads you to a beautiful and slightly humorous site that lets you listen to the unique, ai generated song, which has yourself saved into it (through your mood!!).

However..... the process is actually much more complicated! The discord bot contains several commands, but the main one is "-tempo". This command will cause the bot to collect your past 100 messages, and generate a song based on the mood of the past messages. (The generating part is also very long and complicated and will be explained in the How we built it section)

We decided to use a discord bot because it suited our project perfectly, and it allowed users to easily access their very own music.

We also have a command "-avatar (mention user)" which gives back a user's profile. The user's profile has all the tempos the specific user has created along with when they were created! We added this feature so that what you saved could be shared with others.

How we built it

-Main Discord Bot- We created a discord bot (your tempo#1854) that the user interfaces with. When the user uses the -tempo command, our bot runs a search through the last 100 messages sent in the channel the command was performed in to find which of the 100 were sent by the user calling the bot. It then sends these messages to a Firebase database.

-Sub Discord Bot that runs the ML Model/ TensorFlow Text Mood Classification- We have another bot running that checks when the database is updated. When it gets triggered, it attempts to detect what mood those messages primarily convey. Using Tensorflow, we created a ML Model that detects the mood in a text. A database with more than 40,000 messages was used. These texts were preprocessed using custom functions in order to deal with mentions, contractions, slang, etc. Then, the messages were tokenized and turned into numbers that the model could use. A sequential model with 4 layers were used.

-Saving yourself into a song- The most prevalent mood in the group of texts was used to decide the two songs that would be "interpolated" in our Magenta ML Models. Once the mood of the messages has been determined, the second bot send the User the link of their very own song.

-Magenta (generate one-of-a-kind music through AI)- Magenta is an AI based project that can generate music through code. It provides an environment where developers can train ML models, and use these models to lengthen or create music. We used Magenta to first "interpolate" two songs in midi files. This allowed the model to find similar melodies between the two songs. Then, we lengthened this segment and continued the song. Now, the song was taking more shape, but it was still missing something. We used a drum ml model, which gave our songs a drum beat. And finally, we humanized this drum beat using another ml model from Magenta. In the end, we were able to retrieve a unique, one-of-a-kind music that had never been seen before.

Challenges we ran into

It was hard combining the different elements. We had TWO discord bots, a full website, a database, machine learning models, and an AI song generator that all had to work simultaneously to deliver the "your tempo" experience. We realized later on the hackathon that we failed to clearly organize these connections, and we had to reorganize our entire project. However, we were able to carefully think through and successfully create "your tempo."

What we learned

We learned so so much! None of us know how to make Machine Learning Models or how to Generate Music. Over the past 48 hours, each one of us literally learned a new language (basically :D). We also never before created 2 discord bots that communicated with each other through a database, and served to be a challenge as well.

What's next for your tempo

While "your tempo" is at a functional state right now, there is always more to improve! We would love to improve our ML Model Accuracy, and add other factors that could change how the song is generated. Maybe a grammar spectrum?

We would also love to improve on our bot commands and allow them to be much more versatile.

Github Links?

Our bot GitHub links can't be posted sadly due to discord token issues. If you would like to see the GitHub, please contact aralcc#2384 on discord. Thank you!

Share this project:

Updates