TuneToons

Inspiration

Because this hackathon was very open-ended, we decided to make something fun! We originally thought it would be cute to have an app where an animal dances to audio on the user's phone. We ended up changing it into a desktop application so that it could be used in the background while the user multitasks! Overall, we wanted to make something fun and cute while stepping out of our comfort zone to make something with unfamiliar tools.

What it does

Tunetoons listens to audio input from the user's device and has a visual reaction, showcasing the audio colorfully. It features a cute bird mascot, Chester the Parakeet, who moves along with the music as well. With more development, the user would choose different animals to be the center of their audio spectrum. The user can save recordings of the visualizer to their computer to share later.

How I built it

The assets used in the audio visualizer, such as the animal characters, were hand-drawn during the hackathon in Clip Studio Paint. These assets were separated into pieces and animated in Cubism Live2D. The animated models were then imported into Unity. A variety of background assets were created in Unity and scripted to react to the music; different parts have different sensitivities and react differently, changing color.

Challenges I ran into

Aidan: I found it difficult to make a Unity application capable of listening to general audio from the computer. There was a steep learning curve because I have very little coding experience, and during this hackathon had to learn C# and use Unity for the first time. It was hard to create and put together C# code from various sources in an interface that was brand new to me.

Athena: It was very difficult to transition animations from Cubism Live2D to Unity due to outdated documentation and inconsistencies between the tutorials/readings I referenced versus the material I was working with. It was hard to figure out how Unity would interpret beats in audio, or "hear" where the beats are to react. Being unfamiliar with Live2D and animating in general, it was daunting to convert through various programs.

Accomplishments that I'm proud of

Aidan: I am proud I was able to learn a lot in a short amount of time and that I was able to get the recording function working. Having no prior experience with Unity or C#, I'm proud of all that I learned during this hackathon and how I applied it.

Athena: I'm proud of drawing a really cute bird and figuring out how to get the Cubism Live2D framework working in Unity and being able to create the audio spectrum. I'm proud I was able to use my novice Unity skills to put something new and fun together.

What I learned

Aidan: I learned basic Java and C# as well as how to use Unity. This was my first hackathon, so I learned about how it is to participate in a hackathon and put together a project. I learned how to hide the user interface and save recordings from the Unity application to a computer.

Athena: I learned how to make assets respond to audio in Unity. I also learned how to make models in Live2D and use the Live2D framework in Unity. I learned more about animation and C#.

What's next for TuneToons

Connecting the visualizer to general computer audio or Spotify so that it can respond to music in real-time, adding more new cute animals to the animal spotlight, allowing users to share their recordings directly to social media