WhistlesML

Smart cooker whistles tracker powered by machine learning

Comment

Inspiration

One of my daily jobs at home is to turn off the rice cooker after a certain amount of whistles, proportional to the amount of rice inside. A common mishap that I experience is forgetting to turn it off after the whistles have finished. Hence, I developed an effective solution to this problem using real-time machine learning classification.

What it does

WhistlesML is a utility app that can be utilized at home to track the number of whistles emitted from a pressure cooker and get notified when the count has culminated. Notifications can be in the form of alarms, notifications, or an email. It also has personalization features to modify the style of the app.

How I built it

I developed this app using Xcode 13.1 and the Swift programming language. I first gathered a decent amount of audio data samples (about #40) capturing pressure cooker whistle sounds, as well as background noises to train an accurate sound classification model using CreateML. Then I performed the machine learning classification in real-time using Apple's CoreML framework and updated the text view to reflect what sound it was currently detecting. I also requested the user to allow access to notifications if they choose that method of alert. The email alert feature was implemented using a restful API that I developed in Python using the Flask framework. The rest is very simple, as it only involves UI design and animation.

Challenges I ran into

Sometimes the machine learning model would think the background noise is the whistle (just for a split second, or extremely short duration). The most challenging portion of this project was not incrementing the counter in those cases. To solve this logical issue, I checked if that model was identifying the sound in the audio input stream to be a whistle for at least five seconds.

Accomplishments that I am proud of

I began working on this project very recently. I am proud of being able to develop an elegant and simple solution to a difficult problem, in a relatively short time with some technologies I had never used before.

What I learned

I learned how to implement real-time audio classification in Swift. I also learned more about UI animations, design, time management, and problem-solving.

What's next for WhistlesML

Some future additions would be the following,

  1. SMS text message (to a specified phone number) upon completion of whistles
  2. Customized alarm sounds
  3. Tracking extra whistles if the user does not turn off the cooker in time
  4. Siri integration, to be able to start the app with a certain amount of whistles by querying Siri
  5. Another ambitious project (using a similar approach) would be to classify different types of vocal renditions of Indian classical melodies, also called Ragas.

Built With

Share this project:

Updates