TremorGuard

Inspiration

We were inspired by one of our teammate's grandfather who suffers from Parkinson's Disease (PD) but still keeps a curious attitude towards learning. Thus, we wanted to create something that could help him manage his PD progression so that he can pursue his dreams of lifelong learning.

What it does

TremorGuard presents a multi-faceted disease management system for patients with Parkinson’s Disease (PD). It consists of two IoT-based monitoring devices - one to be worn on the forearm, and another to be clipped on the neckline of a shirt. Through machine learning algorithms, our system records disease progression and classifies patient daily activity to aid doctors in their administration of care. With these generated results, TremorGuard reminds patients to consume medication when their symptoms worsen (indication of medication wearing off), provides regular reports on the patient’s disease progression, chronicles their daily activities, and stores the received sensor data in a database for future training purposes.

How we built it

Sensors / Actuation: We connected a microphone and IMU (accelerometer) sensor to the ESP32 microcontroller which sends data to IoTHub. This code is written and uploaded using Arduino. We also have a LCD screen connected to another ESP32 that listens for messages from IoTHub.

ML: We used models like CNN (convolutional neural networks), MLPC (multilayer perceptron classifier) and LSTM (long short-term memory) to train our datasets. We did our actual training on jupyter notebooks, and used libraries like pytorch and numpy.

Web Application: We developed our application front-end using NextJS and NestJS. For our back-end, we used the Flask framework and postgreSQL for our database that stores our sensor data and predictions.

Challenges we ran into

Since our project focuses on a niche topic, we had issues finding relevant datasets for our use case. We also struggled to replicate the data collection method the dataset uses since they were conducted by medical professionals who had access to advanced sensors and a controlled environment, which meant that the collected data had more variables and was more accurate than ours.

Most of our team members are also from Computer Science background with little electrical engineering experience, so we also spent quite some time figuring out the connections and how to make everything work.

Accomplishments that we're proud of

Working with challenging sensor data: It was particularly difficult for us to process the microphone data since we need to first convert the amplitude readings into fundamental frequency and peak to peak amplitude. However, the study we based our data on had access to proprietary processing software that we didn't. Thus, we had research how such values are computed and create our own estimates for them.

Setting up cloud communications: This was our first time working on IoTHub, and we were unsure of how we could send and receive data form IoTHub on our microcontrollers. We found a library that helps us manage this, and are proud that we were able to make some optimisations to our data pipeline by batching the collected sensor data before sending it to cloud.

What we learned

We gained a better understanding of how IoTHub works, and how to handle the routing between devices. We also learnt a little more about software engineering and the tools we could use to integrate our many components - ML models, live display of results on our web application, and cloud streaming of data.

Our team has also learnt to communicate better with each other, as each of us focused on one segment of the project (eg. Hardware, Web application, ML training), and we had to coordinate many details such as the format of data required, frequency of collection for a more accurate result, etc.

What's next for TremorGuard

We would like for TremorGuard to be fully hosted on cloud. Currently, the data transmission between the microcontrollers and the web application is handled by IoTHub, and the feeding of data into our models for a prediction and storing of data in postgreSQL is handled locally. Given more time and access to Azure services, we would like to use a cloud database, and let the machine learning predictions happen in the cloud.