Inspiration💡
Our team being Machine Learning enthusiasts we often train my models for hours or sometimes days on a remote server. What if we could easily monitor my models progressing in real-time from my mobile devices and let's say monitor my ML Model live while taking a walk. We figured it would be really helpful for all ML enthusiasts if they could do so and thus started our project TF Watcher.
What it does🔎
TF Watcher is a simple to use Python package and a web app that allows you to monitor 👀 your Machine Learning training, validation, or testing processes on mobile devices 📱 very easily. The Python package collects data and sends it to Firebase realtime database after some interval (specified by the user). The web app uses the data written to Firebase's realtime database and allows you to visualize the live progress of your model with graphs and charts.
Some other features:
- Developers can start using this project with as little as 3 lines of code
- All the visualizations on the website work in real-time
- This also integrates seamlessly with your ML workflow, you don't need to change any other code in your workflow to make this work
- ML is highly collaborative, we allow sharing your live or stored logs with anyone else too
- The Web App is a PWA (Progressive Web App) which means you also have limited offline access, blazing-fast performance, and ability to install the web app too
- We believe the web app might be used the most but we also built an Android App and a Windows App sharing the same backend
- In the Devpost media gallery, we show how we monitored 4 metrics (accuracy, val accuracy, loss, and val loss) but this depends on the user, he could also monitor some other let's say 10 metrics of his choice
How we built it🔨
We majorly divide this project into two sub projects:
TF Watcher Python Package
This is a Python Package built using TensorFlow and Firebase APIs. This allows you to easily monitor the metrics you want and write them to Firebase's real-time database. We thought to build this as a Python package so you could easily install it with a single command (see the docs for more info) and start using it very easily in any system.
TF Watcher Web App
This is a Web Application built using React, Chakra-UI, Firebase and Recharts, and is deployed on Azure Static Web Apps. It reads from the Firebase real-time database and displays the logs using charts. We thought to make this as a Progressive Web App to make this a lot faster, allow users to use the web app in a limited manner offline too.
Challenges we ran into⚠️
- While building the Python package designing the base classes appropriately was a huge challenge, we wanted to target users who just wanted to do simple logging and also users who wanted to customize every single bit of logging. We struggled on this for a long but finally fixed this understanding of how some core functionality of Python could be modified to use multiple dispatcher
- Converting the website to PWA was a challenge, as we hadn't tried it before, but after going through a couple of articles we figured it out
- On deploying our web app to Azure did not work as intended, but we later understood that Azure was forcing some of its premade components not allowing us to use fallback routes and fixed it
Accomplishments that we're proud of🥇
- We think we worked really well as a team and had seamless collaboration, all on GitHub
- We also built this project following all open-source standards: comments, writing docs, creating RFCs, code reviews and more. We also plan to work on this open-source project after the fellowship and possibly turn it into a product
- We were able to share our knowledge with the rest of the team: a part of our team learned working with TensorFlow while the other part learned working with React and building UIs
What we learned🧠
- Adding Google Account authentication
- Converting a Web App to Progressive Web App
- Deploying Web Apps to Azure
- Writing and hosting usable documentation
- Testing our code efficiently
- Working with CI/ CD efficiently with GitHub Actions
- Using Kanban-style GitHub projects and Discussions to efficiently manage the project
What's next for TF Watcher💭
Some of our future plans include:
- Allow users to create a dashboard in their account
- As of now we have set up a system so the user can generate a shareable view-only link for his ML job, we plan to add some more features for sharing these visualizations like sharing it with emails and allowing to share a particular role with someone (view-only, edit and so on)
- Create a Discord bot to visualize the dashboard
- Though TensorFlow is one of the most used ML framework and we support it, we also want to support PyTorch
Log in or sign up for Devpost to join the conversation.