Epidemio

Inspiration

The inspiration behind Epidemio came from the ongoing need for proactive epidemic monitoring, especially for respiratory diseases that can rapidly spread and overwhelm healthcare systems. By leveraging predictive analytics, we aimed to empower health professionals and authorities to prepare for and mitigate the impact of infectious diseases. This idea became more relevant with the global health challenges posed by diseases like COVID-19 and influenza. The aim was to create a tool that could give early warning signals based on real-world data and predictions, thus providing a smarter approach to public health management.

What it does

Epidemio is a real-time alert system designed to predict the future spread of respiratory infectious diseases based on epidemiological data. It ingests historical datasets, processes them with machine learning models, and uses Grafana for real-time data visualization and alerting. The system:

Predicts future disease outbreaks.

Tracks the spread of infections every week.

Provides alerts and actionable insights to healthcare providers and public health authorities.

Makes use of real-world data from Catsalut datasets, which provide valuable epidemiological data from the Catalan Health Service.

Visualizes the data through dashboards for easy interpretation and decision-making.

Makes use of real-world data from CSV files that are periodically updated to refine its predictions.

Alert System: The system automatically sends notifications (email, SMS, or app notifications) when certain predefined thresholds for disease spread are reached, enabling quick responses from healthcare teams.

How we built it

The system was built using a combination of tools and technologies:

SQLite was chosen as the database to store disease prediction data and historical records.

Grafana was implemented to display data in interactive dashboards.

Pandas were used for data processing and predictive analytics, respectively.

Machine Learning: We employed time-series forecasting models to predict future disease trends based on past data.

Challenges we ran into

Data quality and accuracy: Inconsistent or incomplete data from various sources made it difficult to create reliable models. Cleaning the data and filling missing values was a time-consuming process.

Model performance: Fine-tuning the predictive models to accurately forecast disease trends proved to be a challenge. The data was highly dynamic and required constant recalibration.

Real-time updates: Ensuring that the system could handle real-time updates without delays or performance issues required careful optimization of data pipelines.

Visualization complexity: Creating intuitive and informative Grafana dashboards for non-technical users was challenging, as it required balancing technical depth with simplicity.

Alert management: Ensuring timely and accurate alert delivery was a challenge, especially when working with various notification channels (email, SMS, push notifications). Fine-tuning the threshold values and avoiding false positives/negatives was crucial.

Accomplishments that we're proud of

Successfully integrated predictive modeling with real-time data monitoring.

Built a scalable solution that can easily handle different datasets and be applied to various diseases.

Developed interactive dashboards in Grafana that allow health authorities to easily visualize trends and make decisions.

The system was able to make accurate predictions based on historical data, helping to simulate how an outbreak could evolve.

Automated the data import and processing pipeline, significantly reducing the time it takes to update the predictions.

Integrated Catsalut datasets, providing real-time epidemiological data from the Catalan Health Service to improve the accuracy of predictions and enable better disease tracking.

What we learned

Data preprocessing is critical: The quality and structure of the input data are key factors in the performance of machine learning models. We learned the importance of cleaning, transforming, and verifying data before using it for predictions.

Collaboration with non-technical users: Designing dashboards that provide both detailed insights and simple, actionable information taught us how to bridge the gap between data science and healthcare professionals.

Time-series forecasting: We learned how to fine-tune forecasting models to improve their accuracy in predicting disease spread.

Real-time monitoring challenges: Ensuring the system could handle real-time data without lag or performance issues helped us gain experience with large-scale data processing.

And then??

Expand data sources: Incorporate additional datasets such as real-time medical reports, hospital capacity, or vaccination rates to improve prediction accuracy.

Machine learning improvements: Integrate more advanced machine learning algorithms, such as deep learning models, to further improve prediction capabilities.

Multi-disease prediction: Extend the system to monitor and predict multiple types of infectious diseases, not just respiratory ones.

Global deployment: Expand the system to include global datasets for worldwide epidemic tracking and prediction.

Advanced Alert Management: Further enhance the alert system by incorporating AI to predict optimal intervention times based on real-time data, integrating more notification channels (e.g., automated phone calls), and incorporating location-based alerts for faster response in specific regions.