SmartBuilding

Lost item detection
data score
Trash detection
Automatic raised ticket for cleaning job
Automatic Raised ticket for lost item

Inspiration

In today's rapidly evolving world, sustainability and resource optimization are critical to creating smarter and more efficient buildings. We were inspired by the growing need to conserve energy, reduce maintenance costs, and enhance security in office and residential spaces. Our goal was to create a comprehensive smart building management solution that utilizes cutting-edge AI and IoT technologies to streamline building operations while reducing energy consumption and improving maintenance workflows.

What it does

Our project, SmartBuilding, leverages Azure IoT and AI /ML services to monitor and manage various aspects of building maintenance, security, and energy consumption. Key features include:

Energy Management: Using Azure Custom Vision, we detect the number of people in the building and adjust power consumption of systems like HVAC, lighting, security, and refrigerators based on occupancy, time of day, and day of the week.
Maintenance: A computer vision model detects overflowing trash bins, misplaced items for lost and found, and trash on the floor. Tickets are generated and sent to cleaning staff, while lost and found data is accessible to the building's help desk.
Security: The system continuously monitors the building and stores data in Cosmos DB, ensuring that building managers have real-time information on the state of the building.

How we built it

We utilized a variety of Azure services and modern web technologies to build SmartBuilding:

Azure Custom Vision: To train and deploy computer vision models for detecting people, trash, and lost items.
Azure IoT Hub: For connecting building devices and sensors to our system in real-time.
Azure Cosmos DB: As our database to store detected data and manage tickets for maintenance.
Azure Queue Bus: To manage task queues for sending alerts and notifications to cleaning staff.
Azure ML Services: To generate COCO files for custom vision model training.
Azure Blob Storage: To store images and other files related to the project.
React Web App: As the control center for monitoring and managing the building's energy systems.
Flask Backend: To handle the logic for task management, email notifications, and interactions with the IoT and AI services.
HTML, CSS: For building the UI components of the web app.

Challenges we ran into

Integrating multiple Azure services with real-time data processing was a significant challenge. Ensuring that the computer vision model accurately detected objects under various conditions was also a difficult task (training took the most time). We also faced hurdles with optimizing the system to make real-time decisions about energy management based on occupancy and time-sensitive data.

Accomplishments that we're proud of

We are particularly proud of how we managed to integrate Azure IoT and Custom Vision services into a seamless system. The creation of an efficient and responsive energy management system, which dynamically adjusts power consumption based on the number of people in the building, was a key achievement. Additionally, implementing a fully functional maintenance alert system that automatically generates tasks and notifies cleaners was a major success.

What we learned

Throughout this project, we gained a deep understanding of how Azure services can be combined to create a full-stack IoT and AI-powered solution. We also learned about optimizing real-time computer vision models, managing cloud resources effectively, and building scalable applications. Moreover, we learned the importance of collaboration and overcoming integration challenges across various platforms.

What's next for SmartBuilding

In the future, we plan to enhance the energy management system by incorporating predictive analytics to forecast energy usage patterns based on historical data. We also aim to expand the system’s capabilities by integrating more IoT sensors for air quality monitoring and additional security features like unauthorized access detection. Moreover, we hope to make the platform more scalable for larger buildings and more complex environments.