Package Cop

Package Delivery Monitoring

Comment 1

Inspiration

As students, we sometimes live with many other people in one house with lots of Amazon or other package deliveries daily. Some of us have experienced missing deliveries and want a solution to prevent these. We came across the product "Ring Doorbell" however with 9 people living in one house and friends coming over constantly, ring sends notifications for all motions near entrance. We want to identify the isolated events of package deliveries and package thefts. We call our solution Package Cop

What it does

Package Cop is Machine Learning based and detects 3 states of the package: 1) Delivered 2) Stolen 3) Brought Indoors

Changes to these states notifies the user using a text message with the state and a photo.

How we built it

The best way to identify package theft is human behavior tracking. To simulate the scenario of monitoring theft at your door, we built an IOS app to change our phone into a Live Camera Feed. However our ML pipeline can use any video source. Our behavior monitoring algorithm is hosted on a flask server in a GCP Virtual Machine which enhances and reduces the noise of outputs from the Google Computer Vision API. Our Pipeline then responds to the user using a combination of the Twilio API and our Security Dashboard Client which we built with React.

Challenges we ran into

Behavior understanding is not easy. We want it to be real time and accurate as much as possible. We tried Amazon AWS Rekognition at first which has an incompatible feature between its demo and real API. That's when we cannot get accurate object tracking result from image APIs. Then we changed to using real-time images and Google Vision API.

We also need to address the need of behavior understanding, we did a lot of experiment and designed an intelligent algorithm to tell from package delivery, theft and taking away. Server configuration and website designing are not easy. We tried a lot and finally turned to RESTful APIs to be compatible with each other's work.

Accomplishments that we're proud of

We built the first package detection algorithm that detects isolated events of delivery and package removal (delivery acceptance and theft states). Currently the market of cameras and softwares only has motion detection. We are super happy to have done this in a short time and made good friends. None of us worked together before and our team chemistry was awesome!

What we learned

The most important thing we learnt is teamwork. We came from everywhere and had different skills, we made a lot of effort in integrating our work as a full-stack accurate package theft monitoring application. Apart from that, we learnt to play with state-of-the-art machine learning algorithms and computer vision pre-trained APIs for object tracking and human behavior monitoring.

What's next for Package Cop

Scale out our pipeline as a an API for doorbell camera manufacturers and home camera systems.

Built With

Share this project:

Updates