-
Getting the weight layers for the YOLOv3. (CNN)
-
Before of a normal pic we tested from google images of a forrest.
-
After using our trained YOLOv3 dataset, we classified it as plants.
-
detecting phone and person, single image shot.
Inspiration
The inspiration behind this project was a new and innovative application for object detection (Darknet YOLOv3 --> https://pjreddie.com/darknet/yolo/) and warning system audio (from Google home mini, just for purposes of demonstration).
What it does
So our project is a computer vision ML algorithm (CNN) which scans the space for objects and provides a warning when near an object. Using Tensorflow, Keras, and the Darknet’s neural network framework we were able to train a dataset of over 250 keys corresponding to approximately 80 classes each corresponding to various objects in regular space such as a person, car, table, chair, etc.
How we built it
Background: COCO dataset - contains a list of 80 classes which are various objects. YOLO “you only look once” CNN object detection good for real-time detection. Pretrained Convolutional weights of yolov3 -- open images file which contains 600 classes of various objects.TensorFlow library for dataflow used for backend.OpenCV a library mainly aimed at real-time computer vision. Imageai library used for vision detection. Keras is an open-source neural network library written in Python. First, we trained our Yolov3 file using the pretrained CNN weights of yolov3 -- openimages which helps in classifying objects in a image through the darknet. We also had began forming the calculations of getting the various areas of the bounding boxes and then based on if the derivative and second derivative is positive correlates with if object is moving closer to the camera or further away. (in theory) This was for the distance however, with lack of time were unable to finish this component. There is an alert feature which will say "Alert" when near camera's focal length area.
Challenges we ran into
We encountered many challenges. Firstly, imageai library required later macOS and windows versions and some team members didn’t have it so they couldn’t run the program. Our first attempt was to use the webcam with the Raspberry Pi but it was giving an error for imageai library. We also tried using google home mini as the voice assistant for making an alert sound but after a lot of attempts, it didn’t work as well as we plan. After computing final rates for the bounding box area, we still had issues with determining distance from the webcam to object, it would also require another camera. Nevertheless, in the end, our hack is working properly and kind of the way we wanted it to be.
Accomplishments that we're proud of
A functioning product with a trained machine learning algorithm with the ability to classify over 80 objects using Darknet and YOLOv3. It was our first time using my different python libraries like imageai, keras, opencv and tensorflow. It was great accomplishments to learn those and used those properly for our hack and make a final working product out of it felt great. On the hardware side, it was an accomplishment to set up the Linux based OS on the rasberry pi and begin with downloading the libraries (except one of them). There were many challenges that our original idea of working raspberry and google home mini didn’t seem like working we adapted to our situation and finally made our hack without using them.
What we learned
We learnt about how we can train yolov3 using darknet. We used raspberry pi for the first time and we couldn’t able to run it but due to some complications, we didn’t use it in our project. We used many different python libraries like imageai, keras, opencv and tensorflow. It was great learning experience on training the algorithm and working with the CNN (convolutional neural network) and seeing how the objects are classified in real time in space.
What's next for IVision
Continue to work with distance and able to do functionality with belt for the visually-impaired person to wear.
VIDEO: https://drive.google.com/file/d/1eKOZB21me_v7vKMQj6mcUgUBTkwJM-4U/view?usp=sharing
Log in or sign up for Devpost to join the conversation.