Bauxite

Inspiration

Our project originally aimed to improve recycling efficiency through the development of a trash image classifier. However, during our exploration, we stumbled upon a novel idea - gamification. We realized that incentivizing individuals to recycle and educating them on recyclable materials could lead to increased recycling rates. By making recycling a more engaging and rewarding experience, people are more likely to participate. This increase in recycling rates could lead to a positive impact on the environment by conserving natural resources and promoting a circular economy. Through gamification, we can create a culture of recycling that encourages individuals to participate in a fun and interactive manner. With proper education and incentives, people will not only recycle more frequently but will also be more mindful of what is and isn't recyclable. This shift in behavior can lead to long-term benefits for the environment and the economy. By reducing waste and conserving resources, we can move towards a more sustainable and equitable future.

We came up with the name Bauxite based on the ore that the commonly recycled material, aluminum, is harvested from.

What it does

Our program combines both the recycling detection and recycling game play into one big program by giving the user rewards when they scan a recyclable item in our database (detected by the to-be-trained neural network). These rewards would be given as points which could upgrade the game play of the retro style video game. Perks can be earned through using the image classifier to recognize more materials. This provides incentive to get users excited about recycling. It also helps to educate users about what can be recycled and what can't.

How we built it

Originally, as we were brainstorming ideas and possible routes for the project, we were considering two separate tracks: object detection of recyclable materials and a videogame about promoting recycling. After discussing these different tracks, we decided on the combination of both tracks into the project. The entire project was then broken down into three main parts based on experience. We had one team member working on the video game, two teammates working on the website and data interactions within, and one teammate focusing on the recycling detection.

The game was built using python and its pygame package. Since the gameplay of the game includes picking up recyclables and avoiding objects like grandma and a puppy, we either used Dall-E or used an online pixel art canvas to create each object. Originally, we wanted to have both the game and the scanning on a website. However, the website was causing too much difficulty. After about eight hours of working on the website, we decided to create a local GUI using tkinter as the main menu for both the scanning and game until we can get the website up and running.

The website’s backend was built using flask and python, we decided to configure it to store user uploaded files that would be accessed by the game to determine how many points they would receive from their recyclable. The frontend of the website was built using HTML and CSS. The design that we decided on for the visuals of the site was a vaporwave style, with page contents formatted similar to Windows 95 elements. On the homepage, the user is able to access the page to upload files that would be stored, as previously mentioned, as well as the game page and resources for the data and style documents.

For the detection of recyclables, we considered some untrained neural networks such as SSD, RetinaNet, and yolov5 but decided to settle on yolov5 for its impressive documentation and number of configurations available. After choosing the yolov5m (yolov5 medium model), we found a dataset which contained images of bottles and other recyclables in a trash bin. Using these images as well as the location and confidence of each object in the image, we were able to train this model with the number of epochs set to 4 and batch size of 10. These are relatively small values for training purposes, and results often may not accurately reflect the truth because of this. However, due to the excellent quality of dataset and high quality of the images that were to be scanned through this program, we hypothesized that the detection model would run sufficiently with the given epoch and batch sizes. After running the training for upwards of three hours on the dataset, our hypothesis proved to be correct. The network successfully detects water bottles of different colors and shapes, cans, and more.

After the finalization of the neural network, we created a gui which opens a camera video, then allows the user to take a photo whenever they are ready. This photo is then run through the detection model and displayed. Using custom points values for each type of recyclable, the user receives the associated number of points for that object.

Challenges we ran into

With regards to the training of the neural network, we wanted to sufficiently train the network. While the detection works, the confidence levels are very low. We believe this is because of the very small number of epochs used to train the model. A typical epoch value is 100, which is 25 times more than what we were able to run, with time being the limiting factor. If we were to train at 100 epochs, the training would have taken over 72 hours. If we had more resources (ie more time or a more powerful machine), we could have created a more confident model.

When the user displays a bottle, the algorithm often detects this bottle as multiple objects and therefore classifies the bottle as different types. If there is actually more than one bottle, it is very hard to determine how many bottles there are on the data side, given that one bottle can be detected as multiple bottles anyways. To counteract this, we only want the user to show one bottle at a time. In addition, we took the detection with the highest confidence as the most probably correct assignment and returned the respective point value and name.

Accomplishments that we're proud of

As is often the case in software development, the things we thought would be hard can end up being easy and things we thought would be easy end up being hard. Some of the things we are proud of include just getting a simple image to be displayed on the web page, getting the server up and running, and finding a decent background image for the game. Of course, one of the things we are most proud of is the accuracy and the usefulness of the image classifier.

What we learned

We collectively learned more about how important recycling is for progress in the sphere of environmental protection and preservation. We explored the process of fine tuning a neural network for usability and efficiency. We were all exposed to some novel design concepts in terms of visual layout and UI themes. Though none of us had any significant experience with web development, we came together as a team and figured it out. We worked to troubleshoot each other’s problems and figure out how to contribute to the team as a whole. Finally, we discovered the many thorny problems in integrating many disparate elements into one functioning system.

What's next for Bauxite

There are many different points to improve upon before Bauxite can be considered a finished product. The main thing would be further integrating the different pieces into one system. Right now, the game and the classifier are separate from the website. We would like to create a mobile version which can access the user’s phone camera, rather than a desktop camera. Another improvement would be creating “upgrades” in the game that are only accessed by recognizing more images of recyclable items. Anything past that point would be the cherry on top, but it would also be desirable to improve the neural network’s accuracy.

Built With

• flask
• javascript
• opencv
• python
• tkinter
• warp
• yolov5