Robot Trash Simulation

• 

  Example of robot simulation in play.

Inspiration

After visiting Galveston recently and seeing how much trash is on the seashore, we decided to create something to aid in the autonomous cleanup process.

What it does

Simulates trash-cleanup scenarios for the robot to test and train itself in. The robot takes pictures and sends it over to an HTTP server, which hosts the model and spits back its best guess at the bounding box for all the trash it detects.

How we built it

We used flask to make the HTTP server, Unity to create the simulation, and YOLOv11 for the model. To train YOLOv11, We built the models using the TACO open-source dataset.

Challenges we ran into

Getting the model to be accurate, creating the simulation space, pipelining the model's data into a serializable format for unity, allowing framerate to persist while asynchronously processing image data.

Accomplishments that we're proud of

The simulation works, although not fully fledged out, the model is accurate, unity can read the JSON the model spits out, and the simulated robot is able to move toward the trash.

What we learned

How to train an AI properly, how to use unity, and the value of teamwork and friendship.

What's next for Robot Trash Simulation

Cleaning up the UI, so that the testers can see what the model is seeing more accurately, optimizing the model to become even more accurate, and creating more scenarios for the trash robot to test and train within.

Built With

• c#
• data
• flask
• python
• unity
• yolo