LEGOGEN

voxel and legos
sample input

Inspiration

Anyone who owns Legos also owns a miscellaneous collection of pieces, a place where Lego are left to collect dust, never to be reunited with their brethren. We decided to provide a use for these forgotten souls, using computer vision, AI tools, and heuristic-guided algorithms to transform them into any given form.

What it does

Our website inputs a text description and an image of a sprawl of Legos and returns a design with instructions and visualizer of the described Lego set using those Legos.

How we built it

We built a full-stack application featuring a Vue.js web interface and a Python backend powered by FastAPI. The system simultaneously uses a computer vision model to inventory your bricks from an image and an AI service (Meshy) to generate a 3D model from your text description. Then, a custom algorithm converts that model into a voxel grid and generates step-by-step building instructions tailored to your specific brick collection, using a complex heuristic to guide assembly.

Challenges we ran into

One challenge we faced was the lack of model and data for identifying bricks, so we had to bring together multiple hand-trained ML models and ML APIs to accurately take inventory of the pieces.

Accomplishments that we're proud of

We are proud of the algorithm we use to put together the pieces to fill the voxel space. We derived these rules from the base up and learned a lot about optimization and accuracy for complex algorithms.