BigBackHacks - StructIQ

Inspiration

Progress tracking on construction sites remains largely manual. Site managers conduct walkthroughs, take photos, and compile reports by hand, which is a time consuming and subjective process, and we wanted to help that by making a product that makes this more accessible and also accurate.

What it does

Standard walkthrough video which can be shot on any phone or camera, and produces automated progress reports. The system analyzes footage frame by frame, identifying construction elements like beams, plumbing

How we built it

Built using VLMs with Kimi and claude vision, uses FastAPI to communicate with the backend and uses three.JS to create the 3d model that we display on our sites. We also use QR codes and used that help identify the elements in the videos.

Challenges we ran into

The primary challenge we faced was having the VLM be consistent across construction stages. Our model was able to assess construction builds that were near to complete reliably, however, the early-stage builds posed more issues as it exposed more complex designing features such as framing, partial pipe runs, and half-installed ductwork. Additionally, without the BIM files that construction sites use as input, the 3D positions we derived from 2D documentation were inherently approximate. If given these BIM files in a real-time project, we could use a lot more precision capabilities in our model.

Accomplishments that we're proud of

Getting the QR technology to work, and being able to connect our tasks with the system and the 3D model. Creating video to image to 3d model status system.

What we learned

We learned how to use different frameworks, and we learned how to create 3d models based on BIM files. Along with that we learned how to create to do lists based on task lists in the construction sites.

What's next for BigBackHacks - StructIQ

BIM Integration: Accepting industry-standard IFC files for their BIM and creating architecturally accurate 3D models with precise element-level comparison against the accurately-designed state. Also exploring 3D Gaussian splatting to reconstruct photorealistic site models directly from walkthrough video. Unlike our current reference models, Gaussian splats would produce visually accurate 3D representations of the as-built environment more efficiently, as well as enabling more precise side-by-side comparison between current site conditions and the design intent.