200-SF38 CadXR

Inspiration

Structural engineering tools still rely on flat 2D screens, making it difficult to understand how forces travel through 3D space. At the SensAI Hackathon, we set out to merge immersive XR interaction with AI-driven structural reasoning to make stress analysis intuitive, visual, and experiential.

What it does

CadXR is an immersive WebXR CAD and structural analysis tool for VR/AR. Users can construct block-based structures, apply forces with precise 3D placement, and instantly visualize stress propagation through pixelated gradient maps. AI agents transform natural-language prompts into generated structures, run multi-step stress reasoning, produce technical explanations, and output detailed engineering summaries—all inside an interactive XR workspace.

How we built it

We built the frontend with Three.js, WebXR, and TypeScript to support both VR/AR modes and desktop fallback. For AI-driven features, we created a multi-agent pipeline handling structure generation, stress reasoning, math derivations, visualization mapping, and verification. Custom DataTexture rendering powers the pixelated stress maps, and template-based geometry ensures deterministic, error-free structure creation. MathJax rendering and PDF export support technical reporting.

Challenges we ran into

Meta Quest requires strict HTTPS for WebXR sessions, rejecting self-signed certificates; we solved this through adb reverse port forwarding. Our AI agents occasionally produced malformed JSON or unescaped LaTeX, so we built a multi-layer recovery system to repair or partially extract valid data. Rendering HTML UI in VR wasn’t supported natively, prompting us to create 3D floating panels using CSS3DObjects.

Accomplishments that we're proud of

We achieved a seamless desktop/VR/AR experience with adaptive UI, a robust five-agent analysis pipeline, and a unique pixelated stress visualization engine that maps force propagation at the sub-block level. Our deterministic geometry system removed all hallucinations in structure generation, and our error-recovery pipeline ensured the app remained stable even with malformed AI responses.

What we learned

We learned how to manage WebXR sessions, controller input, and passthrough rendering on headsets. We gained experience enforcing strict JSON schemas for AI agents, repairing inconsistent outputs, and building texture-based stress visualization using nearest-neighbor filtering. We also learned that deterministic templates are critical for preventing AI errors in geometry.

What's next for CadXR

We plan to support advanced geometries like beams and trusses, import CAD models, introduce real-time multi-user collaboration, add material property customization, and expand the AI agents to provide optimization suggestions and full structural design recommendations.