IdeaWall

An overview of working Project

---> Inspiration We were constantly frustrated by the disconnect between discussion and action. When key decisions were decided in a meeting, they were lost in a lengthy transcript or it took hours of manual synthesis to convert them into actionable steps in a planning tool. We wanted to eliminate that time waste and have ideas materialize directly onto a shared surface. Our goal: transform discussion into formatted AR/2D tasks.

---> What it does IdeaWall is an AI-powered and augmented reality collaborative planning board. It operates in three primary stages:

  1. Speech-to-Plan: Captures real-time speech using the Speech-to-Text (STT) API of the browser, forming a rolling transcript.
  2. Intelligent Structuring: Flows this transcript into a Gemini AI module (aiPlanGemini.ts) that generates an automatically built structured work plan (tasks, lanes, and dependencies).
  3. Spatial Visualization: Shows the resulting plan as 3D sticky notes through WebXR (WebXRScene.tsx) for VR experience, with a 2D canvas fallback for wider accessibility.

---> How we built it We used a React and TypeScript (Vite) frontend layout focused on modularity:

  1. The Conductor: 'src/App.tsx' directs the core application flow, integrating the data pipelines.
  2. Front-End: We isolated the AR/3D visualization logic (src/ar/) from the rest of the app via Three.js under WebXR so that the interface is fast and extremely graphical.
  3. Back-End: We had a clear data structure (src/types.ts) and scalable local persistence layer (src/lib/db.ts) implemented in such a manner that they can be replaced instantly with cloud-based services like Firestore/Firebase.
  4. Deployment: We deployed with Cloudflare Pages for CI/CD and automatic deployment of our static assets.

---> Challenges we ran into 1. TypeScript Typing for APIs: Combining browser-native APIs (like the Web Speech API) and third-party libraries (like localforage) into a strict TypeScript setup required installing and configuring several '@types/' declarations manually in order to have a clean build. 2. CI/CD Configuration: We first faced a deployment failure on Cloudflare Pages because the build process could not identify the root package.json file. And it was fixed by configuring the Root Directory within the Pages config to where the main app resides. 3. Plan Synchronization Logic: Having concluded applyPlanToWall ('planApply.ts') created stable, non-duplicate NoteDocs, and clearAppliedPlan removed only AI-generated notes, translated to careful implementation of data keys and idempotency checks.

---> Accomplishments that we're proud of End-to-End Functionality: Having an entire, live pipeline that takes speech → runs it through AI → and then spits it out as interactive 3D objects—an entire, functioning proof-of-concept. Technical Feasibility: Demonstrating with success that a complex WebXR and AI stack can be built and safely deployed within months utilizing bleeding-edge web tech. Clean Architecture: Developing a highly modular configuration (seen in the isolated dataconnect, functions, and src folders) that is scalable and ready for full−stack integration.

---> What we learned We gained some valuable insights into the unique problems of building a spatial web application. These were: Enlarged knowledge of WebXR lifecycle management (initiating and ending the immersive session). Getting proficient in how to configure Vite and TypeScript for a static-hosting scenario like Cloudflare. The convenience of having type-safe interfaces (src/types.ts) early on to handle the heavy data model between the AI, the local DB, and the 3D renderer.

---> What's next for Idea Wall Cloud Backend Integration: Replacing the local DB (db.ts) with a live, scalable database like Firestore for multi-user collaboration. Firebase Cloud Functions: Host AI processing within Firebase Functions (functions/ directory) to manage API keys and disengage heavy computation from the client. Multi-User AR: Support real-time shared state management so that various users can collaborate with the 3D board concurrently within the same virtual environment.