Pointer Realms

A 3D Educational Game for Visualizing Pointers, Memory, and C Programming Concepts

Comment

Inspiration

Learning C programming, especially pointers and memory management, is notoriously difficult. Traditional tools like Python Tutor provide valuable step-through debugging, but they lack the engagement and intuition that comes from interactive visualization. We were inspired by games like Return of the Obra Dinn and puzzle games that make you feel the logic rather than just read it.

We asked ourselves: What if understanding pointers felt like commanding a catapult? What if memory allocation was as tangible as placing characters on a battlefield? That's how PointerRealms was born - a 3D interactive game that transforms abstract C concepts into physical, manipulable game entities.

What it does

PointerRealms is an educational 3D puzzle game that teaches C programming concepts through interactive gameplay. Players execute C code by manipulating game characters on a grid, where each action corresponds to a line of code.

Core Mechanics

C Types = Fantasy Characters:

  • int → Necromancer (summons entities for integer values)
  • char → Lumberjack (handles byte-sized tasks)
  • short → Mage (transforms for short integers)
  • void* (pointer) → Catapult (links memory cells, fires projectiles)

Gameplay Flow:

  1. A C code snippet is presented (e.g., int x = 5; int *p = &x;)
  2. Players drag characters from a portrait tray onto a 3D grid
  3. Each placement is validated against the current line of code
  4. Correct placements advance through the code, wrong placements get feedback
  5. Complete all lines to finish the level!

Advanced Features:

  • Pointer dereferencing: Catapults "fire" projectiles to modify target variables
  • Array operations: Fields visualize contiguous memory with index-based access
  • Reference operations: Visual connections show pointer relationships
  • Camera controls: Switch between bird's-eye and character perspectives (C key)

How we built it

Tech Stack

  • Frontend: React 19 + TypeScript
  • 3D Rendering: Three.js 0.170
  • Build Tool: Vite 7

Architecture

1. Three.js 3D Scene (The Body)

  • Custom terrain system with jagged procedural geometry
  • GLTF model loading for characters (Necromancer, Catapult, Mage, Lumberjack)
  • Real-time physics for catapult projectile trajectories
  • Selection system with keyboard controls (E/Q for navigation, Space for targeting)
  • Dynamic camera controller with perspective switching

2. Integration Layer (The Nervous System)

  • Event-driven architecture connects React UI to 3D gameplay
  • PortraitSlots handles drag-and-drop with raycasting
  • Terrain registration system tracks unit placements
  • Action validation pipeline ensures code correctness

Development Process

  1. Prototyped 3D scene with basic terrain and character models
  2. Designed type mappings (C types → fantasy characters)
  3. Created level system with C code parsing
  4. Integrated everything with event-based communication
  5. Polished UI/UX with camera controls and visual feedback

Challenges we ran into

1. Merging Parallel Development

We split into frontend and game teams, which led to massive merge conflicts when combining codebases. We resolved this by:

  • Carefully reviewing duplicate code blocks
  • Implementing a unit registration system for terrain validation
  • Creating a shared integration context document

2. Bridging React and Three.js

React's declarative paradigm clashed with Three.js's imperative scene graph. We solved this with:

  • A dispose() pattern for cleanup on unmount
  • Custom useEffect hooks for canvas lifecycle
  • Event emitters to propagate game state to React

3. Validating Complex C Operations

Mapping drag-and-drop actions to C semantics was non-trivial. For example:

  • int *p = &x; requires creating a catapult entity AND linking it to an existing variable
  • *p = 5; requires finding the catapult, traversing its connection, and updating the target

We built a flexible ActionResolver with 11 specialized handlers (declare, reference, deref_write, deref_read, array_access, assign, swap, move, increment, decrement, place).

4. Grid Position Inference

The 3D world uses continuous coordinates, but code validation needs discrete grid positions. We implemented:

  • Raycasting from mouse to terrain tiles
  • World-to-grid coordinate mapping
  • Fallback position inference using spacing calculations

5. Action Format Mismatch

The 3D layer sends { type: 'place', entityType: 'catapult', ... } but levels expect { operation: 'assign', variable: 'x', value: 5 }. This is currently logged but not fully resolved - a learning opportunity for modularity!

Accomplishments that we're proud of

🎨 Intuitive Type Metaphors

We turned abstract concepts into memorable characters:

  • Pointers aren't just "addresses" - they're catapults that fire projectiles across memory cells
  • Arrays aren't just "contiguous data" - they're fields you can traverse and rearrange

🎮 Smooth 3D Experience

  • 60 FPS animation loop with delta-time updates
  • Procedurally jagged terrain for organic feel
  • Realistic catapult physics with parabolic trajectories
  • Camera system with 2 perspectives + smooth transitions

🔧 Production-Ready Patterns

  • TypeScript for type safety
  • Modular component architecture
  • Error handling with try-catch and validation
  • Build optimization with Vite

What we learned

Technical Skills

  • Three.js mastery: Scene graphs, raycasting, materials, lighting, camera controls
  • Event-driven architecture: Decoupling systems with pub/sub patterns
  • State management: Building a custom state engine from scratch
  • Code parsing: Regular expressions, AST-like structures, metadata annotations
  • 3D math: Vector operations, quaternions, coordinate transformations

Soft Skills

  • Team coordination: Merging parallel workstreams without breaking changes
  • Scope management: Prioritizing core features over polish (we wanted multiplayer but focused on single-player first)
  • Documentation: Writing clear guides for future contributors
  • User empathy: Designing intuitive mappings (C types → characters)

Domain Knowledge

  • C memory model: Deep dive into pointers, references, dereferencing, arrays
  • Educational game design: Balancing challenge and feedback
  • WebGL/Three.js performance: Optimizing render loops, geometry, textures

What's next for PointerRealms

Short-term (Next Sprint)

  1. Finish action format integration - Map place actions to current line operations
  2. Add visual feedback - Green highlights on success, red on failure
  3. Implement proper grid tracking - Store grid positions in terrain userData
  4. Create 5 more levels - Cover pointers, arrays, structs, malloc/free
  5. Add level progression UI - Show completion stats, unlock next level

Medium-term (3 Months)

  1. Advanced C concepts:
    • Pointer arithmetic (p++, p + 3)
    • Multi-dimensional arrays (int grid[3][3])
    • Linked lists (next pointers)
    • Function pointers (callback mechanics)
  2. Competitive mode: Speedrun levels, leaderboards
  3. Level editor: Community-created puzzles
  4. Mobile port: Touch controls for drag-and-drop

Long-term (Vision)

  1. Multiplayer co-op: Two players execute code together
  2. Boss battles: Complex algorithms as epic encounters (e.g., "Sort the Array Dragon")
  3. Story mode: Narrative-driven curriculum from variables to data structures
  4. University partnerships: Integrate into CS101 courses as supplemental material
  5. Expand to other languages: Rust ownership system, Assembly registers

Dream Feature: "Memoria" - The Memory Dungeon

A procedurally generated dungeon where each room is a memory puzzle. Stack frames are literal floors, heap allocation spawns treasure chests, and memory leaks create haunted corridors. Defeat the final boss: Segmentation Fault, Lord of the Null Pointers.

Share this project:

Updates