Skip to content

Latest commit

 

History

History

README.md

VRM Cast Plugin — Babylon.js Editor v5

A plugin for Babylon.js Editor v5 (BJSE) that adds a VRM Cast panel to the editor UI. Use it to assemble a cast of VRM avatars, assign VRMA animation clips, set world positions, and build a timeline — then export a scenes/cast.json that the PlayController runtime reads.

BJSE handles:              VRM Cast plugin handles:
  Lights                     Pick which VRM model each actor uses
  Camera                     Pick which VRMA clip plays at t=0
  Environment / sky          Set actor world positions
  Ground / props             Schedule timed clip changes
  Non-VRM animation          Export cast.json → PlayController reads it

Quick start

1. Build the plugin

cd bjse-plugin
npm install
npm run build

This compiles TypeScript to bjse-plugin/build/index.js (CommonJS, no bundler).

2. Register in BJSE

  1. Open Babylon.js Editor, open or create a project
  2. Edit → Project... → Plugins tab
  3. Click "From local disk" → navigate to and select the bjse-plugin/build/ folder
  4. The VRM Cast tab appears next to the Assets Browser
  5. Save the project — the plugin registration is stored in your project's .json config

3. Copy assets into your project's public/ folder

public/
  models/          ← put .vrm files here
    Alice.vrm
    Bob.vrm
  vrma/            ← put .vrma files here
    walk.vrma
    wave.vrma
  scenes/          ← cast.json will be written here by the plugin

4. Use the VRM Cast panel

Button Action
↺ Scan Re-scan public/models/ and public/vrma/ for files
+ Add Actor Add an actor row; pick VRM, set position, assign starting clip
+ Add Event Schedule a clip change or stop at a future time (seconds)
Export cast.json Write public/scenes/cast.json

5. Connect to PlayController

In your BJSE scene, attach vrm-startup.ts to a TransformNode and set:

scriptUrl = scenes/cast.json

Press Play in BJSE — actors load, animate, and respond to the timeline.


Panel reference

Actor row

Field Description
id Unique string identifier used in timeline events
VRM dropdown .vrm file from public/models/
pos x y z World-space position in metres
rotY° Y-axis rotation in degrees
clip at t=0 VRMA clip to start immediately on play
loop Whether the t=0 clip loops

Timeline event row

Field Description
t= Time in seconds when the event fires
Actor dropdown Which actor receives the event
Action animate — play a clip; stop — halt animation
Clip dropdown VRMA clip (animate only)
loop Whether the new clip loops (animate only)

Actor initial clips are automatically added as t=0 timeline events on export — you don't need to add them manually.

Exported cast.json format

This is the SceneScript format consumed by PlayController:

{
  "metadata": { "title": "VRM Cast", "description": "Generated by VRM Cast Plugin" },
  "actors": [
    {
      "id": "alice",
      "vrm": "models/Alice.vrm",
      "startPosition": { "x": -1, "y": 0, "z": 0 },
      "startRotation": { "y": 0 }
    }
  ],
  "timeline": [
    { "start": 0,   "actor": "alice", "action": "animate", "clip": "vrma/walk.vrma",  "loop": true },
    { "start": 5.5, "actor": "alice", "action": "animate", "clip": "vrma/wave.vrma",  "loop": false },
    { "start": 8.0, "actor": "alice", "action": "stop" }
  ]
}

Development workflow

Hot-reload

BJSE watches locally-registered plugins with chokidar. After any build/index.js change it calls close(editor), flushes require cache, then calls main(editor) again — no BJSE restart needed.

# Terminal 1 — keep running while editing
npm run watch

# Make changes to src/ui/VrmCastPanel.tsx
# → tsc rebuilds → BJSE reloads the panel in ~3.5 seconds

Project structure

bjse-plugin/
  package.json          deps: fs-extra, react, react-dom
                        devDeps: typescript + @types/* only (19 packages total)
  tsconfig.json         module: CommonJS, outDir: build/, jsx: react-jsx
  src/
    index.tsx           Plugin entry point — exports main() and close()
    ui/
      VrmCastPanel.tsx  React component — all panel UI and export logic
    types/
      babylonjs-editor.d.ts   Minimal type stub (we do NOT install the full
                              editor package — BJSE provides itself at runtime)
  build/                Generated by tsc (gitignored)
  declaration/          Generated .d.ts files

Why no babylonjs-editor npm dependency? The full babylonjs-editor npm package is the entire Electron editor (~hundreds of deps). Installing it would take minutes and gigabytes. Instead, src/types/babylonjs-editor.d.ts declares just the Editor, EditorLayout, and EditorConsole interfaces this plugin uses. BJSE calls require(pluginPath) from within its own process — the real Editor object is passed in as a function argument, no import needed at runtime.

Plugin contract

BJSE loads build/index.js via Node require(). The module must export:

export const title = "VRM Cast Plugin";   // shown in editor console

export function main(editor: Editor): void {
  // called when plugin loads — add your tab here
  editor.layout.addLayoutTab(<YourPanel editor={editor} />, {
    id: "unique-tab-id",
    title: "Tab Label",
    neighborId: "assets-browser",   // dock next to this panel
    enableClose: false,
  });
}

export function close(editor: Editor): void {
  // called before hot-reload or project close — clean up
  editor.layout.removeLayoutTab("unique-tab-id");
}

Key editor APIs used by this plugin

API What it does
editor.state.projectPath Absolute path to the .json project file
editor.layout.addLayoutTab(node, opts) Add a React panel tab
editor.layout.removeLayoutTab(id) Remove a tab
editor.layout.console.log(msg) Write to the BJSE console
editor.layout.console.error(msg) Write an error to the BJSE console
editor.layout.preview.scene The live Babylon.js scene (for future use)

How BJSE loads plugins

The plugin system was completed in BJSE v5.3.x (Issue #748).

  1. Plugin names/paths are stored in the project's .json config:
    { "plugins": [{ "nameOrPath": "/absolute/path/to/bjse-plugin/build" }] }
  2. On project open, BJSE calls require(nameOrPath) for each entry
  3. If nameOrPath is a local path, BJSE watches it for changes (hot-reload)
  4. If nameOrPath is a package name, BJSE resolves it from <projectDir>/node_modules/

The official Fab plugin source in the BJSE repo is the canonical reference implementation.


Limitations and roadmap

Current Planned
Position typed as numbers Sync position from TransformNode in BJSE scene
Clips from flat public/vrma/ scan Recursive scan + sub-folder grouping
Export only Import existing cast.json back into panel
t=0 + timed events Preview timeline playback inside BJSE
No A2F support Phase 3: morph proxy — A2F ARKit JSON → blend shapes