Open-source auto-zoom and physics-based frame settling algorithms for building Screen Studio-style animated screen recordings.

No UI, no video processing — just the core math. Bring your own renderer.

Screen Studio's signature effect: the camera smoothly zooms in when you're clicking and typing, follows your cursor with spring physics, and zooms out during pauses. This repo contains the algorithms that make that work.

Raw cursor events
  → Shake filter (remove jitter)
  → Densify (fill time gaps)
  → Spring simulation (smooth trajectory)
  → Auto-zoom detection (silence analysis)
  → Per-frame crop evaluation (spring-animated viewport)

Spring-mass-damper physics — All animations use exact analytical solutions to the damped harmonic oscillator ODE (not Euler/RK4). This means:

• Stable at any frame rate
• Frame-rate-independent results
• Three regimes: underdamped (bouncy), critically damped (fast settle), overdamped (slow settle)

Cursor smoothing — Three spring profiles switch based on interaction context:

• default (tension=170, friction=20): natural follow
• snappy (tension=700, friction=30): tight response within 160ms of clicks
• drag (tension=136, friction=26): heavier feel during mouse drag

Viewport panning — The viewport locks on segment entry and only pans when the cursor exits a "safe zone" (inner 70%). Before panning:

1. Lookahead jitter cancellation: checks 1 second ahead — if the cursor returns to the safe zone, the pan is skipped
2. Trajectory-averaged targeting: the pan target is averaged over 0.5s of future cursor positions

Zoom transitions — Spring easing between zoom levels with different curves for zoom-in (slightly bouncy) vs zoom-out (softer, more damped).

npm install screen-studio-effects

import {
  buildSmoothedCursor,
  detectSilenceZones,
  generateAutoZoomSegments,
  createZoomState,
  evaluateZoom,
  VP_PRESETS,
} from 'screen-studio-effects'

// 1. Smooth raw cursor events into a precomputed trajectory
const cursor = buildSmoothedCursor(rawCursorEvents, {
  windowX: 0,        // window origin in screen coords
  windowY: 0,
  captureWidth: 3840, // capture dimensions in pixels
  captureHeight: 2160,
})

// 2. Auto-detect where to zoom based on cursor activity
const silences = detectSilenceZones(rawCursorEvents)
const segments = generateAutoZoomSegments(silences, totalDurationSecs)

// Or define segments manually:
// const segments = [
//   { sourceStart: 2.0, sourceEnd: 5.0, amount: 2.0 },
//   { sourceStart: 7.0, sourceEnd: 9.0, amount: 1.5, manualCenter: [0.3, 0.4] },
// ]

// 3. Evaluate per frame — returns crop bounds in UV space [0,1]
const state = createZoomState()

for (const frame of frames) {
  const pos = cursor.interpolateAt(frame.time)
  const crop = evaluateZoom(
    segments,
    frame.time,
    pos,
    state,
    (t) => cursor.interpolateAt(t),  // lookahead for jitter cancellation
    VP_PRESETS.focused,               // or VP_PRESETS.smooth
  )

  // crop = { x, y, w, h } in normalized 0-1 space
  // Full frame: { x:0, y:0, w:1, h:1 }
  // 2x zoom centered: { x:0.25, y:0.25, w:0.5, h:0.5 }

  // Apply to your renderer:
  // const srcRect = {
  //   x: crop.x * sourceWidth,
  //   y: crop.y * sourceHeight,
  //   w: crop.w * sourceWidth,
  //   h: crop.h * sourceHeight,
  // }
}

The spring feel is controlled by three parameters:

The damping ratio ζ = friction / (2 * √(tension * mass)) determines the character:

• ζ < 1: Underdamped — bouncy, oscillates around target
• ζ ≈ 1: Critically damped — fastest approach without oscillation
• ζ > 1: Overdamped — slow, no oscillation

Built-in presets:

// Viewport panning springs
VP_PRESETS.focused  = { tension: 300, mass: 6.75, friction: 120 }  // ζ ≈ 1.33
VP_PRESETS.smooth   = { tension: 240, mass: 3.375, friction: 80 }  // ζ ≈ 1.40

// Cursor smoothing springs
SPRING_DEFAULT = { tension: 170, mass: 1.0,  friction: 20 }  // ζ ≈ 0.77 (bouncy)
SPRING_SNAPPY  = { tension: 700, mass: 1.0,  friction: 30 }  // ζ ≈ 0.57 (very bouncy, fast)
SPRING_DRAG    = { tension: 136, mass: 1.2,  friction: 26 }  // ζ ≈ 1.02 (critically damped)

A standalone Rust port is included in rust/ for server-side rendering / export pipelines:

cd rust && cargo build

src/
  types.ts      — Core type definitions
  spring.ts     — Spring-mass-damper physics (analytical solver)
  cursor.ts     — Cursor smoothing pipeline (shake filter → densify → spring)
  auto-zoom.ts  — Auto-zoom segment generation from cursor silence analysis
  zoom.ts       — Main zoom evaluator (spring-animated viewport + jitter cancellation)
  index.ts      — Public API barrel export

rust/src/       — Rust reference implementation (same algorithms)
examples/       — Demo with synthetic cursor data

The spring physics and zoom interpolation algorithms are ported from Cap, an open-source screen recording tool. Cap's implementation is itself inspired by Screen Studio.

MIT