Playwright MCP, but for TUI apps.

MCP server for headless TUI automation. Enables LLMs to run, view, and interact with terminal applications through the Model Context Protocol (MCP).

cargo install --git https://github.com/michaellee8/mcp-tui-driver

{
  "mcpServers": {
    "tui-driver": {
      "command": "mcp-tui-driver",
      "args": []
    }
  }
}

{
  "mcpServers": {
    "tui-driver": {
      "command": "/path/to/.cargo/bin/mcp-tui-driver",
      "args": []
    }
  }
}

{
  "mcpServers": {
    "tui-driver": {
      "command": "mcp-tui-driver",
      "args": []
    }
  }
}

{
  "experimental": {
    "modelContextProtocolServers": [
      {
        "transport": {
          "type": "stdio",
          "command": "mcp-tui-driver",
          "args": []
        }
      }
    ]
  }
}

{
  "mcpServers": {
    "tui-driver": {
      "command": "mcp-tui-driver",
      "args": []
    }
  }
}

{
  "mcp_servers": {
    "tui-driver": {
      "command": "mcp-tui-driver"
    }
  }
}

• Launch and manage multiple TUI sessions concurrently
• Read terminal content as plain text or accessibility-style snapshots
• Send keyboard input (keys, text, modifier combinations)
• Mouse interaction (click, double-click, right-click)
• Wait for screen content or idle state
• Take PNG screenshots of terminal output
• JavaScript scripting for complex automation workflows
• Session management (resize, signals, info)
• Session recording to asciicast format for playback with asciinema

TUI sessions can be recorded to asciicast v3 format files (.cast) for later playback with asciinema or other compatible players.

To enable recording, pass a recording configuration when launching a session:

{
  "name": "tui_launch",
  "arguments": {
    "command": "bash",
    "recording": {
      "enabled": true,
      "outputPath": "/tmp/session.cast",
      "includeInput": false
    }
  }
}

Recordings can be played back using the asciinema CLI:

asciinema play /tmp/session.cast

Or uploaded to asciinema.org for web playback:

asciinema upload /tmp/session.cast

Recordings use the asciicast v3 format, which consists of:

• A JSON header line with version, terminal dimensions, timestamp, and command
• Event lines in the format [interval, "type", "data"]

Event types:

• o - Output data (terminal output)
• i - Input data (user input, if includeInput is enabled)
• r - Resize event (terminal dimension changes)
• x - Exit event (process termination with exit code)

cargo build --release

echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | ./target/release/mcp-tui-driver

# Launch a session
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"tui_launch","arguments":{"command":"htop"}}}

# Get screen text
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"tui_text","arguments":{"session_id":"<id>"}}}

# Press a key
{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"tui_press_key","arguments":{"session_id":"<id>","key":"q"}}}

# Close session
{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"tui_close","arguments":{"session_id":"<id>"}}}

Launch a new TUI application session.

{
  "name": "tui_launch",
  "arguments": {
    "command": "htop",
    "args": [],
    "cols": 80,
    "rows": 24
  }
}

Returns: {"session_id": "<uuid>"}

Close a TUI session and terminate the process.

{
  "name": "tui_close",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns: {"success": true}

List all active TUI sessions.

{
  "name": "tui_list_sessions",
  "arguments": {}
}

Returns: {"sessions": ["<id1>", "<id2>", ...]}

Get information about a TUI session.

{
  "name": "tui_get_session",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns:

{
  "session_id": "<uuid>",
  "command": "htop",
  "cols": 80,
  "rows": 24,
  "running": true
}

Resize the terminal window dimensions.

{
  "name": "tui_resize",
  "arguments": {
    "session_id": "<session_id>",
    "cols": 120,
    "rows": 40
  }
}

Returns: {"success": true}

Send a signal to the TUI process.

{
  "name": "tui_send_signal",
  "arguments": {
    "session_id": "<session_id>",
    "signal": "SIGINT"
  }
}

Supported signals: SIGINT, SIGTERM, SIGKILL, SIGHUP, SIGQUIT

Returns: {"success": true}

Get the current plain text content of the terminal.

{
  "name": "tui_text",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns: {"text": "terminal content here..."}

Get an accessibility-style snapshot with element references for clicking.

{
  "name": "tui_snapshot",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns:

{
  "yaml": "- row: 1\n  spans:\n    - ref: span-1\n      text: \"File\"\n      ...",
  "span_count": 42
}

The snapshot provides ref identifiers (like span-1) that can be used with tui_click, tui_double_click, and tui_right_click tools.

Take a PNG screenshot of the terminal.

{
  "name": "tui_screenshot",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns:

{
  "data": "<base64-encoded-png>",
  "format": "png",
  "width": 640,
  "height": 384
}

Press a single key.

{
  "name": "tui_press_key",
  "arguments": {
    "session_id": "<session_id>",
    "key": "Enter"
  }
}

Supported key formats:

• Special keys: Enter, Tab, Escape (or Esc), Backspace, Delete, Insert, Space
• Arrow keys: Up, Down, Left, Right (or ArrowUp, ArrowDown, etc.)
• Navigation: Home, End, PageUp (or PgUp), PageDown (or PgDown)
• Function keys: F1 through F12
• Ctrl combinations: Ctrl+c, Ctrl+z, etc.
• Alt combinations: Alt+x, Alt+f, etc.
• Single characters: a, A, 1, @, etc.

Returns: {"success": true}

Press multiple keys in sequence.

{
  "name": "tui_press_keys",
  "arguments": {
    "session_id": "<session_id>",
    "keys": ["Down", "Down", "Enter"]
  }
}

Returns: {"success": true}

Send raw text to the terminal (useful for typing strings).

{
  "name": "tui_send_text",
  "arguments": {
    "session_id": "<session_id>",
    "text": "Hello, World!"
  }
}

Returns: {"success": true}

Click on an element by reference ID from the snapshot.

{
  "name": "tui_click",
  "arguments": {
    "session_id": "<session_id>",
    "ref_id": "span-1"
  }
}

Returns: {"success": true}

Click at specific terminal coordinates.

{
  "name": "tui_click_at",
  "arguments": {
    "session_id": "<session_id>",
    "x": 10,
    "y": 5
  }
}

Returns: {"success": true}

Double-click on an element by reference ID.

{
  "name": "tui_double_click",
  "arguments": {
    "session_id": "<session_id>",
    "ref_id": "span-1"
  }
}

Returns: {"success": true}

Right-click on an element by reference ID.

{
  "name": "tui_right_click",
  "arguments": {
    "session_id": "<session_id>",
    "ref_id": "span-1"
  }
}

Returns: {"success": true}

Wait for specific text to appear on the screen.

{
  "name": "tui_wait_for_text",
  "arguments": {
    "session_id": "<session_id>",
    "text": "Ready",
    "timeout_ms": 5000
  }
}

Returns: {"found": true} or {"found": false} if timeout

Wait for the screen to stop changing (become idle).

{
  "name": "tui_wait_for_idle",
  "arguments": {
    "session_id": "<session_id>",
    "idle_ms": 100,
    "timeout_ms": 5000
  }
}

Returns: {"success": true}

Get TypeScript interface definitions for tui_run_code. Call this before using tui_run_code to understand the available API.

{
  "name": "tui_get_code_interface",
  "arguments": {}
}

Returns: TypeScript interface definitions as plain text, including the Tui interface with all available methods and the Console interface.

Execute JavaScript code with access to TUI automation functions. Call tui_get_code_interface first to get TypeScript definitions for the available API.

{
  "name": "tui_run_code",
  "arguments": {
    "session_id": "<session_id>",
    "code": "tui.sendText('hello'); tui.pressKey('Enter'); tui.text()"
  }
}

Available tui object methods:

Display:

• tui.text() - Returns the current screen text
• tui.snapshot() - Returns an accessibility snapshot as a JavaScript object
• tui.screenshot(filename?) - Takes a screenshot and saves to file, returns the file path

Input:

• tui.sendText(text) - Sends text to the terminal
• tui.pressKey(key) - Presses a key (e.g., "Enter", "Ctrl+c")
• tui.pressKeys(keys) - Presses multiple keys in sequence

Mouse:

• tui.click(ref) - Clicks on element by reference ID
• tui.clickAt(x, y) - Clicks at the specified coordinates
• tui.doubleClick(ref) - Double-clicks on element by reference ID
• tui.rightClick(ref) - Right-clicks on element by reference ID
• tui.hover(ref) - Hovers over element by reference ID
• tui.drag(startRef, endRef) - Drags from one element to another

Wait:

• tui.waitForText(text, timeoutMs?) - Waits for text to appear, returns boolean
• tui.waitForIdle(timeoutMs?, idleMs?) - Waits for screen to settle, returns boolean

Control:

• tui.resize(cols, rows) - Resizes the terminal
• tui.sendSignal(signal) - Sends a signal (SIGINT, SIGTERM, etc.)

Debug:

• tui.getScrollback() - Returns number of lines scrolled off screen
• tui.getInput(chars?) - Returns raw input buffer
• tui.getOutput(chars?) - Returns raw output buffer

Console output is captured and returned with results:

• console.log(...), console.info(...), console.warn(...), console.error(...), console.debug(...)

Returns:

{
  "result": "<last expression value>",
  "logs": [{"level": "log", "message": "..."}, ...]
}

Get raw input sent to the process (escape sequences included). Useful for debugging what was sent to the terminal.

{
  "name": "tui_get_input",
  "arguments": {
    "session_id": "<session_id>",
    "chars": 10000
  }
}

Returns:

{
  "length": 1234,
  "content": "<raw escape sequences>"
}

Get raw PTY output (escape sequences included). Useful for debugging terminal output.

{
  "name": "tui_get_output",
  "arguments": {
    "session_id": "<session_id>",
    "chars": 10000
  }
}

Returns:

{
  "length": 5678,
  "content": "<raw PTY output>"
}

Get the number of lines that have scrolled off the visible screen.

{
  "name": "tui_get_scrollback",
  "arguments": {
    "session_id": "<session_id>"
  }
}

Returns:

{
  "lines": 42
}

mcp-tui-driver/
  tui-driver/       # Core library for PTY management and terminal emulation
    src/
      driver.rs     # TuiDriver - main automation interface
      keys.rs       # Key parsing and ANSI escape sequences
      mouse.rs      # Mouse event handling
      snapshot.rs   # Accessibility snapshot generation and screenshot rendering
      error.rs      # Error types
      lib.rs        # Public API exports
  mcp-server/       # MCP server binary exposing tools via JSON-RPC over stdio
    src/
      main.rs       # CLI entrypoint and transport setup
      server.rs     # MCP protocol handling and tool implementations
      tools.rs      # Tool parameter and result types
      boa.rs        # JavaScript runtime integration (Boa engine)

• wezterm-term - Terminal emulation from WezTerm
• portable-pty - Cross-platform PTY (pseudo-terminal) management
• boa_engine - JavaScript execution for scripting
• rmcp - MCP protocol implementation
• image - Screenshot rendering

# Run tests
cargo test

# Check formatting
cargo fmt --check

# Run clippy
cargo clippy

# Build release binary
cargo build --release

• RUST_LOG - Set logging level (e.g., debug, info, mcp_tui_driver=debug)

Integration tests require a functional PTY environment:

cargo test --package tui-driver -- --test-threads=1

The server communicates over stdin/stdout using JSON-RPC 2.0 with the MCP protocol (version 2024-11-05).

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}

{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}

{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"<tool_name>","arguments":{...}}}

MIT