feature(context): extend plugin system to support custom context management by jalehman · Pull Request #22201 · openclaw/openclaw

jalehman · 2026-02-20T21:44:15Z

Discussion thread

Summary

Problem: OpenClaw's context management (compaction, assembly, etc.) is hardcoded in core, making it impossible for plugins to provide alternative context strategies. This hasn't been a problem for very long, because other context management strategies haven't existed.
Why it matters: The Lossless Context Management paper from Ehrlich and Blackman demonstrated a promising alternative direction for context management. I have implemented that approach for OpenClaw (with a number of improvements), have been running it for the last week, and to say it works well would be an understatement. That work is in the lossless-claw repository.
What changed: Added a ContextEngine plugin interface with full lifecycle hooks, a slot-based registry, legacy pass-through wrapper, and wired it into the agent run loop, /compact command, and subagent registry. Added scoped subagent methods and gateway request scope for plugin runtimes, as an important aspect of LCM is the ability to spawn subagents for deep context traversal.
What did NOT change: Zero changes to existing compaction logic. The LegacyContextEngine wraps existing behavior identically. No new dependencies. No changes to message format, API contracts, or user-facing config (beyond an optional contextEngine slot).

Change Type

Feature
Refactor

Scope

Gateway / orchestration
Skills / tool execution (plugin SDK)
Memory / storage
API / contracts

Linked Issue/PR

Makes sense in the context of the lossless-claw repository, which is the first alternative context management engine.

User-visible / Behavior Changes

New optional contextEngine config slot in agent plugin configuration. If unset, LegacyContextEngine is used automatically — zero behavior change for existing users.
Plugins can now register context engines via api.registerContextEngine(id, factory).

Security Impact (required)

New permissions/capabilities? No — context engines have the same access scope as existing compaction code
Secrets/tokens handling changed? No
New/changed network calls? No
Command/tool execution surface changed? No — plugin SDK gains registerContextEngine but this is an extension point, not a new attack surface
Data access scope changed? No — context engines access the same message data that compaction already accesses
Scoped subagent interface limits plugin runtime to run/waitForRun/deleteSession/getSession (no full gateway access)

Repro + Verification

Environment

OS: macOS 14 (arm64)
Runtime: Node 22.17.0
Model/provider: N/A

Steps

git checkout feature/context-engine && pnpm install && pnpm build
Run without any context engine plugin configured → should behave identically to main
Install lossless-claw plugin: pnpm openclaw plugins install @martian-engineering/lossless-claw
Configure contextEngine: "lossless-claw" in plugins.slots
Send first message -> lossless-claw will begin incremental compaction of past messages to build the summary DAG

Expected

Without plugin: identical behavior to main branch
With plugin: context engine lifecycle hooks called at each phase

Actual

Both scenarios verified across multiple openclaw installations by different users.

Evidence

Attach at least one:

334-line test suite (context-engine.test.ts) covering interface contract, registry resolution, legacy wrapper, and lifecycle wiring
server-plugins.test.ts extended with 159 lines for plugin registration and scoped runtime
gateway-request-scope.test.ts for scoped subagent method isolation
Full test suite: 774/776 passing (2 pre-existing failures on upstream main, verified independently)

Human Verification (required)

What you personally verified (not just CI), and how:

Verified: Running OpenClaw gateway with no context engine plugin — identical behavior to main
Verified: Running with lossless-claw plugin — full lifecycle (bootstrap, assemble, afterTurn, ingest, compact) working end-to-end
Verified: Subagent lifecycle notifications fire correctly on delete/complete/release/sweep
Verified: /compact routes through context engine when present
Edge cases: Legacy fallback when plugin fails to load, slot resolution with no registered engines, multiple engines registered (first match wins)
Not verified: Windows/Linux (macOS only)

Compatibility / Migration

Backward compatible? Yes
Config/env changes? Yes — optional contextEngine slot added to plugin config schema
Migration needed? No

Failure Recovery (if this breaks)

How to disable/revert this change quickly: delete contextEngine: "lossless-claw" entry from config
Files/config to restore: delete contextEngine: "lossless-claw" entry from config

Risks and Mitigations

Risk: Context engine plugin crashes during assemble() → LLM gets no/wrong context
- Mitigation: Legacy fallback wraps existing proven code path; plugin errors are caught and logged. If a plugin fails to bootstrap, gateway falls back to legacy.
- Mitigation: All messages are still written to JSONL session file even with new context engines like lossless-claw. No data is ever lost.
  • Risk: ownsCompaction flag disables Pi compaction but plugin doesn't actually compact → unbounded context growth
  • Mitigation: Overflow guard in attempt.ts still triggers contextEngine.compact() on token budget exceeded, regardless of ownsCompaction flag. The flag only suppresses the automatic Pi compaction, not the overflow safety net.

Use of AI

AI was used to write this code
It's highly tested, both via automated testing and actual usage

Greptile Summary

Introduced a ContextEngine plugin interface to enable alternative context management strategies beyond OpenClaw's hardcoded compaction logic. The implementation wraps existing behavior in a LegacyContextEngine for backward compatibility, adds full lifecycle hooks (bootstrap, ingest, assemble, compact, afterTurn, prepareSubagentSpawn, onSubagentEnded), and wires it into the agent run loop, /compact command, and subagent registry.

Added ContextEngine interface with AssembleResult, CompactResult, IngestResult, BootstrapResult types
Created LegacyContextEngine wrapper that delegates to existing compactEmbeddedPiSessionDirect
Added slot-based registry with resolveContextEngine() supporting config-driven selection
Wired contextEngine.bootstrap() into run attempt initialization when session file exists
Wired contextEngine.assemble() into context assembly pipeline after sanitize/validate/limit
Wired contextEngine.afterTurn() or fallback ingest()/ingestBatch() after turn completion
Routed overflow compaction through contextEngine.compact() with legacy params bridge
Added ownsCompaction flag with applyPiAutoCompactionGuard() to disable Pi's internal compaction
Exposed api.registerContextEngine() in plugin SDK
Added scoped subagent runtime (run, waitForRun, getSessionMessages, deleteSession) using AsyncLocalStorage for gateway request scope
Added sessions.get gateway method to retrieve session messages
Wired subagent lifecycle notifications (deleted, completed, swept, released) to contextEngine.onSubagentEnded()
Added 334-line test suite covering interface contract, registry, legacy wrapper, and lifecycle wiring

Confidence Score: 4/5

This PR is safe to merge with low risk - the implementation is well-tested, properly scoped, and maintains full backward compatibility through the legacy wrapper.
Score reflects thorough design (clean interface, proper lifecycle hooks, comprehensive error handling), extensive test coverage (334-line context engine test suite + 159 lines plugin tests), zero breaking changes (legacy wrapper preserves existing behavior), and careful integration (bootstrap only on existing sessions, graceful fallbacks on errors). The scoped subagent runtime using AsyncLocalStorage is an appropriate pattern, and the ownsCompaction flag provides safe opt-out of Pi's auto-compaction. Minor deduction for potential edge cases around context engine errors during critical paths (bootstrap/assemble) being logged but not blocking execution.
No files require special attention - the implementation is clean and well-tested throughout

_{Last reviewed commit: 5c491ac}

rabsef-bicrym · 2026-02-20T21:47:24Z

Let 10,000 Context Management Systems Bloom.

rodrigouroz · 2026-02-27T20:16:55Z

Hey @jalehman I'm happy to see that this seems to be moving forward.
I wonder if you think this would clash with my PR: #25554 (and there are follow up PRs linked to that one)

…gement (openclaw#22201) * feat(context-engine): add ContextEngine interface and registry Introduce the pluggable ContextEngine abstraction that allows external plugins to register custom context management strategies. - ContextEngine interface with lifecycle methods: bootstrap, ingest, ingestBatch, afterTurn, assemble, compact, prepareSubagentSpawn, onSubagentEnded, dispose - Module-level singleton registry with registerContextEngine() and resolveContextEngine() (config-driven slot selection) - LegacyContextEngine: pass-through implementation wrapping existing compaction behavior for 100% backward compatibility - ensureContextEnginesInitialized() guard for safe one-time registration - 19 tests covering contract, registry, resolution, and legacy parity * feat(plugins): add context-engine slot and registerContextEngine API Wire the ContextEngine abstraction into the plugin system so external plugins can register context engines via the standard plugin API. - Add 'context-engine' to PluginKind union type - Add 'contextEngine' slot to PluginSlotsConfig (default: 'legacy') - Wire registerContextEngine() through OpenClawPluginApi - Export ContextEngine types from plugin-sdk for external consumers - Restore proper slot-based resolution in registry * feat(context-engine): wire ContextEngine into agent run lifecycle Integrate the ContextEngine abstraction into the core agent run path: - Resolve context engine once per run (reused across retries) - Bootstrap: hydrate canonical store from session file on first run - Assemble: route context assembly through pluggable engine - Auto-compaction guard: disable built-in auto-compaction when the engine declares ownsCompaction (prevents double-compaction) - AfterTurn: post-turn lifecycle hook for ingest + background compaction decisions - Overflow compaction: route through contextEngine.compact() - Dispose: clean up engine resources in finally block - Notify context engine on subagent lifecycle events Legacy engine: all lifecycle methods are pass-through/no-op, preserving 100% backward compatibility for users without a context engine plugin. * feat(plugins): add scoped subagent methods and gateway request scope Expose runtime.subagent.{run, waitForRun, getSession, deleteSession} so external plugins can spawn sub-agent sessions without raw gateway dispatch access. Uses AsyncLocalStorage request-scope bridge to dispatch internally via handleGatewayRequest with a synthetic operator client. Methods are only available during gateway request handling. - Symbol.for-backed global singleton for cross-module-reload safety - Fallback gateway context for non-WS dispatch paths (Telegram/WhatsApp) - Set gateway request scope for all handlers, not just plugin handlers - 3 staleness tests for fallback context hardening * feat(context-engine): route /compact and sessions.get through context engine Wire the /compact command and sessions.get handler through the pluggable ContextEngine interface. - Thread tokenBudget and force parameters to context engine compact - Route /compact through contextEngine.compact() when registered - Wire sessions.get as runtime alias for plugin subagent dispatch - Add .pebbles/ to .gitignore * style: format with oxfmt 0.33.0 Fix duplicate import (ControlUiRootState in server.impl.ts) and import ordering across all changed files. * fix: update extension test mocks for context-engine types Add missing subagent property to bluebubbles PluginRuntime mock. Add missing registerContextEngine to lobster OpenClawPluginApi mock. * fix(subagents): keep deferred delete cleanup retryable * style: format run attempt for CI * fix(rebase): remove duplicate embedded-run imports * test: add missing gateway context mock export * fix: pass resolved auth profile into afterTurn compaction Ensure the embedded runner forwards resolved auth profile context into legacy context-engine compaction params on the normal afterTurn path, matching overflow compaction behavior. This allows downstream LCM summarization to use the intended provider auth/profile consistently. Also fix strict TS typing in external-link token dedupe and align an attempt unit test reasoningLevel value with the current ReasoningLevel enum. Regeneration-Prompt: | We were debugging context-engine compaction where downstream summary calls were missing the right auth/profile context in normal afterTurn flow, while overflow compaction already propagated it. Preserve current behavior and keep changes additive: thread the resolved authProfileId through run -> attempt -> legacy compaction param builder without broad refactors. Add tests that prove the auth profile is included in afterTurn legacy params and that overflow compaction still passes it through run attempts. Keep existing APIs stable, and only adjust small type issues needed for strict compilation. * fix: remove duplicate imports from rebase * feat: add context-engine system prompt additions * fix(rebase): dedupe attempt import declarations * test: fix fetch mock typing in ollama autodiscovery * fix(test): add registerContextEngine to diffs extension mock APIs * test(windows): use path.delimiter in ios-team-id fixture PATH * test(cron): add model formatting and precedence edge case tests Covers: - Provider/model string splitting (whitespace, nested paths, empty segments) - Provider normalization (casing, aliases like bedrock→amazon-bedrock) - Anthropic model alias normalization (opus-4.5→claude-opus-4-5) - Precedence: job payload > session override > config default - Sequential runs with different providers (CI flake regression pattern) - forceNew session preserving stored model overrides - Whitespace/empty model string edge cases - Config model as string vs object format * test(cron): fix model formatting test config types * test(phone-control): add registerContextEngine to mock API * fix: re-export ChannelKind from config-reload-plan * fix: add subagent mock to plugin-runtime-mock test util * docs: add changelog fragment for context engine PR openclaw#22201

* main: Mattermost: harden interaction callback binding (openclaw#38057) WhatsApp: honor outbound mediaMaxMb (openclaw#38097) openai-image-gen: validate --background and --style options (openclaw#36762) Docs: align BlueBubbles media cap wording Telegram/Discord: honor outbound mediaMaxMb uploads (openclaw#38065) CI: run changed-scope on main pushes Skills/nano-banana-pro: clarify MEDIA token comment (openclaw#38063) nano-banana-pro: respect explicit --resolution when editing images (openclaw#36880) CI: drop unused install-smoke bootstrap fix(nano-banana-pro): remove space after MEDIA: token in generate_image.py (openclaw#18706) docs: context engine docs(config): list the context engine plugin slot docs(plugins): add context-engine manifest kind example docs(plugins): document context engine slots and registration docs(protocol): document slash-delimited schema lookup plugin ids docs(tools): document slash-delimited config schema lookup paths fix(session): tighten direct-session webchat routing matching (openclaw#37867) feature(context): extend plugin system to support custom context management (openclaw#22201) Gateway: allow slash-delimited schema lookup paths

…gement (openclaw#22201) * feat(context-engine): add ContextEngine interface and registry Introduce the pluggable ContextEngine abstraction that allows external plugins to register custom context management strategies. - ContextEngine interface with lifecycle methods: bootstrap, ingest, ingestBatch, afterTurn, assemble, compact, prepareSubagentSpawn, onSubagentEnded, dispose - Module-level singleton registry with registerContextEngine() and resolveContextEngine() (config-driven slot selection) - LegacyContextEngine: pass-through implementation wrapping existing compaction behavior for 100% backward compatibility - ensureContextEnginesInitialized() guard for safe one-time registration - 19 tests covering contract, registry, resolution, and legacy parity * feat(plugins): add context-engine slot and registerContextEngine API Wire the ContextEngine abstraction into the plugin system so external plugins can register context engines via the standard plugin API. - Add 'context-engine' to PluginKind union type - Add 'contextEngine' slot to PluginSlotsConfig (default: 'legacy') - Wire registerContextEngine() through OpenClawPluginApi - Export ContextEngine types from plugin-sdk for external consumers - Restore proper slot-based resolution in registry * feat(context-engine): wire ContextEngine into agent run lifecycle Integrate the ContextEngine abstraction into the core agent run path: - Resolve context engine once per run (reused across retries) - Bootstrap: hydrate canonical store from session file on first run - Assemble: route context assembly through pluggable engine - Auto-compaction guard: disable built-in auto-compaction when the engine declares ownsCompaction (prevents double-compaction) - AfterTurn: post-turn lifecycle hook for ingest + background compaction decisions - Overflow compaction: route through contextEngine.compact() - Dispose: clean up engine resources in finally block - Notify context engine on subagent lifecycle events Legacy engine: all lifecycle methods are pass-through/no-op, preserving 100% backward compatibility for users without a context engine plugin. * feat(plugins): add scoped subagent methods and gateway request scope Expose runtime.subagent.{run, waitForRun, getSession, deleteSession} so external plugins can spawn sub-agent sessions without raw gateway dispatch access. Uses AsyncLocalStorage request-scope bridge to dispatch internally via handleGatewayRequest with a synthetic operator client. Methods are only available during gateway request handling. - Symbol.for-backed global singleton for cross-module-reload safety - Fallback gateway context for non-WS dispatch paths (Telegram/WhatsApp) - Set gateway request scope for all handlers, not just plugin handlers - 3 staleness tests for fallback context hardening * feat(context-engine): route /compact and sessions.get through context engine Wire the /compact command and sessions.get handler through the pluggable ContextEngine interface. - Thread tokenBudget and force parameters to context engine compact - Route /compact through contextEngine.compact() when registered - Wire sessions.get as runtime alias for plugin subagent dispatch - Add .pebbles/ to .gitignore * style: format with oxfmt 0.33.0 Fix duplicate import (ControlUiRootState in server.impl.ts) and import ordering across all changed files. * fix: update extension test mocks for context-engine types Add missing subagent property to bluebubbles PluginRuntime mock. Add missing registerContextEngine to lobster OpenClawPluginApi mock. * fix(subagents): keep deferred delete cleanup retryable * style: format run attempt for CI * fix(rebase): remove duplicate embedded-run imports * test: add missing gateway context mock export * fix: pass resolved auth profile into afterTurn compaction Ensure the embedded runner forwards resolved auth profile context into legacy context-engine compaction params on the normal afterTurn path, matching overflow compaction behavior. This allows downstream LCM summarization to use the intended provider auth/profile consistently. Also fix strict TS typing in external-link token dedupe and align an attempt unit test reasoningLevel value with the current ReasoningLevel enum. Regeneration-Prompt: | We were debugging context-engine compaction where downstream summary calls were missing the right auth/profile context in normal afterTurn flow, while overflow compaction already propagated it. Preserve current behavior and keep changes additive: thread the resolved authProfileId through run -> attempt -> legacy compaction param builder without broad refactors. Add tests that prove the auth profile is included in afterTurn legacy params and that overflow compaction still passes it through run attempts. Keep existing APIs stable, and only adjust small type issues needed for strict compilation. * fix: remove duplicate imports from rebase * feat: add context-engine system prompt additions * fix(rebase): dedupe attempt import declarations * test: fix fetch mock typing in ollama autodiscovery * fix(test): add registerContextEngine to diffs extension mock APIs * test(windows): use path.delimiter in ios-team-id fixture PATH * test(cron): add model formatting and precedence edge case tests Covers: - Provider/model string splitting (whitespace, nested paths, empty segments) - Provider normalization (casing, aliases like bedrock→amazon-bedrock) - Anthropic model alias normalization (opus-4.5→claude-opus-4-5) - Precedence: job payload > session override > config default - Sequential runs with different providers (CI flake regression pattern) - forceNew session preserving stored model overrides - Whitespace/empty model string edge cases - Config model as string vs object format * test(cron): fix model formatting test config types * test(phone-control): add registerContextEngine to mock API * fix: re-export ChannelKind from config-reload-plan * fix: add subagent mock to plugin-runtime-mock test util * docs: add changelog fragment for context engine PR openclaw#22201

openclaw-barnacle bot added gateway Gateway runtime agents Agent runtime and tooling size: XL labels Feb 20, 2026

jalehman changed the title ~~Feature/context engine~~ [feature] extend plugin system to support custom context management Feb 20, 2026

jalehman force-pushed the feature/context-engine branch 2 times, most recently from c790069 to bb961f5 Compare February 20, 2026 22:00

openclaw-barnacle bot added channel: bluebubbles Channel integration: bluebubbles extensions: lobster Extension: lobster labels Feb 20, 2026

jalehman changed the title ~~[feature] extend plugin system to support custom context management~~ feature(context): extend plugin system to support custom context management Feb 20, 2026

jalehman force-pushed the feature/context-engine branch 3 times, most recently from b16b28d to 02f7ca4 Compare February 23, 2026 15:23

jalehman mentioned this pull request Feb 25, 2026

Feature Request: Context Provider Plugin Slot (plugins.slots.context) #12660

Open

jalehman force-pushed the feature/context-engine branch 8 times, most recently from f7023b7 to 389cb64 Compare February 27, 2026 17:23

jalehman mentioned this pull request Feb 27, 2026

Compaction/Safeguard: preserve recent turns verbatim #25554

Merged

18 tasks

jalehman force-pushed the feature/context-engine branch from 67ddcf7 to 51c10e4 Compare February 28, 2026 22:27

openclaw-barnacle bot added the maintainer Maintainer-authored PR label Feb 28, 2026

jalehman added extensions: NEW Placeholder for new extension labels and removed channel: bluebubbles Channel integration: bluebubbles extensions: lobster Extension: lobster extensions: NEW Placeholder for new extension labels labels Feb 28, 2026

jalehman added 4 commits March 5, 2026 19:30

test(cron): fix model formatting test config types

1c67b10

test(phone-control): add registerContextEngine to mock API

ec637ab

fix: re-export ChannelKind from config-reload-plan

2ab3cee

fix: add subagent mock to plugin-runtime-mock test util

de60de6

jalehman force-pushed the feature/context-engine branch from fef0a56 to 6061882 Compare March 6, 2026 04:14

jalehman added a commit to jalehman/clawdbot that referenced this pull request Mar 6, 2026

docs: add changelog fragment for context engine PR openclaw#22201

6061882

jalehman added a commit to jalehman/clawdbot that referenced this pull request Mar 6, 2026

docs: add changelog fragment for context engine PR openclaw#22201

002fffd

jalehman force-pushed the feature/context-engine branch from 6061882 to 002fffd Compare March 6, 2026 13:22

docs: add changelog fragment for context engine PR openclaw#22201

ab538c2

jalehman force-pushed the feature/context-engine branch from 002fffd to ab538c2 Compare March 6, 2026 13:28

Merge branch 'main' into feature/context-engine

ce41563

jalehman merged commit fee91fe into openclaw:main Mar 6, 2026
9 checks passed

github-actions bot mentioned this pull request Mar 6, 2026

📡 Upstream Digest — 2026-03-06 14:31 UTC curtismercier/openclaw-mods#192

Open

alexyyyander mentioned this pull request Mar 7, 2026

fix/gateway token mismatch 38617 #38676

Closed

jbmx77 mentioned this pull request Mar 8, 2026

[Bug]: Cannot find module diagnostic-session-state-CDEk5vOn.js after updating to 2026.3.7 — all tools fail #39586

Open

sungjaeshim mentioned this pull request Mar 8, 2026

Feature: System-context prepend as config switch #39616

Open

RobertoGongora mentioned this pull request Mar 8, 2026

feat: implement ContextEngine interface for native Graphiti-managed memory RobertoGongora/openclaw-graphiti-plugin#11

Closed

12 tasks

bm-clawd mentioned this pull request Mar 8, 2026

Adopt OpenClaw ContextEngine plugin interface for deeper memory integration basicmachines-co/openclaw-basic-memory#34

Closed

davidrudduck mentioned this pull request Mar 9, 2026

feat: add enabled flag for compaction config davidrudduck/openclaw#7

Closed

Windelly mentioned this pull request Mar 9, 2026

feat: add before_context_send plugin hook #14544

Open

davidrudduck mentioned this pull request Mar 9, 2026

fix(context-engine): guard compact() throw + fire hooks for ownsCompaction engines #41361

Merged

4 tasks

slantview mentioned this pull request Mar 10, 2026

feat(gateway): add sessions.get RPC method (openclaw v2026.3.7) a3tai/openclaw-go#10

Merged

danhdoan mentioned this pull request Mar 12, 2026

feat(context-engine): add ByteRover context engine plugin #43920

Open

4 tasks

jalehman mentioned this pull request Mar 14, 2026

docs: add context engine documentation #46409

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feature(context): extend plugin system to support custom context management#22201

feature(context): extend plugin system to support custom context management#22201
jalehman merged 25 commits intoopenclaw:mainfrom
jalehman:feature/context-engine

jalehman commented Feb 20, 2026 •

edited

Loading

Uh oh!

rabsef-bicrym commented Feb 20, 2026

Uh oh!

rodrigouroz commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jalehman commented Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Change Type

Scope

Linked Issue/PR

User-visible / Behavior Changes

Security Impact (required)

Repro + Verification

Environment

Steps

Expected

Actual

Evidence

Human Verification (required)

Compatibility / Migration

Failure Recovery (if this breaks)

Risks and Mitigations

Use of AI

Greptile Summary

Confidence Score: 4/5

Uh oh!

rabsef-bicrym commented Feb 20, 2026

Uh oh!

rodrigouroz commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jalehman commented Feb 20, 2026 •

edited

Loading