Skip to content

improvement(terminal): performance improvements#3796

Merged
TheodoreSpeaks merged 20 commits intostagingfrom
improvement/terminal-store-mem
Mar 27, 2026
Merged

improvement(terminal): performance improvements#3796
TheodoreSpeaks merged 20 commits intostagingfrom
improvement/terminal-store-mem

Conversation

@icecrasher321
Copy link
Collaborator

@icecrasher321 icecrasher321 commented Mar 26, 2026

Summary

  • Several performance improvements for terminal console logs rendering.
  • Fix copilot fetch chat call

Type of Change

  • Other: Performance

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link

vercel bot commented Mar 26, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Mar 27, 2026 2:21am

Request Review

@cursor
Copy link

cursor bot commented Mar 26, 2026

PR Summary

Medium Risk
Refactors terminal console storage/persistence and execution reconnection flow; regressions could impact log rendering, reconnect reliability, or client-side execution state across refreshes.

Overview
Improves terminal performance and durability of execution logs. Terminal log rendering is refactored to a virtualized list (react-window) backed by a new flattenVisibleExecutionRows model, and selection now tracks by entryId with targeted selectors (useWorkflowConsoleEntries, useConsoleEntry) to reduce re-renders.

Reworks terminal console storage and persistence. The console store moves from a single entries array to per-workflow workflowEntries with indexes for fast updates, adds normalization/truncation plus safe stringification for large/circular payloads, and replaces Zustand persist with an execution-aware IndexedDB persistence manager (consolePersistence) including an ExecutionPointer used to resume reconnects.

Makes SSE execution/reconnect streams resumable by event id. Server execution and reconnect APIs now attach eventId to buffered SSE events and extend reconnect polling to ~55 minutes; client stream processing surfaces onEventId and uses persisted pointers to reconnect from the last seen event.

Also: fetchChatHistory now falls back to the Copilot chat endpoint when the Mothership fetch fails, Copilot orchestrator can abort client-tool waits on disconnect, and workflow diffs treat parentId moves (and locked) as editable changes.

Written by Cursor Bugbot for commit 27a0e22. Configure here.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 26, 2026

Greptile Summary

This PR delivers a well-structured set of performance improvements to the terminal console log rendering system, replacing recursive full-tree rendering with a virtualized list, moving to per-workflow granular store subscriptions, and introducing an execution-aware persistence manager that avoids unnecessary IndexedDB serialization during hot execution paths.

Key changes:

  • Virtualized log list: ExecutionGroupRow recursive rendering is replaced by TerminalLogsPane backed by a virtual List component. A new flattenVisibleExecutionRows utility converts the hierarchical execution-group tree into a flat VisibleTerminalRow[] array that respects expandedNodes.
  • Granular store subscriptions: useWorkflowConsoleEntries (using useShallow) subscribes only to a specific workflow's entries. useConsoleEntry resolves a single entry by ID via the new entryLocationById O(1) index.
  • Execution-aware persistence (ConsolePersistenceManager): Persistence is now triggered only at execution lifecycle boundaries, eliminating Zustand persist middleware overhead on every store update.
  • renderChildren prop: Added to node row components so each virtual list row renders only itself; children are provided as separate flattened rows.
  • outputDataStringified moved into OutputPanel: Computation is deferred and skipped when in structured or code-display view.
  • Diff engine fix: parentId changes are now detected, flagging blocks that move into or out of a subflow/loop.

Confidence Score: 4/5

Safe to merge after verifying the fixed 32 px row height does not clip real-world entry content.

The architecture is sound — virtualization, granular subscriptions, and lifecycle-aware persistence are all well-reasoned improvements backed by tests. The one concrete concern (fixed 32 px row height for variable-height content) could cause visual clipping in production but is not a data-loss or correctness bug. The documentation mismatch in the safety timer comment is trivial. No logic regressions, no security issues.

apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx — verify LOG_ROW_HEIGHT_PX = 32 is sufficient for all rendered entry types.

Important Files Changed

Filename Overview
apps/sim/stores/terminal/console/storage.ts Replaces Zustand persist middleware with a hand-rolled ConsolePersistenceManager that writes to IndexedDB only at execution lifecycle boundaries. Minor doc inconsistency: comment says "Every 30s" but timer fires every 5 s.
apps/sim/stores/terminal/console/store.ts Refactors store from flat entries array to per-workflow map with O(1) index lookups. Adds useWorkflowConsoleEntries (useShallow) and useConsoleEntry for granular subscriptions.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx Replaces recursive DOM rendering with a react-window virtualized list via TerminalLogsPane + flattenVisibleExecutionRows. Fixed 32px row height may clip variable-height entries.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts Adds flattenVisibleExecutionRows to convert hierarchical execution groups into a flat VisibleTerminalRow array respecting expandedNodes. Adds LOG_ROW_HEIGHT_PX = 32 constant.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/components/output-panel/output-panel.tsx Moves outputDataStringified computation into the component with extra conditions to skip work when in structured/code view. Changes filteredEntries prop to a simpler hasEntries boolean.
apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts Wraps rawSetIsExecuting to call consolePersistence.executionStarted/Ended automatically; fixes stale entries check to use getWorkflowEntries.
apps/sim/lib/workflows/diff/diff-engine.ts Adds parentId to block change-detection and field-diff logic. Well-covered by new test.
apps/sim/stores/terminal/console/utils.ts New file with safeConsoleStringify, normalization helpers, and trimWorkflowConsoleEntries extracted for independent testing.
apps/sim/stores/terminal/console/types.ts Adds entryIdsByBlockExecution and entryLocationById to ConsoleStore for O(1) lookups. addConsole return type updated to ConsoleEntry

Comments Outside Diff (1)

  1. apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx, line 416-427 (link)

    Fixed row height clips variable-height entries

    Every row — separators, workflow nodes, subflow nodes, and plain block entries — is rendered at exactly TERMINAL_CONFIG.LOG_ROW_HEIGHT_PX (32 px). WorkflowNodeRow, SubflowNodeRow, and IterationNodeRow all render block-name text, status icons, duration badges, and sometimes warning/error indicators whose combined height can easily exceed 32 px. Content beyond that threshold will be silently clipped by the virtual list container.

    If all entry rows are guaranteed to fit within 32 px, this is fine as-is, but please verify against the tallest real-world row. If heights vary you'll need either:

    • A VariableSizeList with per-row height estimation, or
    • A slightly larger constant (e.g. 40–44 px) that comfortably fits the tallest possible row.

Reviews (2): Last reviewed commit: "fix diff comment" | Re-trigger Greptile

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

@greptile

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

bugbot run

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@TheodoreSpeaks TheodoreSpeaks merged commit ed56451 into staging Mar 27, 2026
12 checks passed
@TheodoreSpeaks TheodoreSpeaks deleted the improvement/terminal-store-mem branch March 27, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants