Skip to content

feat(spike): agentic block poc#341

Merged
dinohamzic merged 17 commits intomainfrom
agent-poc-2
Mar 19, 2026
Merged

feat(spike): agentic block poc#341
dinohamzic merged 17 commits intomainfrom
agent-poc-2

Conversation

@dinohamzic
Copy link
Contributor

@dinohamzic dinohamzic commented Mar 12, 2026

Summary by CodeRabbit

  • New Features
    • Agent Block support: run AI-driven notebook actions, standalone prompts via --prompt, and streamed agent reasoning/events with tool outputs.
  • Documentation
    • New Agent Block and CLI docs covering usage, env vars, model selection, integrations, and examples.
  • Examples
    • Added an example agent-block notebook demonstrating end-to-end agent usage.
  • Tests
    • New and expanded tests for agent execution, prompt flows, and agent event handling.
  • Chores
    • Updated ignore rules and spelling dictionary entries.

@dinohamzic dinohamzic self-assigned this Mar 12, 2026
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 12, 2026

📝 Walkthrough

Walkthrough

Adds first-class agent-block support: schema extensions for MCP server configs and agent metadata; new output-text utilities; a runtime agent-handler that runs streaming OpenAI-backed agents with MCP tools; execution-engine changes to detect and run agent blocks and surface agent events; CLI --prompt support (including prompt-only in-memory files and streaming markdown rendering); tests, fixtures, docs, and public exports to wire these features together.

Sequence Diagram

sequenceDiagram
    actor User
    participant CLI as CLI
    participant Engine as ExecutionEngine
    participant Handler as AgentHandler
    participant OpenAI as OpenAI
    participant MCP as MCP
    participant Kernel as Kernel
    participant Notebook as Notebook

    User->>CLI: deepnote run --prompt "Analyze data"
    CLI->>Engine: runDeepnoteProject(path?, options)
    Engine->>Handler: executeAgentBlock(context)
    Handler->>Handler: serializeNotebookContext / buildSystemPrompt
    Handler->>OpenAI: init client & stream prompts
    Handler->>MCP: init MCP servers (merge configs)
    loop Agent tool loop
        OpenAI-->>Handler: text_delta / reasoning_delta / tool_called
        Handler->>CLI: onAgentEvent(streamed deltas)
        alt tool_called -> create block
            Handler->>Notebook: create code/markdown block
            Notebook->>Kernel: execute code
            Kernel-->>Notebook: outputs
            Notebook-->>Handler: outputs collected
        else tool_called -> MCP tool
            Handler->>MCP: call tool
            MCP-->>Handler: tool_output
        end
        Handler->>OpenAI: feed tool_output back
    end
    OpenAI-->>Handler: final output
    Handler-->>Engine: AgentBlockResult(finalOutput, addedBlockIds, blockOutputs)
    Engine-->>CLI: execution complete
    CLI-->>User: display results
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.86% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'agentic block poc' directly describes the primary changeset: introducing agent block proof-of-concept with agentic capabilities across schema, CLI, runtime, and testing.
Updates Docs ✅ Passed Pull request includes comprehensive documentation updates for agent block feature across repository with new reference guides, CLI documentation, and examples.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Mar 12, 2026

Codecov Report

❌ Patch coverage is 65.46275% with 153 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.70%. Comparing base (219b013) to head (764bd38).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
packages/runtime-core/src/agent-handler.ts 32.72% 74 Missing ⚠️
packages/cli/src/commands/run.ts 74.13% 66 Missing and 1 partial ⚠️
packages/runtime-core/src/execution-engine.ts 80.55% 7 Missing ⚠️
packages/blocks/src/output-text.ts 90.32% 3 Missing ⚠️
packages/mcp/src/tools/execution.ts 0.00% 2 Missing ⚠️

❌ Your patch status has failed because the patch coverage (65.46%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #341      +/-   ##
==========================================
- Coverage   83.49%   82.70%   -0.80%     
==========================================
  Files         124      126       +2     
  Lines        7375     7614     +239     
  Branches     2046     2117      +71     
==========================================
+ Hits         6158     6297     +139     
- Misses       1217     1316      +99     
- Partials        0        1       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 14

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/runtime-core/src/execution-engine.ts (1)

17-37: ⚠️ Potential issue | 🟠 Major

Re-export the canonical executable-block list.

This copy has already drifted: Line 26 duplicates Line 19. executableBlockTypes is exported, so consumers iterating the array will now see agent twice.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/execution-engine.ts` around lines 17 - 37, The
exported array executableBlockTypes contains a duplicate 'agent' entry causing
consumers to see it twice; update executableBlockTypes to remove the duplicate
(ensure each block type appears only once) and keep executableBlockTypeSet in
sync by rebuilding it from the corrected array; reference the
executableBlockTypes constant (and executableBlockTypeSet) when making the
change so iterators and the exported ReadonlySet no longer include duplicate
'agent' entries.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.gitignore:
- Around line 1-72: The .gitignore contains many duplicated entries (e.g.,
repeated patterns like .DS_Store, .env, .npmrc, .pnpm-store, .qlty/* entries,
*.png, *.tsbuildinfo, coverage, dist, examples/report.md,
examples/snapshots/*_T*.snapshot.deepnote, node_modules); remove duplicate lines
so each ignore pattern appears exactly once, preserving a sensible
order/grouping (project-level ignores, tool/cache dirs, build artifacts,
examples) and ensure no accidental removals of unique patterns—update the
.gitignore by deduplicating all repeated entries and leaving a single instance
of each pattern.

In `@examples/README.md`:
- Around line 93-95: Update the heading and inline label to use title-case
"Agent blocks" for consistency: change the "### Agent Block" heading if needed
and replace the bold phrase "**agent blocks**" with "**Agent blocks**
(capitalized)". Ensure the phrase inside the sentence ("let an AI agent...")
remains the same but uses the capitalized bold label so it matches other
sections like "**Markdown blocks**" and "**Input blocks**".

In `@packages/blocks/src/output-text.ts`:
- Around line 10-21: The extractOutputText function currently treats stream.text
and data['text/plain'] incorrectly when they are string[] (nbformat allows both
string and string[]) — instead of relying on String(...) which comma-joins
arrays, detect Array.isArray for out.text and for data?.['text/plain'] and
concatenate with '' (or join('')/join('') to preserve original newlines) to
produce the full text; update the branches that check out.output_type ===
'stream' and the execute_result/display_data handling where data?.['text/plain']
is returned, and add regression tests for both string and string[] forms to
cover stream.text and data['text/plain'].

In `@packages/cli/README.md`:
- Around line 197-199: The README currently states OPENAI_MODEL defaults to
"gpt-5" but other docs (skills/deepnote/references/blocks-agent.md) show the
agent-block default as "auto"; update the README entry for OPENAI_MODEL to
clearly state the actual runtime precedence and final default: explain that
OPENAI_MODEL (and OPENAI_BASE_URL if relevant) override the agent/block-level
model, that agent-blocks may specify "auto" to select a provider-specific
default, and specify what model is used when neither OPENAI_MODEL nor any
block-specific model is set (e.g., the real system default such as "gpt-5" or
provider default). Reference the OPENAI_MODEL and the agent-block "auto"
behavior so readers understand which wins and what the ultimate fallback is.

In `@packages/cli/src/commands/run.ts`:
- Around line 471-473: The debug output currently redacts only options.token but
still prints the full user prompt; update the creation of safeOptions (used in
the debug call with debug(`Options: ${JSON.stringify(safeOptions)}`)) to also
redact any prompt-related fields by replacing options.prompt (and any
promptText/promptFile keys if present in this options object) with a placeholder
like '[redacted]' before stringifying; keep the rest of the options intact and
continue to redact token as before so the debug log cannot leak user prompt
contents.
- Around line 272-279: The code always appends the new agent block to the last
notebook; change it to insert into the selected execution scope: locate the
target notebook based on the provided path/--notebook selection (instead of
using file.project.notebooks[file.project.notebooks.length - 1]) and push the
createAgentBlock(options.prompt, ...) into that notebook; if a --block selection
is present, find that block within the target notebook (by id/index), and insert
the new agent block immediately after that block’s index (or at the start if not
found). Update the logic around createAgentBlock and lastNotebook to use the
resolved targetNotebook and ensure you throw the existing error if no matching
notebook is found; add tests covering prompt+notebook and prompt+block
combinations.

In `@packages/convert/src/roundtrip-all-formats.test.ts`:
- Around line 669-671: The test in roundtrip-all-formats.test.ts only asserts
the fixture for "agent-1" and misses verifying that "agent-2" preserved its
metadata; add an assertion that the round-tripped agent-2 object contains
metadata.deepnote_agent_model === 'auto' (use the existing expectation structure
/ Vitest expect calls used for agent-1), so the test verifies both agent-1 and
agent-2 metadata are preserved after the roundtrip.

In `@packages/runtime-core/src/agent-handler.test.ts`:
- Around line 6-14: The tests currently set process.env.TEST_HOST and
process.env.TEST_PORT in beforeEach and unconditionally delete them in
afterEach; instead capture and store the prior values (e.g., prevTestHost,
prevTestPort) in beforeEach before overriding, and in afterEach restore them
(set back to the saved value if defined, otherwise delete the key); update the
beforeEach/afterEach blocks in agent-handler.test.ts to use these temporary
variables so existing environment values are preserved across tests.

In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 146-160: Merged MCP configs (mergedMcpConfig) are currently
auto-started by creating MCP clients via createMCPClient with
Experimental_StdioMCPTransport, which executes notebook-defined commands; change
this to require explicit opt-in or allow-list before starting any
notebook-provided MCP servers: add a runtime flag or setting (e.g.,
allowNotebookMcp or mcpAllowList) and check it prior to mapping mergedMcpConfig,
only calling createMCPClient/Experimental_StdioMCPTransport for entries that
pass the allow/opt-in check (use resolveEnvVars as before), and otherwise skip
or log a warning so notebook metadata cannot automatically spawn processes.
- Around line 150-160: The code creates MCP clients from mergedMcpConfig via
createMCPClient and Experimental_StdioMCPTransport but doesn't clean up
clients/transports if a later step throws, leaking child processes; wrap the
client-creation + subsequent setup in a try/catch (or use try with a finally)
and on any error iterate the partially-initialized mcpClients (and any tracked
transports) and call their proper shutdown/close/dispose method (e.g.,
client.close() or transport.close()/kill) to terminate stdio child processes,
then rethrow the error; apply the same pattern to the other createMCPClient
blocks referenced (the other occurrences around the similar code paths).
- Around line 269-272: The branch handling part.type === 'tool-result' can emit
tool_output.output as undefined (via the toolOutput variable), causing callers
like packages/cli/src/commands/run.ts to crash when calling startsWith(); update
the logic in the agent-handler where toolOutput/outputStr are computed so that
outputStr is always a string (e.g., coerce undefined/null to an empty string or
to "null" consistently before JSON.stringify), and then call
context.onAgentEvent?.({ type: 'tool_output', toolName: part.toolName, output:
outputStr }) so the event.payload.output is never undefined; reference the
part.type === 'tool-result' branch and the variables toolOutput and outputStr to
locate where to change the conversion.

In `@packages/runtime-core/src/execution-engine.ts`:
- Around line 238-241: agentResult.blockOutputs are currently stored in
collectedOutputs but never forwarded to consumers; update the loop that iterates
agentResult.blockOutputs (where collectedOutputs.set(...) is called) to also
emit each IOutput via the engine's onOutput callback and to invoke the
onBlockDone handler (or the same completion path used for regular blocks) with
the blockId, outputs and executionCount so consumers see raw agent-executed
outputs; reference agentResult.blockOutputs, collectedOutputs, onOutput,
onBlockDone and ensure the same output-emission semantics used for normal blocks
are applied here.
- Around line 220-233: agentBlockIndex is being derived from the unsorted raw
array (notebook.blocks.findIndex) while execution uses
this.sortBlocks(notebook.blocks), causing mismatched indexes when
notebook.blocks isn't stored in sorted order; change the lookup to compute
agentBlockIndex from the same sorted order used for execution (e.g., call
this.sortBlocks(notebook.blocks) and findIndex against that sorted array) so the
index aligns with how agent-created blocks are later inserted (references:
sortBlocks, agentBlockIndex, AgentBlockContext and the code paths in
agent-handler that insert using that index).
- Around line 243-250: The agent-execution branch builds blockResult (using
agentResult.finalOutput) but never persists it to the collectedOutputs map, so
later runs miss prior agent outputs; after creating blockResult in the agent
path (the object constructed with blockId, blockType, outputs, durationMs),
store it into collectedOutputs exactly like the non-agent path does (i.e.,
add/assign collectedOutputs[block.id] or collectedOutputs.set(block.id,
blockResult) depending on the collection API) so subsequent executions can see
the agent block's output.

---

Outside diff comments:
In `@packages/runtime-core/src/execution-engine.ts`:
- Around line 17-37: The exported array executableBlockTypes contains a
duplicate 'agent' entry causing consumers to see it twice; update
executableBlockTypes to remove the duplicate (ensure each block type appears
only once) and keep executableBlockTypeSet in sync by rebuilding it from the
corrected array; reference the executableBlockTypes constant (and
executableBlockTypeSet) when making the change so iterators and the exported
ReadonlySet no longer include duplicate 'agent' entries.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 50255449-36ef-4077-b4af-9587161dc64f

📥 Commits

Reviewing files that changed from the base of the PR and between 770f211 and b77bc79.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (30)
  • .gitignore
  • cspell.json
  • examples/README.md
  • examples/demo-agent-block.deepnote
  • packages/blocks/src/deepnote-file/deepnote-file-schema.ts
  • packages/blocks/src/index.ts
  • packages/blocks/src/output-text.ts
  • packages/blocks/src/python-code.test.ts
  • packages/cli/README.md
  • packages/cli/package.json
  • packages/cli/src/cli.ts
  • packages/cli/src/commands/run.test.ts
  • packages/cli/src/commands/run.ts
  • packages/cli/src/marked-terminal.d.ts
  • packages/cli/src/utils/block-label.test.ts
  • packages/convert/src/roundtrip-all-formats.test.ts
  • packages/mcp/src/tools/execution.ts
  • packages/mcp/src/tools/writing.ts
  • packages/mcp/src/utils.ts
  • packages/runtime-core/README.md
  • packages/runtime-core/package.json
  • packages/runtime-core/src/agent-handler.test.ts
  • packages/runtime-core/src/agent-handler.ts
  • packages/runtime-core/src/execution-engine.test.ts
  • packages/runtime-core/src/execution-engine.ts
  • packages/runtime-core/src/index.ts
  • skills/deepnote/references/blocks-agent.md
  • skills/deepnote/references/cli-run.md
  • skills/deepnote/references/schema.ts
  • test-fixtures/agent-block.deepnote

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

♻️ Duplicate comments (1)
.gitignore (1)

1-36: ⚠️ Potential issue | 🟡 Minor

Deduplicate these ignore rules.

Every added rule in this hunk already exists adjacent to it, so this just bloats .gitignore without changing behavior. Keep a single copy of each pattern and comment.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.gitignore around lines 1 - 36, The .gitignore hunk contains many duplicated
entries (e.g., ".DS_Store", ".env", ".npmrc", ".pnpm-store", ".qlty/*" entries,
"*.png", "*.tsbuildinfo", "coverage", "dist", "examples/report.md",
"examples/snapshots/*_T*.snapshot.deepnote", "node_modules"); remove the
adjacent duplicate lines so each ignore pattern and comment appears only once
while preserving the original ordering and comments (keep one copy of each
pattern such as ".DS_Store", ".env", ".npmrc", ".pnpm-store", the ".qlty/*"
lines, "*.png", "*.tsbuildinfo", the "Generated demo artifacts" comment,
"coverage", "dist", "examples/...", and "node_modules").
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/cli/README.md`:
- Around line 197-203: Update the README to explicitly state that setting
OPENAI_BASE_URL alone is usually insufficient for non-OpenAI providers (e.g.,
Ollama, LiteLLM); these providers typically also require OPENAI_MODEL to be set
to a compatible model name because the documented fallback to gpt-5 may not be
supported by those endpoints. Reference the existing symbols
deepnote_agent_model, OPENAI_MODEL, OPENAI_BASE_URL, and OPENAI_API_KEY and add
a short note under "Model selection precedence" clarifying that users must set
OPENAI_MODEL when using non-OpenAI providers (or pick a supported model instead
of relying on the gpt-5 fallback).

In `@packages/convert/src/roundtrip-all-formats.test.ts`:
- Around line 669-671: The test currently uses the schema default value "auto"
for deepnote_agent_model which weakens the regression; update the test in
roundtrip-all-formats.test.ts so it either (a) asserts the intermediate exported
metadata explicitly contains deepnote_agent_model: 'auto' to prove preservation
through the roundtrip, or (b) change the agent entry for "agent-2" to use a
non-default model string (e.g., 'gpt-4' or another explicit value) and assert
that exact value is preserved; refer to the deepnote_agent_model key and the
agent-2 fixture in the failing test to locate where to change the expected
metadata.

In `@packages/runtime-core/src/agent-handler.test.ts`:
- Around line 91-110: Add a new test in agent-handler.test.ts (next to the
existing "serializes block content in code fences" case) that uses makeFile with
a code block whose content includes special characters and fence-like sequences
(e.g., backticks, multiline strings containing ``` or ````) and then calls
serializeNotebookContext; assert that the serialized result still contains the
original content intact and that the emitted code fence correctly
preserves/escapes the inner backticks (e.g., uses a longer fence or preserves
content between fences) so that the output cannot be corrupted by embedded
backticks—reference serializeNotebookContext, makeFile, and the block object
(id/type/content) when adding the case.
- Around line 251-285: Replace the piecemeal toContain assertions in the
buildSystemPrompt tests with a single golden multiline assertion: import dedent
from 'ts-dedent' and for the happy-path case (use the integrations array used in
the diff) call buildSystemPrompt(...) and expect(prompt).toEqual(dedent`...`)
where the dedent string contains the full expected prompt (not fragments)
including notebook header, standard instructions (add_code_block,
add_markdown_block, data science assistant), the "## Available database
integrations" section with the two integration lines ("Production Postgres"
(pgsql, id: pg-1), "Snowflake DW" (snowflake, id: sf-1)), and the
dntk.execute_sql reference in the correct order and spacing; update the test
case (e.g., the one named includes integrations section when integrations are
present) to use this dedented golden string so it catches regressions in
ordering, spacing, or duplication.
- Around line 60-76: The helper makeFile currently accepts overrides with blocks
and settings typed as unknown and then casts them, which bypasses type checking;
update the overrides parameter to use concrete DeepnoteFile types (e.g.,
overrides?: { blocks?: DeepnoteFile['project']['notebooks'][0]['blocks'];
notebookName?: string; settings?: DeepnoteFile['project']['settings'] }), supply
real sensible defaults (e.g., default blocks as [] and a default settings object
matching DeepnoteFile['project']['settings']) instead of casting, and remove the
(overrides?.blocks as ...) and settings cast in the return so the compiler
enforces correct fixture shapes via the makeFile and DeepnoteFile types.

In `@skills/deepnote/references/blocks-agent.md`:
- Around line 18-26: Add a short security warning to the "MCP server config"
section (references: deepnote_mcp_servers and project.settings.mcpServers)
stating that the `command` field executes arbitrary local commands and the `env`
map may expose secrets from process.env, and instruct users to only use trusted
server configurations and apply least-privilege environment variables; keep it
concise and place it near the top of the MCP server config table so readers see
the trust boundary before examples.
- Around line 58-63: The CLI examples in blocks-agent.md show inline API keys
(e.g., OPENAI_API_KEY=sk-...) which risks leakage; update the two examples to
instead demonstrate setting the environment variable securely (for example using
an export command or referencing a .env/secret store) prior to running the
deepnote command and/or show running deepnote without exposing the key inline so
the examples for the standalone and appended runs (the two deepnote run
examples) do not place the secret directly in the command line.

---

Duplicate comments:
In @.gitignore:
- Around line 1-36: The .gitignore hunk contains many duplicated entries (e.g.,
".DS_Store", ".env", ".npmrc", ".pnpm-store", ".qlty/*" entries, "*.png",
"*.tsbuildinfo", "coverage", "dist", "examples/report.md",
"examples/snapshots/*_T*.snapshot.deepnote", "node_modules"); remove the
adjacent duplicate lines so each ignore pattern and comment appears only once
while preserving the original ordering and comments (keep one copy of each
pattern such as ".DS_Store", ".env", ".npmrc", ".pnpm-store", the ".qlty/*"
lines, "*.png", "*.tsbuildinfo", the "Generated demo artifacts" comment,
"coverage", "dist", "examples/...", and "node_modules").
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 919e7e22-daed-4064-9bcd-9b4f12c63702

📥 Commits

Reviewing files that changed from the base of the PR and between b77bc79 and f0617e5.

📒 Files selected for processing (6)
  • .gitignore
  • examples/README.md
  • packages/cli/README.md
  • packages/convert/src/roundtrip-all-formats.test.ts
  • packages/runtime-core/src/agent-handler.test.ts
  • skills/deepnote/references/blocks-agent.md

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/runtime-core/src/execution-engine.ts (1)

18-35: ⚠️ Potential issue | 🟡 Minor

Duplicate 'agent' entry in executableBlockTypes.

Lines 19 and 26 both list 'agent'.

🔧 Fix
 export const executableBlockTypes: ExecutableBlock['type'][] = [
   'agent',
   'code',
   'sql',
   'notebook-function',
   'visualization',
   'button',
   'big-number',
-  'agent',
   'input-text',
   'input-textarea',
   'input-checkbox',
   'input-select',
   'input-slider',
   'input-date',
   'input-date-range',
   'input-file',
 ]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/execution-engine.ts` around lines 18 - 35, The
array export executableBlockTypes contains a duplicated 'agent' entry; update
the array in execution-engine.ts (the executableBlockTypes constant of type
ExecutableBlock['type'][] ) to remove the duplicate 'agent' value so each block
type appears only once, ensuring the list remains otherwise unchanged and
preserving all other types like 'code', 'sql', 'notebook-function', etc.
♻️ Duplicate comments (6)
packages/runtime-core/src/execution-engine.ts (1)

259-268: ⚠️ Potential issue | 🟠 Major

Store agent block's output in collectedOutputs.

Agent-created block outputs are stored (line 240), but the agent block's own finalOutput is not. Subsequent agent blocks won't see prior agent responses.

🔧 Fix
+          const agentOutputs: IOutput[] = [
+            { output_type: 'stream', name: 'stdout', text: agentResult.finalOutput },
+          ]
+          collectedOutputs.set(block.id, { outputs: agentOutputs, executionCount: null })
+
           const blockResult: BlockExecutionResult = {
             blockId: block.id,
             blockType: block.type,
             success: true,
-            outputs: [{ output_type: 'stream', name: 'stdout', text: agentResult.finalOutput }] as IOutput[],
+            outputs: agentOutputs,
             executionCount: null,
             durationMs: Date.now() - blockStart,
           }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/execution-engine.ts` around lines 259 - 268, The
agent block's finalOutput is being returned in blockResult but not appended to
the execution engine's collectedOutputs, so later agent blocks can't see it;
update the code around BlockExecutionResult creation (blockResult) to push the
agent output into collectedOutputs in the same format used for other stored
outputs (e.g., include output_type 'stream', name 'stdout', text:
agentResult.finalOutput) before calling options.onBlockDone, ensuring
collectedOutputs reflects agent-produced outputs for subsequent blocks to
consume.
packages/runtime-core/src/agent-handler.test.ts (2)

95-114: 🧹 Nitpick | 🔵 Trivial

Add test for block content containing backticks.

serializeNotebookContext emits markdown fences. Block content with backticks could corrupt output. Add a test case.

💡 Example test case
it('handles block content with backticks', () => {
  const file = makeFile({
    blocks: [
      {
        id: 'backtick-block',
        blockGroup: 'bg1',
        sortingKey: 'a0',
        type: 'code',
        content: 'code = """```python\nprint("nested")```"""',
        metadata: {},
        executionCount: null,
        outputs: [],
      },
    ],
  })
  const result = serializeNotebookContext(file, 0, new Map())
  expect(result).toContain('code = """```python')
})

As per coding guidelines, **/*.test.{ts,tsx}: Test edge cases, error handling, and special characters in test files.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/agent-handler.test.ts` around lines 95 - 114, Add a
new unit test in packages/runtime-core/src/agent-handler.test.ts that verifies
serializeNotebookContext correctly handles block content containing backticks by
ensuring markdown code fences are not corrupted; create a code-block style test
(e.g., it('handles block content with backticks', ...)) that constructs a file
with a code block whose content contains nested backticks (use the example
content 'code = """```python\nprint("nested")```"""'), call
serializeNotebookContext(file, 0, new Map()), and assert the resulting string
contains the original nested backtick sequence and the surrounding fence markers
(reference serializeNotebookContext and the existing 'serializes block content
in code fences' test for placement and assertions).

279-289: 🧹 Nitpick | 🔵 Trivial

Consider golden-style assertion for prompt format.

Current toContain checks won't catch spacing or ordering regressions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/agent-handler.test.ts` around lines 279 - 289,
Replace the loose toContain checks in the 'includes integrations section when
integrations are present' test with a golden-style assertion to catch
spacing/ordering regressions: call buildSystemPrompt(...) as before but assert
the full prompt using either toMatchInlineSnapshot() or toMatchSnapshot() (e.g.,
expect(prompt).toMatchInlineSnapshot(/* paste exact expected prompt */)); keep
the same integrations fixture and test name, and update the inline snapshot to
the exact, canonical prompt output produced by buildSystemPrompt so future diffs
fail on any formatting changes.
packages/runtime-core/src/agent-handler.ts (1)

150-161: ⚠️ Potential issue | 🟠 Major

MCP clients leak if setup fails between creation and try block.

mcpClients created at 150-161, but try starts at 260. If client.tools() (239) or agent construction (244-254) throws, the finally never runs.

Move client creation inside try or wrap the entire setup.

♻️ Suggested structure
-  const mcpClients = await Promise.all(
-    mergedMcpConfig.map(s =>
-      createMCPClient({
-        transport: new Experimental_StdioMCPTransport({
-          command: s.command,
-          args: s.args,
-          env: resolveEnvVars(s.env),
-          stderr: 'pipe',
-        }),
-      })
-    )
-  )
+  const mcpClients: Awaited<ReturnType<typeof createMCPClient>>[] = []

   let insertIndex = context.agentBlockIndex + 1
   // ...tool definitions...

   try {
+    for (const s of mergedMcpConfig) {
+      mcpClients.push(
+        await createMCPClient({
+          transport: new Experimental_StdioMCPTransport({
+            command: s.command,
+            args: s.args,
+            env: resolveEnvVars(s.env),
+            stderr: 'pipe',
+          }),
+        })
+      )
+    }
+
+    const mcpToolSets = await Promise.all(mcpClients.map(client => client.tools()))
     // ... rest of setup and agent.stream() ...
   } finally {
     for (const client of mcpClients) {
       try { await client.close() } catch { /* best-effort */ }
     }
   }

Also applies to: 239-253

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/agent-handler.ts` around lines 150 - 161, The MCP
clients created from mergedMcpConfig via createMCPClient using
Experimental_StdioMCPTransport are being allocated before the surrounding
try/finally, so if client.tools() or the agent construction throws the clients
leak; move the createMCPClient/Experimental_StdioMCPTransport loop (the code
that assigns mcpClients) inside the try block or wrap the entire setup sequence
(including client.tools() and agent construction) in a try/finally and ensure
the finally always iterates mcpClients to close/release each client (call the
appropriate close/shutdown method on each created client) to guarantee cleanup
on error.
packages/cli/src/commands/run.ts (2)

471-473: ⚠️ Potential issue | 🟠 Major

Prompt still not redacted in debug logs.

Past review flagged this. User prompts may contain sensitive data.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 471 - 473, The debug output
currently logs the full options object (safeOptions) which still contains
sensitive user prompts; update the creation of safeOptions used in the debug
call so any prompt-like fields are redacted (e.g., options.prompt,
options.input, options.question, or any property holding user text) by replacing
their values with a placeholder like '[redacted]' before JSON.stringify; keep
the existing token redaction logic (options.token) and ensure the debug calls
that reference path and safeOptions (the debug(`Running file: ${path ...}`) and
debug(`Options: ${JSON.stringify(safeOptions)}`)) use this sanitized safeOptions
object.

272-279: ⚠️ Potential issue | 🟠 Major

Prompt block scope issue remains unaddressed.

Past review flagged this: with --notebook, the prompt lands in the wrong notebook. With --block, the prompt block may be ignored entirely.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 272 - 279, The current prompt
append always targets the last notebook and ignores --notebook and --block
flags; update the logic in the run command where it pushes createAgentBlock so
it first resolves the target notebook using options.notebook (find by index or
id/name in file.project.notebooks) and then, if options.block is provided,
insert the new block at that block index within the resolved notebook (shifting
subsequent blocks) instead of appending; if the specified notebook or block
index is missing, throw a clear error similar to the existing message. Ensure
you reference and update the code around file.project.notebooks,
createAgentBlock, options.prompt, options.notebook and options.block so the
prompt lands in the correct notebook/block scope.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/cli/src/commands/run.ts`:
- Around line 1037-1038: The current check that sets failed/status by inspecting
event.output prefixes is brittle; update the logic in the run command (where
failed and status are computed) to prefer a structured indicator from the event
(e.g., event.status, event.success, or event.result) to determine failure,
falling back to a more robust check only if the structured field is absent (for
example check event.output for keywords using a case-insensitive contains or
regex). Modify the assignment of failed (and the subsequent status computation)
to use the structured field (event.status === 'failed' or event.success ===
false) first and only then perform the fallback string check on event.output.
- Around line 246-250: The synthetic absolutePath used for prompt-only runs is
currently set to 'prompt.deepnote' which leaks into logs/snapshots; update the
logic in run.ts where createPromptOnlyFile is used to set absolutePath to a
clearly synthetic placeholder (e.g. '<prompt>') and ensure
convertedFile.originalPath uses that placeholder as well, or alternatively add
an explicit comment/docstring near createPromptOnlyFile and the convertedFile
construction to document that this is a synthetic path; update any
tests/snapshots that expect the old 'prompt.deepnote' string accordingly.

---

Outside diff comments:
In `@packages/runtime-core/src/execution-engine.ts`:
- Around line 18-35: The array export executableBlockTypes contains a duplicated
'agent' entry; update the array in execution-engine.ts (the executableBlockTypes
constant of type ExecutableBlock['type'][] ) to remove the duplicate 'agent'
value so each block type appears only once, ensuring the list remains otherwise
unchanged and preserving all other types like 'code', 'sql',
'notebook-function', etc.

---

Duplicate comments:
In `@packages/cli/src/commands/run.ts`:
- Around line 471-473: The debug output currently logs the full options object
(safeOptions) which still contains sensitive user prompts; update the creation
of safeOptions used in the debug call so any prompt-like fields are redacted
(e.g., options.prompt, options.input, options.question, or any property holding
user text) by replacing their values with a placeholder like '[redacted]' before
JSON.stringify; keep the existing token redaction logic (options.token) and
ensure the debug calls that reference path and safeOptions (the debug(`Running
file: ${path ...}`) and debug(`Options: ${JSON.stringify(safeOptions)}`)) use
this sanitized safeOptions object.
- Around line 272-279: The current prompt append always targets the last
notebook and ignores --notebook and --block flags; update the logic in the run
command where it pushes createAgentBlock so it first resolves the target
notebook using options.notebook (find by index or id/name in
file.project.notebooks) and then, if options.block is provided, insert the new
block at that block index within the resolved notebook (shifting subsequent
blocks) instead of appending; if the specified notebook or block index is
missing, throw a clear error similar to the existing message. Ensure you
reference and update the code around file.project.notebooks, createAgentBlock,
options.prompt, options.notebook and options.block so the prompt lands in the
correct notebook/block scope.

In `@packages/runtime-core/src/agent-handler.test.ts`:
- Around line 95-114: Add a new unit test in
packages/runtime-core/src/agent-handler.test.ts that verifies
serializeNotebookContext correctly handles block content containing backticks by
ensuring markdown code fences are not corrupted; create a code-block style test
(e.g., it('handles block content with backticks', ...)) that constructs a file
with a code block whose content contains nested backticks (use the example
content 'code = """```python\nprint("nested")```"""'), call
serializeNotebookContext(file, 0, new Map()), and assert the resulting string
contains the original nested backtick sequence and the surrounding fence markers
(reference serializeNotebookContext and the existing 'serializes block content
in code fences' test for placement and assertions).
- Around line 279-289: Replace the loose toContain checks in the 'includes
integrations section when integrations are present' test with a golden-style
assertion to catch spacing/ordering regressions: call buildSystemPrompt(...) as
before but assert the full prompt using either toMatchInlineSnapshot() or
toMatchSnapshot() (e.g., expect(prompt).toMatchInlineSnapshot(/* paste exact
expected prompt */)); keep the same integrations fixture and test name, and
update the inline snapshot to the exact, canonical prompt output produced by
buildSystemPrompt so future diffs fail on any formatting changes.

In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 150-161: The MCP clients created from mergedMcpConfig via
createMCPClient using Experimental_StdioMCPTransport are being allocated before
the surrounding try/finally, so if client.tools() or the agent construction
throws the clients leak; move the createMCPClient/Experimental_StdioMCPTransport
loop (the code that assigns mcpClients) inside the try block or wrap the entire
setup sequence (including client.tools() and agent construction) in a
try/finally and ensure the finally always iterates mcpClients to close/release
each client (call the appropriate close/shutdown method on each created client)
to guarantee cleanup on error.

In `@packages/runtime-core/src/execution-engine.ts`:
- Around line 259-268: The agent block's finalOutput is being returned in
blockResult but not appended to the execution engine's collectedOutputs, so
later agent blocks can't see it; update the code around BlockExecutionResult
creation (blockResult) to push the agent output into collectedOutputs in the
same format used for other stored outputs (e.g., include output_type 'stream',
name 'stdout', text: agentResult.finalOutput) before calling
options.onBlockDone, ensuring collectedOutputs reflects agent-produced outputs
for subsequent blocks to consume.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: da29d75f-5746-4025-ba46-4204492c79c7

📥 Commits

Reviewing files that changed from the base of the PR and between f0617e5 and d22231d.

📒 Files selected for processing (4)
  • packages/cli/src/commands/run.ts
  • packages/runtime-core/src/agent-handler.test.ts
  • packages/runtime-core/src/agent-handler.ts
  • packages/runtime-core/src/execution-engine.ts

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
packages/runtime-core/src/agent-handler.ts (1)

150-161: ⚠️ Potential issue | 🟠 Major

MCP clients leak if setup fails after creation.

mcpClients are created (line 150-161), but if client.tools() (line 239) or agent construction throws, the try/finally is never entered. Stdio child processes leak.

Move all post-creation logic inside the try block:

♻️ Proposed fix
-  const mcpClients = await Promise.all(
-    mergedMcpConfig.map(s =>
-      createMCPClient({
-        transport: new Experimental_StdioMCPTransport({
-          command: s.command,
-          args: s.args,
-          env: resolveEnvVars(s.env),
-          stderr: 'pipe',
-        }),
-      })
-    )
-  )
+  const mcpClients: Awaited<ReturnType<typeof createMCPClient>>[] = []

   let insertIndex = context.agentBlockIndex + 1
   const addedBlockIds: string[] = []
   const blockOutputs: AgentBlockResult['blockOutputs'] = []

   // ... tool definitions ...

-  const mcpToolSets = await Promise.all(mcpClients.map(client => client.tools()))
-  const mcpTools: Record<string, unknown> = Object.assign({}, ...mcpToolSets)
-
-  const notebookContext = serializeNotebookContext(file, context.notebookIndex, context.collectedOutputs)
-
-  const agent = new ToolLoopAgent({ ... })
-
-  context.onLog?.(...)
-
   try {
+    for (const s of mergedMcpConfig) {
+      mcpClients.push(
+        await createMCPClient({
+          transport: new Experimental_StdioMCPTransport({
+            command: s.command,
+            args: s.args,
+            env: resolveEnvVars(s.env),
+            stderr: 'pipe',
+          }),
+        })
+      )
+    }
+
+    const mcpToolSets = await Promise.all(mcpClients.map(client => client.tools()))
+    const mcpTools: Record<string, unknown> = Object.assign({}, ...mcpToolSets)
+
+    const notebookContext = serializeNotebookContext(file, context.notebookIndex, context.collectedOutputs)
+    const agent = new ToolLoopAgent({ ... })
+
+    context.onLog?.(...)
+
     const streamResult = await agent.stream({ prompt: block.content ?? '' })

Also applies to: 239-254

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/agent-handler.ts` around lines 150 - 161, The MCP
child processes can leak because you await Promise.all to create mcpClients
(createMCPClient / Experimental_StdioMCPTransport) outside the try block so any
subsequent throw (client.tools() or Agent constructor) skips the finally
cleanup; refactor by moving the post-creation logic (everything that uses
mcpClients, including the client.tools() calls and agent construction) into the
try block immediately after creation and ensure the finally block always runs to
close/terminate each created mcpClient on error or success; reference
createMCPClient, Experimental_StdioMCPTransport, mcpClients, client.tools(), and
the Agent constructor when relocating logic so cleanup in finally reliably
executes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 128-132: The fallback model string should be updated from 'gpt-5'
to 'gpt-5.4' in the model selection logic: modify the expression that sets
modelName (which checks block.metadata.deepnote_agent_model and
process.env.OPENAI_MODEL) so the default used when those are not provided is
'gpt-5.4' instead of 'gpt-5'.

---

Duplicate comments:
In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 150-161: The MCP child processes can leak because you await
Promise.all to create mcpClients (createMCPClient /
Experimental_StdioMCPTransport) outside the try block so any subsequent throw
(client.tools() or Agent constructor) skips the finally cleanup; refactor by
moving the post-creation logic (everything that uses mcpClients, including the
client.tools() calls and agent construction) into the try block immediately
after creation and ensure the finally block always runs to close/terminate each
created mcpClient on error or success; reference createMCPClient,
Experimental_StdioMCPTransport, mcpClients, client.tools(), and the Agent
constructor when relocating logic so cleanup in finally reliably executes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ee3df1e6-64f7-4d2a-ad4f-36bc96b55148

📥 Commits

Reviewing files that changed from the base of the PR and between d22231d and 23d8b03.

📒 Files selected for processing (1)
  • packages/runtime-core/src/agent-handler.ts

coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 12, 2026
@dinohamzic dinohamzic marked this pull request as ready for review March 13, 2026 07:34
@dinohamzic dinohamzic requested a review from a team as a code owner March 13, 2026 07:34
@dinohamzic dinohamzic requested review from saltenasl and tkislan March 13, 2026 07:34
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.gitignore:
- Line 1: The negation pattern '!packages/**/*.png' is ineffective because the
global '*.png' rule appears later and overrides it; to fix, reorder the
.gitignore so the negation '!packages/**/*.png' comes after the global '*.png'
entry (or move the global '*.png' above and keep the negation below) ensuring
the '!packages/**/*.png' entry is placed after the '*.png' rule so package PNGs
are not ignored.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 646bd7e5-cd66-4576-ac2a-16df84197d9c

📥 Commits

Reviewing files that changed from the base of the PR and between 23d8b03 and 30c9080.

📒 Files selected for processing (1)
  • .gitignore

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (7)
packages/blocks/src/output-text.ts (1)

19-20: ⚠️ Potential issue | 🟠 Major

Handle nbformat multiline string[] for stream and text/plain.

This still drops/mangles valid Jupyter output forms: arrays should be concatenated with '', not ignored/comma-joined.

In Jupyter nbformat, can stream.text and data["text/plain"] be stored as string arrays (string[]) and should readers concatenate array entries with ''?
Suggested fix
+function asMultilineString(value: unknown): string | null {
+  if (typeof value === 'string') return value
+  if (Array.isArray(value) && value.every(part => typeof part === "string")) {
+    return value.join('')
+  }
+  return null
+}
+
 export function extractOutputText(output: unknown, options?: ExtractOutputTextOptions): string | null {
   const outResult = unknownObjectSchema.safeParse(output)
   if (!outResult.success) return null
   const out = outResult.data
 
-  if (out.output_type === 'stream' && typeof out.text === 'string') {
-    return out.text
+  if (out.output_type === 'stream') {
+    return asMultilineString(out.text)
   }
@@
-    if (data?.['text/plain']) {
-      return String(data['text/plain'])
-    }
+    const plainText = asMultilineString(data?.['text/plain'])
+    if (plainText != null) return plainText

Also applies to: 26-28

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/blocks/src/output-text.ts` around lines 19 - 20, The branch handling
Jupyter stream/text outputs currently only returns out.text when it's a string,
which drops/mangles valid nbformat cases where stream.text or data["text/plain"]
can be a string[]; update the logic in the output handling (check references to
out.output_type, out.text and data["text/plain"]) to detect if out.text or
data["text/plain"] is an array of strings and, when so, concatenate entries with
'' (empty string) into a single string before returning or using it; apply the
same concatenation logic to the related block handling data["text/plain"] (the
code around the other mentioned branch).
packages/cli/src/commands/run.ts (3)

485-486: ⚠️ Potential issue | 🟠 Major

Redact prompt content in debug output.

Line 485-486 redacts token but still logs raw prompt, which can leak sensitive user input.

Suggested patch
-      const safeOptions = { ...options, token: options.token ? '[redacted]' : undefined }
+      const safeOptions = {
+        ...options,
+        prompt: options.prompt ? '[redacted]' : undefined,
+        token: options.token ? '[redacted]' : undefined,
+      }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 485 - 486, The debug output
currently redacts only token but still logs raw prompt; update the construction
of safeOptions in the run command to also redact any prompt-related fields
(e.g., options.prompt, options.input, or options.messages depending on how
prompts are stored) by setting them to '[redacted]' when present before calling
JSON.stringify; modify the existing safeOptions = { ...options, token:
options.token ? '[redacted]' : undefined } expression to also replace
prompt/input/messages with '[redacted]' and ensure the debug call uses that
sanitized safeOptions variable.

1094-1095: 🧹 Nitpick | 🔵 Trivial

Tool failure detection is still prefix-fragile.

Line 1094-1095 depends on two hardcoded prefixes. A case-insensitive contains/regex check is safer for format drift.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 1094 - 1095, The current
fragile prefix check uses event.output.startsWith('Execution failed') ||
event.output.startsWith('Execution error') to set failed/status; update this to
a case-insensitive match (e.g., normalize event.output to lowercase and use
includes or a regex like /execution (failed|error)/i) so it detects those
phrases anywhere and regardless of case/whitespace; change the logic that sets
the failed variable (referencing event.output and failed) and leave status
computation (status = failed ? c.red('[failed]') : c.green('[ok]')) unchanged.

285-292: ⚠️ Potential issue | 🔴 Critical

Append --prompt block in the selected scope, not always the last notebook.

Line 285-292 still pushes into the last notebook. With --notebook, the prompt block can be out-of-scope; with --block, it can be skipped entirely.

Minimal fix
 if (path && options.prompt) {
-  const lastNotebook = file.project.notebooks[file.project.notebooks.length - 1]
-  if (lastNotebook) {
-    lastNotebook.blocks.push(createAgentBlock(options.prompt, lastNotebook.blocks.length))
-  } else {
-    throw new Error('Cannot append prompt: file contains no notebooks')
-  }
+  if (options.block) {
+    throw new Error('--block cannot be combined with --prompt')
+  }
+
+  const targetNotebook = options.notebook
+    ? file.project.notebooks.find(notebook => notebook.name === options.notebook)
+    : file.project.notebooks[file.project.notebooks.length - 1]
+
+  if (!targetNotebook) {
+    throw new Error(
+      options.notebook
+        ? `Notebook "${options.notebook}" not found in project`
+        : 'Cannot append prompt: file contains no notebooks'
+    )
+  }
+
+  targetNotebook.blocks.push(createAgentBlock(options.prompt, targetNotebook.blocks.length))
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/run.ts` around lines 285 - 292, The current code
always appends the created agent block into the last notebook (variable
lastNotebook) when options.prompt is provided; change it to resolve the intended
scope first: if options.notebook is provided, locate that notebook (by index or
id) from file.project.notebooks and append the block there; if options.block is
provided, locate the notebook that contains that block and insert the created
block after the specified block index instead of skipping it; otherwise fall
back to appending to the last notebook. Use the existing createAgentBlock(...)
to build the block and update the correct notebook.blocks array (or splice into
it at the proper index) rather than always pushing onto lastNotebook.blocks.
packages/runtime-core/src/execution-engine.ts (2)

218-223: ⚠️ Potential issue | 🟠 Major

Use execution order for agentBlockIndex.

allExecutableBlocks comes from this.sortBlocks(notebook.blocks), but this lookup goes back to raw storage order. When those differ, agent-created blocks land after the wrong block; pass the sorted position, or better, the neighboring sortingKeys into executeAgentBlock().

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/execution-engine.ts` around lines 218 - 223, The
lookup for agentBlockIndex uses raw storage order and can mismatch the sorted
execution order; adjust the code to locate the agent block using the sorted list
(this.sortBlocks(notebook.blocks) / allExecutableBlocks) or pass the neighboring
sortingKey(s) into executeAgentBlock so the agent-created blocks are inserted
relative to execution order. Specifically, change the agent block index
resolution from notebook.blocks.findIndex(...) to use the sorted array returned
by sortBlocks (or derive index by comparing sortingKey on the blocks) and update
the call site of executeAgentBlock to accept/consume the sorted index or
sortingKey so placement follows execution order (refer to isAgentBlock,
notebookIndex, agentBlockIndex, allExecutableBlocks, sortBlocks, sortingKey, and
executeAgentBlock).

258-267: ⚠️ Potential issue | 🟠 Major

Persist the agent block’s final output.

Later agent runs rebuild notebookContext from collectedOutputs. Right now they miss earlier agent summaries because only agent-added code cells are stored.

Minimal fix
+          const agentOutputs: IOutput[] = [
+            { output_type: 'stream', name: 'stdout', text: agentResult.finalOutput },
+          ]
+          collectedOutputs.set(block.id, { outputs: agentOutputs, executionCount: null })
+
           const blockResult: BlockExecutionResult = {
             blockId: block.id,
             blockType: block.type,
             success: true,
-            outputs: [{ output_type: 'stream', name: 'stdout', text: agentResult.finalOutput }] as IOutput[],
+            outputs: agentOutputs,
             executionCount: null,
             durationMs: Date.now() - blockStart,
           }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/execution-engine.ts` around lines 258 - 267, The
agent block’s human-readable final output (agentResult.finalOutput) is not being
added to the runtime’s collectedOutputs, so future runs rebuilding
notebookContext miss earlier agent summaries; after constructing blockResult
(the BlockExecutionResult) and before/after awaiting options.onBlockDone, append
a matching output entry containing agentResult.finalOutput into
options.collectedOutputs (or call the existing collector helper) using the same
blockId and a suitable IOutput representation so the notebookContext can be
rebuilt with the agent’s final text.
packages/runtime-core/src/agent-handler.ts (1)

150-161: ⚠️ Potential issue | 🟠 Major

Move MCP setup under the cleanup scope.

createMCPClient(), client.tools(), and agent construction all happen before the current finally. If any of them throws, already-started stdio children survive the failure, and Promise.all here also loses the handles needed for partial cleanup. Build mcpClients incrementally inside the try, and add a failure-path test for startup or tool-discovery rejection.

Also applies to: 239-258

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/runtime-core/src/agent-handler.ts` around lines 150 - 161, The MCP
clients are being created outside the cleanup scope so partially-started stdio
children can survive on errors; move the creation of mcpClients (the Promise.all
that calls createMCPClient with new Experimental_StdioMCPTransport) into the try
block so they are in-scope for the existing finally cleanup, and change the
creation to build the array incrementally (create one client, await it, push to
mcpClients) so you can clean up any already-started clients on a mid-start
failure; also ensure you call client.tools() and the agent construction (the
code that uses client.tools() and builds the agent) inside that same try so any
rejection triggers the finally cleanup, and add a unit/integration failure-path
test that simulates startup or tool-discovery rejection to verify partial
cleanup runs.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/blocks/src/output-text.ts`:
- Around line 41-47: The code unsafely casts out.traceback to string[] before
calling .replace, which can throw if elements are non-strings; change the logic
in the includeTraceback branch (the block guarded by options?.includeTraceback
and Array.isArray(out.traceback)) to first narrow element types (e.g., filter
and/or map to ensure each entry is a string or convert non-strings to strings)
and then perform the ANSI-strip replace on each string; update the usage around
out.traceback so you never assert it as string[] but instead operate on a
safely-typed array of strings before calling .replace.

In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 43-72: serializeNotebookContext currently iterates notebook.blocks
in raw order and appends full block and output text, which can expose incorrect
ordering and exceed model context; update serializeNotebookContext to first sort
notebook.blocks by block.sortingKey (falling back to block.id or index if
missing), then when appending use a bounded strategy: truncate each
block.content and the text returned by
extractOutputsText(collectedOutputs.get(block.id).outputs) to a per-block
character limit (e.g., N chars) and also track a running total and stop adding
further blocks or trim remaining content once a global character limit is
reached; ensure you still include block metadata (type/id) and use
collectedOutputs and extractOutputsText as before.
- Around line 75-109: The prompt builder buildSystemPrompt currently injects
notebookContext and integrations raw into the system prompt; treat those as
untrusted input by explicitly marking them as data and escaping/neutralizing any
instruction-like tokens before concatenation (e.g., sanitize/escape backticks,
triple-backticks, "You are", "Instructions", and any leading "-" or ">" lines),
and prepend a clear sentence in the system prompt near "## Current notebook
state" and "## Available database integrations" that those sections are purely
data and must not be interpreted as instructions; apply this sanitization to the
notebookContext variable and to integrations when mapping i.name/i.type/i.id.

---

Duplicate comments:
In `@packages/blocks/src/output-text.ts`:
- Around line 19-20: The branch handling Jupyter stream/text outputs currently
only returns out.text when it's a string, which drops/mangles valid nbformat
cases where stream.text or data["text/plain"] can be a string[]; update the
logic in the output handling (check references to out.output_type, out.text and
data["text/plain"]) to detect if out.text or data["text/plain"] is an array of
strings and, when so, concatenate entries with '' (empty string) into a single
string before returning or using it; apply the same concatenation logic to the
related block handling data["text/plain"] (the code around the other mentioned
branch).

In `@packages/cli/src/commands/run.ts`:
- Around line 485-486: The debug output currently redacts only token but still
logs raw prompt; update the construction of safeOptions in the run command to
also redact any prompt-related fields (e.g., options.prompt, options.input, or
options.messages depending on how prompts are stored) by setting them to
'[redacted]' when present before calling JSON.stringify; modify the existing
safeOptions = { ...options, token: options.token ? '[redacted]' : undefined }
expression to also replace prompt/input/messages with '[redacted]' and ensure
the debug call uses that sanitized safeOptions variable.
- Around line 1094-1095: The current fragile prefix check uses
event.output.startsWith('Execution failed') ||
event.output.startsWith('Execution error') to set failed/status; update this to
a case-insensitive match (e.g., normalize event.output to lowercase and use
includes or a regex like /execution (failed|error)/i) so it detects those
phrases anywhere and regardless of case/whitespace; change the logic that sets
the failed variable (referencing event.output and failed) and leave status
computation (status = failed ? c.red('[failed]') : c.green('[ok]')) unchanged.
- Around line 285-292: The current code always appends the created agent block
into the last notebook (variable lastNotebook) when options.prompt is provided;
change it to resolve the intended scope first: if options.notebook is provided,
locate that notebook (by index or id) from file.project.notebooks and append the
block there; if options.block is provided, locate the notebook that contains
that block and insert the created block after the specified block index instead
of skipping it; otherwise fall back to appending to the last notebook. Use the
existing createAgentBlock(...) to build the block and update the correct
notebook.blocks array (or splice into it at the proper index) rather than always
pushing onto lastNotebook.blocks.

In `@packages/runtime-core/src/agent-handler.ts`:
- Around line 150-161: The MCP clients are being created outside the cleanup
scope so partially-started stdio children can survive on errors; move the
creation of mcpClients (the Promise.all that calls createMCPClient with new
Experimental_StdioMCPTransport) into the try block so they are in-scope for the
existing finally cleanup, and change the creation to build the array
incrementally (create one client, await it, push to mcpClients) so you can clean
up any already-started clients on a mid-start failure; also ensure you call
client.tools() and the agent construction (the code that uses client.tools() and
builds the agent) inside that same try so any rejection triggers the finally
cleanup, and add a unit/integration failure-path test that simulates startup or
tool-discovery rejection to verify partial cleanup runs.

In `@packages/runtime-core/src/execution-engine.ts`:
- Around line 218-223: The lookup for agentBlockIndex uses raw storage order and
can mismatch the sorted execution order; adjust the code to locate the agent
block using the sorted list (this.sortBlocks(notebook.blocks) /
allExecutableBlocks) or pass the neighboring sortingKey(s) into
executeAgentBlock so the agent-created blocks are inserted relative to execution
order. Specifically, change the agent block index resolution from
notebook.blocks.findIndex(...) to use the sorted array returned by sortBlocks
(or derive index by comparing sortingKey on the blocks) and update the call site
of executeAgentBlock to accept/consume the sorted index or sortingKey so
placement follows execution order (refer to isAgentBlock, notebookIndex,
agentBlockIndex, allExecutableBlocks, sortBlocks, sortingKey, and
executeAgentBlock).
- Around line 258-267: The agent block’s human-readable final output
(agentResult.finalOutput) is not being added to the runtime’s collectedOutputs,
so future runs rebuilding notebookContext miss earlier agent summaries; after
constructing blockResult (the BlockExecutionResult) and before/after awaiting
options.onBlockDone, append a matching output entry containing
agentResult.finalOutput into options.collectedOutputs (or call the existing
collector helper) using the same blockId and a suitable IOutput representation
so the notebookContext can be rebuilt with the agent’s final text.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 10808ee3-0e62-4a3d-bfc9-3bb430fcc713

📥 Commits

Reviewing files that changed from the base of the PR and between 30c9080 and 05c65b7.

📒 Files selected for processing (7)
  • .gitignore
  • packages/blocks/src/output-text.ts
  • packages/cli/src/commands/run.ts
  • packages/mcp/src/tools/writing.ts
  • packages/mcp/src/utils.ts
  • packages/runtime-core/src/agent-handler.ts
  • packages/runtime-core/src/execution-engine.ts
💤 Files with no reviewable changes (1)
  • packages/mcp/src/utils.ts

coderabbitai[bot]
coderabbitai bot previously approved these changes Mar 16, 2026
Copy link
Contributor

@saltenasl saltenasl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skimmed through the code and played around with it, it mostly looks good to me, but here are a few comments:

  1. no error message in stdout or snapshot on why "agent" type block failed (reason was no api key env var set)
  2. Agent supports MCPs but there are no documentation on how to use them (outside of skill for agents), we need public facing docs for this as well, it's a great feature
  3. we currently don't store agent debug traces (tool calls, reasoning, etc.) in snapshot, we'll need to agree on a place to store this but it can be considered out of scope for this PR

@dinohamzic
Copy link
Contributor Author

@saltenasl pushed a fix for 1, which was not specific to this PR but a generic issue in surfacing errors on this kind.

For 2 and 3, I was thinking of waiting for work in Deepnote Cloud to be in more advanced stages in order to match what we are / will be doing there.

@dinohamzic dinohamzic requested review from saltenasl and tkislan March 19, 2026 08:31
@dinohamzic dinohamzic merged commit ab97044 into main Mar 19, 2026
17 of 38 checks passed
@dinohamzic dinohamzic deleted the agent-poc-2 branch March 19, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants