Codemode Addendum: Handing Multi-step Tool Work to a Script

Xiao A had the Agent call six tools in a row: search files, read two results, look up an issue, and finally stitch four text fragments into a summary. Every call made the model decide the next step again, and the long tool output all poured into the context. Staring at the ever-longer conversation, he asked Lao Z: "Can the model just write a small program, have it orchestrate the tool calls, and hand back only the conclusion?"

"That interface in Pi is Codemode," Lao Z said. "The model submits JavaScript as one tool call. The script can chain, parallelize, and filter other tools, and only the script's output reaches the model's context. Say the boundaries up front: a failed script does not undo the tool side effects that already happened, and QuickJS's capability limits are not an OS-level security sandbox."

This addendum serves Part II and Part III: consult it when reading Part II Chapter 8's tools and extensions, or when you want the Agent to combine tool calls in batches. It is anchored to upstream tag v1.0.0 (commit a13d35a742c6ef8462812a28fbe1d8c8b7431c32). Pi's integration lives in packages/coding-agent/src/extensions/codemode/ and core/nested-tool-calls.ts; the standalone sandbox package is packages/codemode. It explains the script model, nested calls, discovery, state, model tools, and failure boundaries — without turning it into a general workflow engine.

It moves where the tool calls are orchestrated

A normal tool call is roughly "the model proposes a call → the host executes → the result returns to the model." Codemode folds multiple internal steps into one script:

model submits JavaScript
    ↓
QuickJS runs the script
    ├─ tools.read(...) ──┐
    ├─ tools.grep(...) ──┼─ results come back to the script, which filters or merges them
    └─ tools.bash(...) ──┘
    ↓
text(...) / return value
    ↓
only the tidied output reaches the model

"This is not the model bypassing tools — it moves the multi-step control flow into the script," Lao Z said. "The model still decides what script to write, and the host still runs the tools. The difference is that intermediate results can be handled by deterministic code first, without every step returning to the model to decide the next."

A fitting example is reading several independent files in parallel and returning only the lines with a marker:

const paths = ["src/a.ts", "src/b.ts", "src/c.ts"];
const results = await Promise.allSettled(
  paths.map((path) => tools.read({ path })),
);

for (let index = 0; index < results.length; index++) {
  const result = results[index];
  if (result.status === "fulfilled") {
    const matches = result.value.split("\n").filter((line) => line.includes("TODO"));
    if (matches.length > 0) text(`${paths[index]}\n${matches.join("\n")}`);
  } else {
    text(`${paths[index]}: ${result.reason}`);
  }
}

Promise.allSettled() lets the other results still be processed when one read fails. The script returns only the matching lines to the model; the full file contents stay inside the script's execution. The filtering logic is plain JavaScript — the model does not need to do another text filter.

"Say 'in parallel' precisely," Lao Z cautioned. "Whether multiple calls actually run at once depends on the session's tool-execution mode: under a global sequential mode, or when a tool declares executionMode: \"sequential\", nested calls run serially; otherwise they run concurrently. The script reads the same either way; whether they actually interleave is the host's decision."

QuickJS limits what a script can touch directly

@earendil-works/pi-codemode runs a QuickJS WebAssembly VM on a Worker thread. The script has no Node APIs, no filesystem, no network requests, no process, and no timers. What it can reach on the host side are the injected functions such as tools and models. The host owns deadlines, abort, and the Worker lifecycle; every execution creates a fresh VM, so one run's state does not leak into the next. The sandbox (worker, QuickJS wasm module) loads on the first call, not at startup.

The script input is a raw JavaScript function body, so top-level await and return work. It accumulates output with text(), image(), or console.*, ends early and successfully with exit(), or simply returns a value. Tool names are converted to JavaScript identifiers: e.g. the MCP tool mcp__dev-radius__search is callable as tools.mcp__dev_radius__search(...). An optional first line // @options: {...} declares the output budget and a deadline; a parse failure throws CodemodeSourceError, and the options line is replaced by a blank line so stack-trace line numbers still line up with the script.

Two runtime rules run counter to intuition: promises not awaited when the script ends are discarded, and running calls are cancelled through their signal; and because the VM has no timers, awaiting a promise that can never settle fails immediately instead of hanging.

Capability boundaries

• A script cannot read files or touch the network on its own; it can only ask host-provided tools to do those actions.
• A tool's permissions are still decided by the tool itself and Pi's tool pipeline. Codemode does not give a tool a smaller file or network permission.
• QuickJS's worker and memory limits control the script's execution environment; they do not roll back external side effects a tool already completed.
• A script cannot start another codemode script: the codemode tool registers at exposure level model-only, so only the model can initiate it.

Pi's integration runs nested calls through ctx.executeTool(). Argument validation, the tool_call／tool_result extension events, and permission extensions all go through the session's tool pipeline (runToolCall), the same path a model-issued call takes; the JSON Schema inside the sandbox only renders declarations, it does not validate values. Observability splits into two layers: the render layer's call rows record the tool name, an argument preview (truncated at 200 chars), status, duration, an error preview (truncated at 500 chars), and the cost of models.* calls; the session record's nestedCalls sit on the tool-result message with limits of at most 256 calls, 8 KiB of arguments per call, and 32 KiB total — over budget drops only the arguments or the call and sets complete: false, and a comment states plainly that this record is "kept for the session record; not sent to the model."

Intermediate results stay in the script; that does not erase side effects

A script can use Promise.all() for independent calls, or Promise.allSettled() to handle successes and failures separately. Call return values take two common shapes: a tool that declares an output schema can return a structured value; other tools usually return text. An MCP tool gives the script the whole CallToolResult, including structuredContent and isError.

"Mind MCP's error semantics," Lao Z said. "A call where the server returns isError: true is still a fulfilled result object on the script side — it does not reject; the script must check isError itself. What actually rejects is a call that fails, is blocked, or has invalid arguments."

Xiao A asked: "If the second tool call fails, are the things the first tool did rolled back automatically?"

"No," Lao Z said. "Codemode is an orchestration container, not a transaction manager. If write already wrote a file, or a remote API already accepted a request, those side effects happened even if a later step throws. The script result keeps its partial output and call state, with a line like 'tools called before the failure (they are not undone),' but it will not undo completed calls for you."

So a script must order steps by their nature: run independent read-only work in parallel first, then merge; when there are writes or outbound actions, clearly separate the prepare phase from the commit phase, and design idempotence for retries. Do not mistake a group of tool calls written in one JavaScript function for an atomic transaction.

The tool catalog expands on demand

Putting dozens of tools' full schemas into the prompt makes every model request carry a long tool description. Codemode provides ALL_TOOLS, searchTools(), describeTool(), and describeNamespace() for discovery. searchTools() ranks by BM25 relevance, defaults to 8 results, and can filter by namespace; describeNamespace() accepts mcp__dev-radius, mcp__dev_radius, dev-radius, and dev_radius.

Pi limits inline declarations with codemode.inlineBudget (default 3000 estimated tokens). Picking the catalog is not a simple truncation: selectCatalog rotates by namespace group, each round placing every group's cheapest remaining tool; a group whose next tool does not fit drops out while the others continue — every namespace gets a representative before any single group is listed in full.

Keep two exposure-level namespaces apart here: the MCP config layer's McpExposure (codemode/deferred/direct/hidden) and the pi tool layer's ToolExposure. An MCP tool configured codemode is registered into pi's layer as deferred, and pi-layer deferred tools are never listed in the codemode description — so a default-configured MCP tool does not appear in the codemode tool catalog, and a script finds it via searchTools(). The two indirect mechanisms reach each other: a script can call deferred tools, and tool_search can also load codemode tools.

This matters especially for MCP. An MCP server can carry a long tool list, and may connect only after the session starts; not stuffing their declarations into every request is exactly why Pi sets MCP tools to codemode exposure by default (see Appendix 11, MCP Addendum).

Pi has two Codemode presentation modes:

Mode	What the model sees	Where it fits
on (default)	Directly declared tools stay individually callable and their descriptions add the script-call form; the codemode description lists tools that are neither direct nor deferred	Gradual adoption, keeping the direct call path
only	Callable tools are listed centrally in the codemode description, and direct tools are no longer declared in requests	When you want every tool call orchestrated by a script

only hides only the declarations: those tools remain in the session's registry, remain callable by scripts, and their already-loaded declarations stay on the session branch — /tree, resume, and fork are unaffected.

Pi's v1.0.0 release notes report about 40% fewer prompt tokens under the default tool configuration — the comparison is the default tools plus active codemode, where one GPT-5.6 request shrinks from about 5,300 to about 3,300 tokens. That is the release notes' figure for that configuration, not a constant saving across all models, tool counts, and settings. 1.0 also improved errors: reading a nonexistent tools.Bash suggests the close match tools.bash, a malformed models.* argument returns the expected shape, and an unknown model points to models.getAvailableOfType().

Store is session state, not arbitrary persistence

store(key, value) and load(key) pass small JSON state between separate script executions. Pi appends the writes as codemode-store session entries only after a script succeeds; on resume or when switching to a branch, Pi rebuilds the store from the entries on that branch. So the state follows the session's history branch, not a global database. store(key, undefined) deletes a key; without a session context (a bare Agent or a direct tool call), load() starts empty and writes are dropped.

const cursor = load("cursor") ?? "";
const page = await tools.fetch_page({ cursor });

if (page.nextCursor) store("cursor", page.nextCursor);
text(JSON.stringify({ count: page.items.length, hasMore: Boolean(page.nextCursor) }));

The store suits cursors, task IDs, and short summaries — not images, whole tool output, or secrets. A single value is at most 256 * 1024 JSON characters and the total at most 1024 * 1024; exceeding the limit throws RangeError inside the script and the script fails. A failed script's store writes never land in the session, but that rule still does not undo the tool side effects its earlier calls produced.

models lets a script combine different model capabilities

Pi exposes the current session's model catalog and credential resolution to Codemode. A script can list and select classifier or image models, then call models.classify() or models.generateImages(); chat models are listed but cannot be re-invoked from a script. Model-call usage is added to the Codemode result and the session cost.

Within one script, classifier and image-model calls run at most four at a time; extra calls queue in submission order. Model-call failures come back as stopReason and errorMessage, and a script should check the result. Generated image blocks are not written to disk automatically; to hand one back to the current model, emit each with image(block). If a script generates images but never shows them, Pi appends a notice so they are not silently lost. image() accepts only base64 PNG, JPEG, GIF, and WebP, derives the MIME type from magic bytes rather than the declaration, and rejects remote URLs.

Credentials are resolved by Pi's model registry; a script selects an entry only by provider and id. The implementation does not expose a model entry's custom headers to the script (the headers field is stripped from catalog entries before handing them over), and resolves targets only from {provider, id} — a script-supplied baseUrl or headers will never receive the host's credentials.

Failures, budgets, and where it fits

Codemode failures come in four kinds: script parse／runtime errors, timeout, abort, and sandbox errors. A failure result keeps the partial output and nested call state already produced. The default output budget is 10,000 tokens (the constant DEFAULT_MAX_OUTPUT_TOKENS, counted at 4 characters per token): over budget, the text collapses into one "head…elided…tail" item and the full text is written to a temporary file on a best-effort basis with its path attached — if the write fails, the result reports the failure reason plainly and gives no path.

The deadline splits into two: Pi's integration defaults to no deadline when @options is absent (timeoutMs ?? Infinity), bounded only by abort and the memory cap; the standalone sandbox package CodemodeSandbox defaults to 300,000 ms (5 minutes). Pi's integration sets the QuickJS VM heap limit to 256 * 1024 * 1024 bytes (256 MiB; the upstream doc writes 256 MB), throwing InternalError: out of memory inside the script when exceeded. Image generation can take minutes, so set the script deadline with the task in mind, not a short one.

Work Codemode suits

• A batch of independent read-only tool calls, run in parallel then summarized;
• call tools first, then apply deterministic filtering, sorting, merging, or counting to the results;
• many MCP tools, where the model must search and call them on demand;
• combine classifier or image models with tool results into one script flow.

If a task needs complex approvals, atomic cross-tool commits, long-running scheduling, rollback, or human-confirmation state machines, a single script does not automatically provide those guarantees — a host service, the tool protocol, or a durable workflow must.

What can be derived from the source as engineering value: keep deterministic orchestration in code, hand the compressed evidence back to the model, and still preserve call validation and extension interception points via Pi's tool pipeline. The cost is one more runtime and one more debugging boundary, and script authors must handle partial failures, concurrency, and side effects. It is not "the model wrote code, so it is safe," nor "tool returns don't reach context, so there is no risk."

Source Walkthrough

1. From createCodemodeToolDefinition() in packages/coding-agent/src/extensions/codemode/tool.ts, find the tool description, the prepareLoadout rewrite, and the lazy-load entry; read selectCatalog() to understand the rotation that picks the catalog under a budget.
2. In execute.ts, find ctx.executeTool() to confirm nested calls run through Pi's tool pipeline; follow toScriptValue() to see how structured results, text results, and errors reach the script, and why an MCP isError does not reject.
3. Read NESTED_CALL_LIMITS and NestedCallRecorder in core/nested-tool-calls.ts; compare against the "not sent to the model" comment on nestedCalls in packages/ai/src/types.ts.
4. Compare the order in executeCodemode() of reading the branch store, running the script, and appending the entry only on success; predict the store outcome for three cases: a failing script, store(key, undefined), and no session context.
5. In packages/codemode/src/runtime/host.ts and the worker, check the QuickJS Worker lifecycle, default timeout, memory limit, and interrupt flag; then contrast Pi's timeoutMs ?? Infinity in execute.ts.
6. Check models.generateImages()'s credential source, concurrency cap, and usage accounting; find the "generated but not shown images" notice branch.
7. Design a scenario that calls read-only tools, then a write tool, and fails mid-script; record separately which results are kept and which side effects are not undone.