Part III Chapter 5: Letting Memory Continue — Session

Chapter guide: You will write the JSONL parsing in src/session.ts and the trimContext character-budget trimming in src/context.ts. This step has to separate two states: the full record on disk is auditable, while the trimmed version sent to the model is budget-bound; a character budget is only an approximation — it is not a token budget.

The Agent could do real work now, but when Xiao A closed the program and reopened it, he found it remembered nothing. He asked Lao Z: "Didn't we agree it 'can continue'?"

"It can, but first get one thing straight: this is an optional capability already implemented at the library boundary," Lao Z said. "runAgent() loads and appends the JSONL only when it receives sessionPath, and calls trimContext() before a model request only when it receives contextMaxChars. Neither the current main.ts nor runCli() passes these two options, so npm start and the existing CLI do not continue, persist, or trim sessions."

Xiao A followed up: "Then does that count as 'already delivered'?"

"Writing 'the function supports it' as 'the CLI has delivered it' confuses the assembly facts," Lao Z said. "In this chapter we first get the library boundary right; assembly is next chapter's job."

The Discriminated Read of JSONL

Xiao A started with the most direct question: "So how do we read and write the session file?"

"src/session.ts appends messages as JSONL, and loadSession() parses line by line as JSON and validates each line with a role discriminator," Lao Z said. He opened the code:

if (item.role === "user") return { role: "user", content: item.content };
if (item.role === "assistant" && calls(item.toolCalls)) return { ... };
if (item.role === "tool" && typeof item.toolCallId === "string") return { ... };
return undefined;

"Parsing first requires an object, a role, and a string content, then requires each role's own fields. A valid user needs only content; an assistant must have toolCalls whose every item carries id/name/input; a tool must have an associated ID and a boolean error flag," Lao Z said. "loadSession() turns undefined into an 'invalid at line N' exception; corrupt lines are never skipped."

Xiao A spotted another detail: "Why does parseSessionMessage return Message | undefined instead of throwing directly?"

"Separate 'what this line is' from 'what to do when this line is broken'," Lao Z said. "parseSessionMessage answers only the discriminative question: a valid line returns a Message, an invalid one returns undefined; the responsibility to throw lives in loadSession's map, which raises invalid session record at line N when it gets undefined. The discriminator stays a pure function and is easy to unit-test; the line-number logic stays in the IO layer, so the error can carry its position."

"Does the order of the discrimination matter?" Xiao A pressed.

"It does. The real code's order is: check the shared preconditions first (object, role is a string, content is a string), then dispatch by role. If the role itself is invalid — say "nope" — none of the three branches matches and it falls through to the final return undefined. This is the same as a switch without a default: with no fallback, unrecognized roles get silently skipped." Lao Z laid out a discrimination table:

Condition	Verdict
Not an object, or role is not a string, or content is not a string	undefined straight away (shared preconditions fail)
role === "user" and the preconditions hold	valid, returns a user message
role === "assistant" and calls(toolCalls)	valid, returns an assistant message
role === "tool" with toolCallId/isError of the right type	valid, returns a tool message
Everything else (including unknown role, missing branch fields)	undefined

"The rejects malformed session records test in agent.test.ts feeds all four categories: a broken role, an assistant missing toolCalls, a tool missing toolCallId, plus a round trip of one valid user message — every discrimination branch has a matching test."

Xiao A stared at the calls() type guard for a long while: "Why do the assistant's toolCalls need their own validation?"

"Because it's the only nested structure," Lao Z said. "user and tool fields are all scalars; typeof suffices. But toolCalls is an array, and each item must further be an object with id/name/input; one level of typeof can't handle it — you have to check recursively. Look at the real calls():"

// examples/mini-agent/src/session.ts, excerpt
function calls(value: unknown): value is ToolCall[] {
  return (
    Array.isArray(value) &&
    value.every((call) => {
      const item = record(call);
      return (
        item !== undefined &&
        typeof item.id === "string" &&
        typeof item.name === "string" &&
        "input" in item
      );
    })
  );
}

"Notice the last line is "input" in item, not typeof item.input === "object", because input can be any JSON (including null); it's enough that the field exists. That is what discriminated validation means: each role validates exactly the fields it truly needs — no more, no less."

"Then why does each item inside calls first go through record()?"

"Because elements of an array arrive as unknown, and you can't write call.id directly," Lao Z said. "record(), this helper, first confirms 'it is a non-null object, not an array' before the code dares to access fields as if it were a record. This record in session.ts, toolInput in model-client.ts, and asObject in tools.ts are the same technique: untrusted JSON comes in, structural narrowing first, field assertions after. This isn't duplicated code — every layer's boundary needs its own narrowing."

Validation choice	Why it	What was given up
"input" in item	input may be any JSON, including null	typeof === "object" (would wrongly reject null)
typeof id/name === "string"	id and name must be strings by semantics	a loose truthy check
Each item through record() first	elements are unknown and need narrowing	direct indexing (would crash)
value.every(...)	every item of the array must pass	checking only the first (later bad items slip through)

Xiao A guessed along the table: "Then id must be unique and name must match a real tool — does calls check those too?"

"No. calls validates structure, not semantics: it never checks whether an id duplicates another message's, or whether name really has a matching tool. Semantic correctness is backstopped at dispatch time in runAgent, where toolsByName.get(call.name) looks the name up; when it finds nothing, it records unknown tool: ${call.name}. The parsing layer only guarantees 'recoverable'; the execution layer is the one that guarantees 'runnable'."

The Normal Path of Writing and Recovery

export async function appendSession(path: string, message: Message): Promise<void> {
  await appendFile(path, `${JSON.stringify(message)}\n`, "utf8");
}

"One message becomes one line of JSON and gets appended; this avoids rewriting the whole history every round. On the next launch, loadSession() reads the file, splits on newlines, skips empty lines, runs JSON.parse line by line, and then applies the discriminated validation," Lao Z said. He added a warning: "a successful append does not mean the writes stay transactional when multiple processes write at once; the current implementation has no locks and no fsync policy."

The Error Path of Corrupted Records

"So what happens when the file gets corrupted?"

"The error path has two layers. In the first layer, the JSON can't be parsed at all and it throws invalid session JSON at line N; in the second layer, the JSON parses but the role doesn't exist, the assistant lacks toolCalls, or the tool lacks a call ID or the error flag, and it throws invalid session record at line N." Lao Z turned the two failure shapes into a table:

Line content	Discrimination result	load behaviour
{"role":"user","content":"x"}	valid user	returns the message
assistant without toolCalls	incomplete	error with line number
tool without toolCallId	cannot be associated	error with line number
truncated JSON	cannot be parsed	JSON error with line number

"The tests do cover three kinds of illegal role/fields plus one valid user round trip; they do not cover a full disk, permission denial, or concurrent writes."

"So what does the whole loadSession chain look like?" Xiao A asked.

Lao Z drew a diagram:

loadSession(path)
   │
   ├─ readFile throws ENOENT ────────────────────► return [] (first launch, empty session)
   ├─ readFile throws other (permission, IO) ────► rethrow unchanged (not swallowed)
   │
   ▼
   text.split("\n").filter(Boolean)
   │
   ▼
   line-by-line map:
      │
      ├─ JSON.parse throws ────────────────────────► throw `invalid session JSON at line N`
      ├─ parseSessionMessage returns undefined ────► throw `invalid session record at line N`
      └─ valid ────────────────────────────────────► return Message

"Note that ENOENT is the only swallowed error, and it's swallowed deliberately: a missing file means 'no session yet', not 'the session is corrupted'. Permission denials and disk errors propagate unchanged — they aren't normal, recoverable cases. Corrupt lines are always refused recovery; bad data never sneaks into memory quietly. That is the key difference between a session and the Skill directory (walked through in Part III Chapter 10): a session is one chain, so one bad line scrambles everything after it; a directory is a set of sibling files, and one broken file should not drag the others down."

Context Units, Not Message Slices

"Then what about trimming? What if the context doesn't fit?"

"trimContext() is a character-approximate budget, not a token budget. It keeps from the newest message backwards, and treats an assistant carrying multiple tool calls together with its corresponding tool results as one coherent unit," Lao Z said. He typed out a real skeleton:

// examples/mini-agent/src/context.ts, excerpt
const units: Message[][] = [];
for (let index = messages.length - 1; index >= 0;) {
  const message = messages[index];
  if (message.role === "tool") {
    // scan backwards for the assistant that declared this toolCallId
    let assistantIndex = -1;
    for (let i = index - 1; i >= 0; i--) {
      const candidate = messages[i];
      if (
        candidate.role === "assistant" &&
        candidate.toolCalls.some((call) => call.id === message.toolCallId)
      ) {
        assistantIndex = i;
        break;
      }
    }
    if (assistantIndex >= 0) {
      units.unshift(messages.slice(assistantIndex, index + 1));
      index = assistantIndex - 1;
      continue;
    }
  }
  units.unshift([message]);
  index--;
}

"When the scan from the tail hits a tool result, the function walks backwards to find the assistant that declared that call ID, then takes the stretch from one to the other as a single unit. This way no half is left behind — no 'result without a request', no 'request without a result'. If the budget is smaller than the smallest unit, it raises an error rather than returning an empty array."

The Unit Construction, Step by Step

"Can you give a concrete example?"

"Suppose the history ends with assistant (calling a and b), tool a, tool b. The backward scan meets tool b first, walks back to the assistant that declared b; the implementation takes the slice from that assistant down to the current tool as one unit. Then it processes the remaining messages," Lao Z said. "The intuition here is 'treat one tool round trip as one receipt', but the precise terms are message correlation IDs and coherent trim units."

"What does each step look like?" Xiao A asked.

Lao Z drew the backward scan:

history:  [user] [assistant{calls:a,b}] [tool a] [tool b]
                                                 ↑index=3
backward scan meets tool b: walk back for the assistant declaring id=b
   ─► assistant found at index=1; slice(1, 4) becomes one unit
        units = [ [assistant, tool a, tool b] ], index jumps to 0
next scan meets user: no association, the single message becomes a unit
        units = [ [user], [assistant, tool a, tool b] ]
reverse, then pack from the newest end: pack [assistant, tool a, tool b] first, then [user]

"Notice the slice runs all the way from the assistant down to the current tool, so tool a, sandwiched in between, is swept into the unit as well. What travels together is 'a whole round trip', not 'exactly two messages'. If a round trip happens to call only one tool, the unit is exactly two messages, assistant plus tool; with two calls, the unit has three. The size of a unit is decided by the call structure, not by the budget."

"What's good about doing it this way?"

"The model never sees a tool result without knowing who requested it; the cost is that one unit can be large, so the budget error triggers earlier than per-message trimming would. The current tests cover keeping one unit with two tool calls and two results, plus the budget-too-small error; they do not cover complex interleaved calls or repeated IDs across rounds."

Trim granularity	Advantage	Cost
Per message	makes the most of the remaining budget	can leave a half like 'a result with no request'
Coherent unit	guarantees a round trip stays complete	large units hit the ceiling and the error sooner
Summary / compaction	keeps meaning and saves budget	needs an extra model call plus a distortion policy; not implemented

"When the budget is too small to hold even the smallest unit, the function raises an error instead of quietly returning nothing, because an empty context means asking the model to answer a question with no preceding text at all."

The Normal and Failure Paths of the Character Budget

"So how exactly is the character budget computed?" Xiao A asked.

"estimateContextChars() uses the character length of JSON.stringify(message) for each complete Message, then adds a separation overhead. A circular reference or an unserializable message raises an error carrying its index; this makes budget failures locatable. It is not a token budget: Chinese characters, ASCII, the provider's hidden overhead, and server-side serialization are none of them reflected precisely," Lao Z said. "The normal path starts from the newest unit and keeps adding until the next unit would exceed the limit; if at least one unit is already kept, it stops and returns; if even the first unit exceeds the limit, it throws that the smallest coherent unit cannot fit."

Lao Z drew out the budget-packing algorithm:

units (newest to oldest)
   │
   ▼
kept = []
for each unit (newest to oldest):
   │
   ├─ estimateContextChars(unit ∪ kept) > maxChars?
   │     ├─ kept is empty ──────► throw "context budget cannot contain the smallest consistent message unit"
   │     └─ kept is not empty ──► break (stop packing backwards)
   └─ within budget ────────────► kept.unshift(...unit), continue to the next unit
   │
   ▼
kept is still empty ────────────► throw "context budget cannot be empty"
return kept

"Both throws have tests: trimContext(messages, 100) triggers the smallest-unit error (context.test.ts), and so does trimContext(messages, 1) (in agent.test.ts's keeps a multi-tool assistant unit with its results). A budget failure must be locatable and reproducible; it must never be 'no idea why a stretch of history went missing'."

Xiao A jumped in with half an answer: "JSON.stringify throws by itself on a circular reference — why does estimateContextChars still raise its own 'unserializable' error?"

"The error stringify throws carries no message index," Lao Z said. "If the circular object is buried in the fifth message, all you get is Converting circular structure to JSON, with no way to tell which message it is. Wrap it once, add message ${index}, and the locating cost drops from 'inspect everything' to 'glance at an index'. Besides, stringify returns undefined rather than throwing for values like undefined or functions, so the code checks serialized === undefined separately. There are two failure paths — one throws, one comes back empty — and both must be blocked."

"Then why not just auto-summarize? Isn't a summary cheaper?"

"A summary needs another model call, extra cost, and a distortion policy," Lao Z said. "The current deliverable chooses to error or trim explicitly, rather than pretending to have semantic compaction."

The Responsibility Boundary between Session and Context

"So who exactly owns each of these states?"

"First look at which path a piece of state takes once it enters runAgent." Lao Z drew the whole path:

runAgent(options)
   │
   ├─ options.sessionPath present? ────► loadSession → messages (full record on disk)
   │        absent? ───────────────────► messages = [] (start with an empty session)
   │
   ▼
   before each round of client.stream:
   │
   ├─ options.contextMaxChars present? ────► trimContext(messages, maxChars) → sent to the model
   │        absent? ───────────────────────► messages sent as-is (no trimming)
   │
   ▼
   each round's record():
       messages.push (in-memory state)
       appendSession when sessionPath is present (on-disk state)

"Note that these two paths know nothing about each other: trimContext only reads the messages array and has no idea it came from disk; appendSession only writes JSONL and has no idea the model received a trimmed version. The full record and the trimmed input are two states, each running its own pipe, meeting briefly only inside the in-memory messages array."

Lao Z then listed the four states and their owners:

State	Where it lives	Failure semantics	Not covered today
Full events	JSONL	corrupt line refused recovery	encryption, locks, deletion
This round's input	trimContext return value	budget-too-small error	token counting, summaries
Tool correlation	assistant/tool unit	kept as a group	complex concurrent correlations
Long-term preferences	not implemented	not applicable	provenance, expiry, privacy

Design Costs and the Walkthrough

"JSONL is easy to read but has no query index; character trimming is deterministic but understands nothing about the task; keeping the complete file has audit value, and also brings the risk of retaining sensitive data," Lao Z said. "During the Source Walkthrough, open record(), appendSession(), loadSession(), parseSessionMessage(), trimContext() in turn, and hand-trace the branches once with a valid three-line JSONL and once with a JSONL whose second line is corrupted."

"Among these files, which spot is easiest to walk wrong?"

"Two," Lao Z said. "First, the shared preconditions in parseSessionMessage: the typeof checks on role and content sit at the very front and are shared by all three branches; if you move them inside one branch, the other two branches have to repeat the checks, and it's easy to miss one. Second, the double loop in trimContext: the outer loop runs backwards from the tail, the inner loop walks forward to find the assistant; the group forms only when assistantIndex >= 0, otherwise the tool falls out as a lone unit. A lone tool unit is also legal: when a tool result finds no assistant that declared it, it can only stay on its own. That is the trimming layer's leniency toward a dirty history — but it never fabricates an assistant to fill the gap."

Design trade-off	Chosen	Alternative given up
Record format	JSONL append	full rewrite of one file (O(n) write amplification per round)
Recovery policy	throw on corrupt lines	skip corrupt lines (loses evidence, silently misaligns)
Trim metric	character approximation	token counting (would introduce a tokenizer dependency)
Context compaction	trim / error	auto summary (one extra model call)
Concurrency safety	no locks	file locks / fsync (complexity mismatched with the current scenario)

Normal and Failure Sequences

Xiao A wanted to see it running: "If we call runAgent directly with the two optional parameters, what do the normal and the failure sequences look like?"

"With both optional parameters passed in, the normal sequence is: append writes one JSON line → the next load validates line by line → the Agent restores from the full record → trim selects the units that fit. The failures: broken JSON, an illegal role, or a missing field reports a line number; a budget too small reports that the smallest coherent unit cannot fit," Lao Z said. "Without summaries, once an old conversation is trimmed it is unrecoverable from the model's input, though the disk keeps the original text. The existing CLI never enters this sequence."

Mechanism	Currently guaranteed	Not guaranteed
JSONL	append and discriminated validation	encryption, concurrent write locks
trim	a deterministic character ceiling	token exactness, meaning preservation
tool unit	calls and results travel as a group	cross-session business consistency

Summary

The session layer makes one key distinction: the full record on disk and the trimmed context sent to the model are two different states. The former is one complete message per line — auditable, replayable; the latter is bound by a budget and must be trimmed. Conflate the two, and you end up either letting the context grow without limit or letting the disk record lose key information. The two states run their own pipes and meet only briefly inside runAgent's in-memory messages array: trimContext doesn't know the messages came from disk, appendSession doesn't know the model received a trimmed version, and the responsibility boundary is maintained precisely by this mutual ignorance.

The JSONL parsing is strict, and strict layer by layer. parseSessionMessage stays a pure function: it checks the shared preconditions first, dispatches by role, and returns a Message for a valid line or undefined for an invalid one; the line-number responsibility stays in loadSession, which raises invalid session record at line N only when it gets undefined. An illegal role, a missing field, or truncated JSON all report by line number, so bad data never sneaks in. The structural narrowing of record() and calls() is the same technique as toolInput in model-client.ts: untrusted JSON comes in, narrow the structure first, then assert the fields. The parsing layer guarantees only "recoverable", not "runnable": whether a tool name exists is answered by the execution layer's toolsByName at dispatch time.

The trimming itself, trimContext, is a character-approximate budget that keeps from the tail backwards, and treats an assistant with multiple tool calls and its corresponding results as one indivisible unit, so the model never sees half a conversation — a result without its request, or a request without its result. But a character budget is not a token budget: estimateContextChars approximates with the length of JSON.stringify, and Chinese characters, ASCII punctuation, and the provider's hidden overhead are none of them reflected precisely. A failed stringification must raise an error carrying the message's index; a budget smaller than the smallest coherent unit must raise an error instead of returning an empty array, because an empty array leaves the model facing an isolated request with no preceding text at all.

The current deliverable has no auto-summary or compaction; it chooses to error or trim explicitly rather than pretend to have semantic compression. Both roads lead to the same judgment: no tokenizer dependency, no extra model call — trade a deterministic character approximation for implementation simplicity, and state honestly that it is imprecise. The CLI has not yet wired up sessionPath either, so session continuation currently takes effect only when runAgent is called explicitly; at the session layer, "the function supports it" and "the CLI has delivered it" are two different things, and assembly is next chapter's job.

Stage Acceptance

Run npm test: the existing tests call runAgent directly through a fake client, and can verify the append and recovery of an explicit sessionPath; the standalone context tests verify character estimation and coherent-unit trimming. What they verify is the library boundary, not session delivery through npm start / runCli(); and the current tests do not prove that the entry point passes contextMaxChars either.