Chapter 10: What to Remember, What to Forget — Memory
Xiao A's agent could loop now, but after running a while it showed two ailments: mid-conversation the model "forgot" the original task, and once the program was closed and reopened the whole prior process was gone. He brought both to Lao Z, a little deflated.
"Is it just a bad memory?" Xiao A asked.
"Not a bad memory — there's simply no one recording on its behalf," Lao Z said. "The loop lets a task advance across many steps, and it also lets the history grow without end. What you've found are actually two different problems: the model's context fills up, and after the process exits it can't resume work."
"Then if I just save all the history, wouldn't that fix everything?"
"Don't rush to store," Lao Z said. "What to save, how long to save it, who may access it, when to delete it — each of these is a design decision. You think you're solving 'can't remember,' but what you really face is 'remembers too much': the context fills, storage bloats, and sensitive information lingers. This chapter won't set your storage budget for you. It first separates the responsibilities and costs of four layers — context, session persistence, compaction, and long-term memory — with special attention to the privacy boundary."
Align the names first
| Term | English / abbreviation | Working definition in this book | What it is not |
|---|---|---|---|
| context | context | the set of material visible to the model in one current request | the full history in a database |
| session | session | the application's record of organizing messages, calls, and state across many turns | memory the model holds naturally |
| short-term / long-term memory | short-term / long-term memory | current task material / information confirmed for reuse across sessions | every history kept forever |
| truncation / compaction | truncation / compaction | dropping some material / replacing old material with a summary | lossless archiving |
| retrieval | retrieval | fetching material from controlled storage by query | automatically trusting whatever is matched |
| retention | retention | the rules for keeping, accessing, updating, and deleting data | "might be useful later" |
10.1 The model is stateless; the application must manage state
"Can't the model remember things itself?" Xiao A asked.
"No. Every model request only sees the material the application brings in," Lao Z said. "An application usually keeps a full session record, then selects part of it to place in the current turn's context. Don't conflate the 'context window' with 'persistent storage': the former is the finite budget visible to one request, the latter is the mechanism that keeps records across processes."
Metaphor: the consultant and the assistant
The model is a consultant who must have the material placed on their desk fresh for every question, and you are the assistant in charge of the filing. The consultant can't remember — the assistant is the one doing the remembering.
"But vendors have advertised 'long context.' Once the window grows big, won't the problem disappear?"
"A longer window only raises the budget ceiling for a single request; it does not change the premise that 'each request only sees the material the application brings in,'" Lao Z said. "However large the window, it is still a finite budget billed by token and trimmed by capacity, and putting something in doesn't mean the model attends to it equally: in a long context, key constraints near the front often get diluted. A long window can't replace your deciding what to put in and what to leave out, let alone replace the recovery mechanism after a process exits."
"Then I'll just fill the window and hand over all the material — that can't go wrong, right?" Xiao A came up with an easy way out.
"Not necessarily," Lao Z said. "The fuller you pack it, the higher the cost and latency per request, and the greater the risk that important constraints get buried. The window is a budget, not insurance. What determines reliability is how the application selects, organizes, and labels that material — not how high a pile you stack."
"Does how the material is arranged affect what the model pays attention to? Is putting it at the front that different from putting it in the middle?" Xiao A pressed.
"It does — this is the most practical trap in long context," Lao Z said. "Instructions near the beginning and the end are usually followed better, while long passages buried in the middle are often skimmed as background. The same constraint placed in a system instruction versus buried in the five-hundredth history message has very different force. How much difference depends on the model and the task, so there's no universal number — but one useful design default: put the most critical rules at the front, dynamic content at the back, and reference material in the middle. That's no theorem, just a common engineering failure mode. Don't expect the model to treat every stretch of tokens in the window equally. Position isn't the whole memory strategy either, but it is a knob you can turn at zero cost: rearranging the order of material is far cheaper than widening the window, and more predictable than repeatedly rewriting the prompt."
10.2 Four kinds of memory, four responsibilities
"So 'memory' has to be layered? How many layers, engineering-wise?"
"In practice it's often split into four kinds, each with a different responsibility," Lao Z said, and laid out a table:
| Kind | What it stores | Main use | Main risk |
|---|---|---|---|
| working context | the messages and tool results the current request needs | let the model finish this turn's task | overrun, cost, prompt injection |
| session record | resumable events and messages | continuation, audit, debugging | sensitive data lingering |
| compaction summary | a lossy digest of old records | free up context budget | losing constraints or hardening errors |
| long-term memory | confirmed stable preferences or project facts | cross-session personalization | staleness, unauthorized storage, contamination |
"Can these four share one store?"
"They are a design taxonomy split by responsibility. A real system may keep them on the same medium, but it must not give them the same retention period and access rights."
"Same medium, different policy — where exactly does the difference lie?"
"In three places: retention period, access scope, and deletion entry," Lao Z said. "Working context should be cleared when the request ends; the session record needs to be kept a while for audit; a compaction summary's expiry should be tied to the original records it summarizes; and long-term memory must give the user a way to view and delete. If you shove them all into one table behind one delete script, then either sessions get cleared too early and you can't debug, or summaries and preferences linger indefinitely and widen the privacy surface."
"Isn't the line between long-term memory and working context easy to blur?" Xiao A asked.
"It is — a trap people step in constantly," Lao Z said. "If the model casually says in one turn, 'the user seems to prefer brevity,' and that gets written straight into long-term memory, you've hardened a judgment of unknown origin that the user never consented to. The real difference between the two layers is the bar for review. Information in working context is temporary: if it's wrong, the next turn overwrites it, so a model inference there has limited impact. Information in long-term memory is reused across sessions: once one wrong entry hardens, every later session acts carrying it. So that bar must rise with the layer's durability — the longer-term the information, the more it needs confirmation before being written. The precondition for entering long-term memory is a clear source, a legitimate use, and the user's permission; unconfirmed model inference stays at most in working context and dissolves when the turn ends. This isn't conservatism — it's keeping change-risk where its blast radius is smallest."
"Can the four kinds be lined up side by side on time span, write bar, access scope, and deletion entry?"
Lao Z merged the four dimensions into one table:
| Dimension | Working context | Session record | Compaction summary | Long-term memory |
|---|---|---|---|---|
| Time span | current request | during the session | follows the original records | across sessions |
| Write bar | low, organized per request | recorded automatically | generated as a derivative | requires confirmation |
| Access scope | visible to the model | application, audit | for continuation | user-controllable |
| Deletion entry | cleared when the request ends | deleted with the session | expires with the source | must be provided |
"This table isn't for you to build four stores by the book," Lao Z added. "It's for you to answer, column by column, when you design. The cell you can't answer is where the trouble will show up later."
10.3 What to do when the context fills up
"What do you do when the context is nearly full?" Xiao A asked.
"The usual strategies are truncation, sliding window, summarization, and on-demand retrieval," Lao Z said. "Truncation is simple but may drop the task goal; summarization preserves more meaning but carries extra cost and a risk of distortion; on-demand retrieval suits large bodies of material but needs a reliable index, permission filtering, and source attribution. There is no strategy that is optimal for every session."
"Is there an order among these — which to do first?"
"A robust flow first pins the items that can't be lost (safety rules, the user's current goal, unfinished actions, key constraints), then truncates or references tool outputs that can be re-fetched, and generates a source-tagged summary of old dialogue," Lao Z said. "A summary must be labeled as a summary; it can't masquerade as the original record."
"Between truncation and summarization, which is safer?"
"Each has its own failure mode," Lao Z said, and listed a comparison:
| Strategy | Budget freed | Main cost | Typical failure |
|---|---|---|---|
| Truncation | direct, deterministic | drops the truncated content | the first turn's goal or constraints get cut away |
| Summarization | usually larger, more token-saving | the summary's own cost and distortion | key constraints get generalized away or rewritten |
"Truncation fails by 'explicit loss' — you can see what's missing. Summarization fails by 'silent change' — the content looks like it's still there, but its meaning has been rewritten or diluted," Lao Z said. "So a summary may only be marked as a derivative, and must be traceable back to the source when needed. Summarizing a constraint that says 'do not delete files' into 'discussed file operations' is a textbook distortion."
"Draw those two failures so I can compare them?"
Lao Z drew two lines:
Truncation: [A][B][C][D][E] → remove the tail → [A][B]
(constraint here) the constraint is gone — but the "gone" is visible
Summary: [A][B][C][D][E] → replace with a summary → [sum(A..E)]
(constraint here) the constraint is still in the text — but its meaning is rewritten"After truncation you can count three blocks missing; after summarization the thing is superficially still there, only 'changed' — and the model can't tell whether it's reading the original or a paraphrase," Lao Z said. "So a summary keeps a reference pointing from the summary back to the original messages, so you can trace back when distortion shows up. Truncation's error is one of quantity; summarization's error is one of quality, and the latter is far more insidious."
"Where does the sliding window fall?"
"On the truncation side — it's a structured, directed truncation: keep the last N turns, discard everything older," Lao Z said. "Its benefit is predictable behavior and no distortion; its cost is hostility to constraints that are 'early but important.' Once the goal sits in turn 1 and you're in turn 20, the goal has been cut away. So a sliding window usually pairs with a 'pinned region': nail the items that can't be lost to the window so they don't scroll, then trim the rest by time."
"What does a pinned region look like? Draw one."
Lao Z drew a window:
Window (keep the last 8 turns)
┌─────────────────┬───────────────────────────────┐
│ Pinned region │ Rolling region │
│ goal, safety, │ turns N-7 ... N messages │
│ open actions │ (trimmed as turns roll by) │
└─────────────────┴───────────────────────────────┘"The pinned region doesn't scroll by time; it's kept by importance," Lao Z said. "Its cost is that this content holds budget continuously and never yields as time passes. So what you put in must be few and sharp: in and it's present for the long run; put the wrong thing in and it occupies the slot for the long run. The pinned region is 'nailed,' not 'cached' — when you design it, think through how long it should serve."
"And on-demand retrieval? When there's a lot of material, that's when you use it, right?" Xiao A pressed.
"It's apt only where you have a reliable index, permission filtering, and source attribution," Lao Z said. "Retrieval isn't as simple as 'store a pile, query once.' No index means a full scan — slower than just stuffing the context. No permission filtering means you've placed data the model shouldn't see right in front of it. No source attribution means a hit degrades into an unsourced excerpt. And retrieval itself can fail: no match, wrong match, or a stale version. So retrieval suits scenarios with a lot of material that needs precise fetching; forcing retrieval onto a small session is solving a cheap problem with an expensive tool."
"Put the costs of the four strategies in one table?"
Lao Z laid the four strategies into one table:
| Strategy | Implementation cost | Inference cost | Main failure | Suits |
|---|---|---|---|---|
| Truncation | low | low | loses the task goal | short history, re-fetchable |
| Sliding window | low | low | drops early constraints | long dialogue, needs predictable behavior |
| Summarization | medium | medium | distortion, constraints generalized away | long history, needs to preserve meaning |
| On-demand retrieval | high | medium | index, permission, source problems | large material, needs precise fetching |
"In this table, 'implementation cost' and 'inference cost' do move," Lao Z cautioned. "Summarization, for instance, is generated offline, so its cost is amortized across the gaps between sessions; on-demand retrieval, if it scans the whole store every time, sees its cost climb with the volume of material. No strategy is free — you either pay now (generate summaries, build indexes) or pay later (failures, distortion, collateral damage). Choosing a strategy is really choosing which cost to pay."
10.4 Session persistence and recovery
"'Close it and reopen it and still resume' — what makes that possible?"
"An append-only event log — say, one structured event per line — is easy to restore after a crash and easy to audit for tool side effects," Lao Z said. "The record should at least contain the message, the tool-call ID, the execution state, timing information, and a version; a failed write or a half-written record is an error path you must rehearse."
"When recovering, do you just shove the whole history back and call it done?"
"No — that's the 'keep everything' thinking again," Lao Z said. "On recovery you rebuild state, identify unfinished calls, confirm whether a retry is allowed, then reselect material according to the current context budget. For unfinished calls with external side effects, the default should require proof of idempotence or human confirmation."
"How does a half-written record come about, and how do you actually handle one?"
"The process gets killed mid-write to the event stream, and that last line is a half record — possibly truncated JSON, possibly a call record missing a state field," Lao Z said. "The recovery program can't assume the previous entry is complete: either tolerate the tail at line boundaries and discard the incomplete line, or give each record a checksum and an end marker and treat a failed check as unfinished. More important, whether the tool call behind a half record has already executed externally can't be judged from the record alone — that's exactly where a 'pending confirmation' state comes from."
"So a tool call dispatched right before the crash — after restart, does it count as done or not?"
"The record can't answer; only a verifiable state in the outside world can," Lao Z said. "For a file write, check whether the file really exists and its content matches; for an email, check whether the recipient side has dedup or an idempotence key. The persistence record captures 'what the application meant to do,' not 'what happened in the outside world.' The gap between the two gets filled by re-checking and confirming — never by blindly replaying or blindly skipping."
"Are 'replay' and 'recovery' the same thing?"
"No. Replay mechanically executes every event again; recovery rebuilds state without repeating actions," Lao Z said. "An event stream carries both 'state changes' and 'side-effect actions.' Replaying state changes is safe — the state in memory can be derived back to any historical point. Replaying side-effect actions is dangerous — file writes and messages have already happened in the outside world, so running them again means doing them twice. So the recovery program must tell the two kinds of event apart: replay the former, and only mark the latter, never auto-replay, until confirmation or proof of idempotence."
"Can an event say which kind it is?"
"The event type alone can't fully decide it; you verify against external state," Lao Z said. "The same 'write file' event is a side-effect action in the recovery program's eyes, yet its result field 'file written' reflects a state change. So in practice it's 'confirm side-effect actions before replaying; treat a result as state only once confirmed' — judged by a verifiable state in the outside world, not by guessing from the event's name."
"One more thing," Lao Z added, "where you crashed determines how deep the check must go. Crash before the action, and the event may never have happened externally; crash after the action but before the record, and the action may have completed without leaving a result. Those two cases call for very different recovery moves. The recovery program has to branch on 'before-action / after-action / before-record / after-record' rather than run one bit of logic over every record."
"How many categories can pending actions fall into?"
Lao Z laid out the common ones:
| Pending action | Safe recovery move | Requires human confirmation | Never auto-retry |
|---|---|---|---|
| read file, query status | re-read | none | none |
| write file | first check whether the file already exists | if it exists, verify the content | none |
| send message / email | none | verify whether it already went out | judge after confirming |
| charge, transfer | none | verify the external billing state | no replay before confirming |
"This table is grouped by action type as an illustration, not a list to copy verbatim," Lao Z said. "Each action's 'verifiable state' is different: for a file it's existence and content, for a message it's whether the other side received it, for a bill it's the external transaction log. When you design the recovery program, first define the verifiable state for each kind of side-effect action, then decide how to split the three tiers — safe recovery / human confirmation / never retry."
10.5 Long-term memory must be correctable and deletable
"Isn't more 'long-term memory' always better? Store all the history, retrieve later — can't that work?"
"No — that's a retention-policy question, not a purely technical choice," Lao Z said. "Keeping 'user preferences' or 'project conventions' long-term looks convenient, but it widens the privacy and mislead surface, and 'might be useful later' is usually not a sufficient reason. Data minimization, encryption, access control, retention periods, and export-and-delete capability should be settled at design time; every record should have a source, an update time, a visibility scope, and a deletion mechanism. Stale preferences, expired project facts, and personality labels the model inferred on its own should none of them be made permanent."
"So how do you judge whether one piece of information qualifies to go in?"
"Ask three questions: who is the source, what is the use, and does the user know," Lao Z said. "Consider making it long-term only when all three hold — the source is something the user stated explicitly, the use is reuse in the next session, and the user has a place to view and delete. Only information with a clear source, a legitimate use, and the user's permission should enter long-term memory. If even one is fuzzy — the source is the model's own inference, the use is vague, or the user never knew the record existed — it stays in working context and dissolves when the task ends."
"For the deletion entry, is a button enough?"
"Not enough. Deletion comes in three granularities: delete a single entry, bulk-delete by source, and export everything then wipe," Lao Z said. "There's also a boundary question to answer: when you delete a long-term memory, what happens to the same information already compacted into a summary? If the summary isn't updated accordingly, the deletion is only skin-deep. So updates and deletions of long-term memory must propagate along the source chain, not just edit one table."
"Deletion itself has subtleties too?"
"It does — logical deletion versus physical deletion," Lao Z said. "Logical deletion sets a 'deleted' flag; the data is still on disk. Its benefit is recovering an accidental delete; its harm is that sensitive data still lingers in storage. Physical deletion wipes the data; its cost is that it's thorough and irreversible. For a user-initiated deletion, the expectation is usually physical deletion; for automatic cleanup inside the system, logical deletion plus periodic physical purging is safer. Deletion is product semantics, not one SQL statement — the user thinks it's deleted while the data's still there is worse than not deleting."
"After deleting, could a stale copy still sit in a cache?"
"That's the hardest place deletion covers," Lao Z said. "When a long-term memory is updated or deleted, the current context is reorganized by the application and generally won't carry the deleted content. But if the application has cached past messages, an old snapshot may still hold the deleted information. So when you delete a long-term memory, mark the context cache, the summaries, and the retrieval index as invalid together — don't clean only the main table. Miss any one layer and the deletion is only half done."
10.6 Recovery is not resending the history
"So what does a correct recovery flow look like?"
"After persistence, recovery should first replay events to rebuild the application's state, then decide what content enters the next context," Lao Z said. "The message history is the evidence source; a summary is a derivative; both should carry a version, a source, and a retention policy. If a summary has overwritten the original messages, it can no longer prove whether it dropped a key constraint."
Lao Z drew the flow:
append events → crash → read valid records → rebuild call state
↓
choose context / flag pending actions"Do summaries and long-term memories go stale?"
"They do — so both need an expiry mechanism," Lao Z said. "You can give project facts an update time and a source, and give user preferences places to view, edit, and delete; retrieval hits should be sourced too. More storage improves recovery, but it increases privacy exposure, migration cost, and the risk of entrenching errors."
"Between a retrieval hit and a summary, which counts as evidence?"
"Neither — both are derivatives," Lao Z said. "The original evidence is the message history itself. A retrieval hit is a fragment fetched from history or an external store; a summary is a distillation of history. Both can go stale, exceed permissions, or get contaminated. The model should not take a retrieval hit as a verified fact and trust it outright; the application should label source, time, and access scope when feeding it into context, so the model treats them as 'citations to verify,' not as 'settled conclusions.'"
"Then is the raw history guaranteed reliable?"
"The raw history is just the layer that was 'recorded earliest'; it may contain wrong calls, contaminated input, or a half record," Lao Z said. "But compared to summaries and retrieval results, it's closest to the truth and least rewritten, so debugging and audit trace back to this layer. It's a matter of priority: derivatives serve smooth continuation first; when the evidence chain has a problem, you trace back to the raw layer."
"Is there a standard for what to trace back and what not to?"
"The standard is 'which step rewrote or generalized this sentence,'" Lao Z said. "If the model cited a retrieval hit, trace back to the original text. If a summary says 'discussed file operations,' trace back to which turns it generalized. If a user preference got hardened into long-term memory, trace back to its original source. Anything that has passed through excerpting, summarizing, or inference carries a derivation chain, and the raw layer is the end of that chain — tracing back is walking that chain in reverse. A sentence that was never rewritten, say a directive the user stated directly, needs no trace-back."
"How many levels of the derivation chain are enough to keep?"
"At least the 'raw to first-layer derivative' level," Lao Z said. "A summary marks which raw messages it generalized, a retrieval hit marks which file it came from, a long-term memory marks which session it came from. Keeping that one level isn't costly, but it lets debugging jump straight to the raw layer. Keep it too shallow and every trace-back sifts the whole store; keep it too deep and every layer has one more reference relation to maintain. Keep one level first; add more when you need them."
Illustration: the process crashes after a tool has written a file but before the result is recorded. The recovery program must not assume the write succeeded or failed; it should check the verifiable file state, match the call ID, and turn any external action it can't judge into pending confirmation — rather than replaying it automatically.
10.7 Recovery must separate messages, the world, and permissions
"Can you draw this chain?"
"Sure," Lao Z said, and drew one:
append session event → process interrupted → read valid records → rebuild internal state
→ select context / retrieve material
→ flag unconfirmed external actions"What about the counterexamples?"
"The happy path: the session records the call ID, state, and time; after recovery it loads only the current goal, key constraints, and verifiable evidence into context," Lao Z said. "Counterexample one: a compaction summary omits the 'do not delete files' constraint — a summary may only be marked as a derivative, traceable to its source when needed, never passed off as the original. Counterexample two: the process crashes after a tool writes a file but before the result is persisted — persistence won't by itself say whether the external write succeeded or failed, so you check the verifiable state and require proof of idempotence or human confirmation."
"A retrieval hit is at least trustworthy, right?"
"Not necessarily — those can go stale, exceed permissions, or get contaminated too, so keep the source, update time, access scope, and deletion entry," Lao Z said. "Compaction is not lossless, and persistence is not a transactional recovery of external side effects — those are the two hard boundaries of this chapter."
"What does it look like when those two boundaries clash?"
"The most common clash is 'to recover completely, keep everything,'" Lao Z said. "The more you keep, the stronger the continuation and audit capability — but privacy exposure, migration cost, and the risk of entrenching errors rise in step. A sensitive path or token mixed into an early session, kept indefinitely under 'complete recovery,' becomes a buried landmine. So retention isn't 'store what you can store'; it's setting a retention period and a deletion entry for every layer, and clearing each when its time is up."
"Can privacy and recoverability both be satisfied?"
"You approach it by layering; no single trick gets you both," Lao Z said. "Keep the raw history short-term, for debugging; keep summaries medium-term, for continuation; keep long-term memory only for stable, user-confirmed information, and provide a deletion entry. The three layers have different retention periods and access rights, so the privacy exposure concentrates in the longest-lived layer — and that layer's volume is squeezed to the minimum. Trade volume for privacy surface: the longer-term the information, the stricter the filtering."
"What if the user demands 'delete everything'?"
"The three layers move together, not one table," Lao Z said. "Delete the raw history, and the summaries and long-term memories that reference it should be marked invalid at once — otherwise a summary still cites deleted original text. Delete a long-term memory, and clear any stale snapshot that may sit in the context cache. If one layer is deleted while another still holds it, the 'deleted' the user sees is fake. Deletion propagates along the source chain — said once in 10.5, and it holds in the recovery scenario too: recovery isn't only restoring state, it's also restoring the 'deleted' state."
"And permissions? Who's entitled to see what recovery brings back?"
"Access rights follow the source, not the fact that it was 'recovered,'" Lao Z said. "If material the recovery process reads was authorized only to the current user, another session or another agent shouldn't fetch it. Recovery itself doesn't change the access scope — it only scoops records back off disk, and every filter that should apply still applies. Permission filtering is done at the read entry of every layer, and the recovery path is the easiest one to miss: it looks like an 'internal flow,' so it's the likeliest place to be treated as needing no authentication."
"So how do the three layers rank in priority?"
Lao Z drew the ordering of the three layers:
raw message history ── the end of the trace-back for debugging and audit (closest to the truth, least rewritten)
↑ referenced by
summary ── used for continuation (a derivative; can be marked stale, can be traced to source)
↑ referenced by
long-term memory ── reused across sessions (strictest filtering, must be deletable)"Debugging traces back to the raw layer, continuation reads the summary layer, personalization uses the long-term memory layer," Lao Z said. "The farther from the raw layer, the more the information has been rewritten, and the stricter both the filtering bar and the need for labeling and deletion. This priority isn't here so you think of the raw layer only when something breaks; it's so that when you design you know clearly, for each layer: who it serves, who may change it, who may delete it."
Summary
Memory is not a capability the model ships with; it's a few layers of storage the application layer designs carefully. What people most often conflate are context and session. Context is all the material the model can see in one current request, serving the reasoning at hand; a session is the record kept across many requests, serving recovery and audit. The two are entirely different: context vanishes when the request ends, while the session stays on disk waiting for you to revisit it. Beyond those there's the summary, which trades less space for a semantic approximation — sacrificing detail so the window can hold more turns — and long-term memory, which keeps the preferences and information that stay stable across sessions. Each layer must answer the same set of questions: where does the data come from, how much can be stored, who may access it, how long is it kept, and when is it deleted.
The illusion this chapter sets out to kill is "the more you remember, the more reliable you are." More records don't automatically bring a more reliable agent; they can bring more noise, higher cost, and greater privacy risk. A system that shoves every conversation into context hits the window ceiling quickly; a system that keeps every session forever faces data residue and compliance problems. The core of memory design is trade-off: what each layer keeps and what it discards needs an explicit policy, not mindlessly remembering everything.
When the context fills up, the four roads — truncation, sliding window, summarization, and on-demand retrieval — each pay their own price. Truncation is cheap but drops the task goal, and drops it in plain sight; summarization saves tokens but may silently rewrite constraints, and rewrites them soundlessly; the sliding window is predictable but hostile to content that's "early but important"; on-demand retrieval saves context but carries the three costs of index, permission, and source. Choosing a strategy is choosing which cost to pay. Recovery is the same: persistence records "what the application meant to do," not "what happened in the outside world," so recovery must separate state changes from side-effect actions — replay the former, check the external verifiable state for the latter, then decide safe recovery, human confirmation, or never retry.
Between the layers run three standing rules. First, the derivative rule: summaries, retrieval hits, and long-term memory are all derived from the raw history, so each carries a source, a version, and an expiry mechanism, and each must be traceable back to the raw layer. Second, the deletion-propagation rule: deletion travels along the source chain — delete one layer and the summaries, caches, and indexes that reference it must go invalid too, and logical versus physical deletion must be told apart. Third, permissions follow the source: the material the recovery path reads has access rights that follow the source, not the fact that it was "recovered." Apply these three to every layer, and the memory system stops being a pile of accumulated data and becomes a storage design with boundaries, trade-offs, and the ability to clean itself up.
Hands-on Verification
For a session that contains tool calls, design an event log, and simulate a crash after the tool has executed but before the result is written. On restart, list the actions that can be recovered safely, those that require human confirmation, and those that must never be retried automatically. Then check whether the log contains paths, tokens, or user content that should not be kept long-term, and define a clearing policy for them.