Chapter 7: A Machine That Relays Automatically — Agent

Xiao A finally got the model connected. He asked it a few questions, and it answered each one well. But the moment he told it to "run all the tests in this repository and fix the failures," it replied "I can't execute commands" and stopped.

"The model is clearly smart. Why does it freeze as soon as there is real work to do?" Xiao A asked.

"Because it only generates text; it does not perform actions." Lao Z said. "To make it do work you need another machine: one that strings generation, execution, and observation into a loop. That machine is the Agent."

What this chapter answers

• The working definition of an Agent: what parts it is made of
• How a loop turns "saying" into "doing"
• Workflow and Agent are the two ends of one spectrum, not opposites
• Autonomy is not the goal itself, and using an Agent takes judgment

Not covered: the mechanics of the loop — its state machine, stop conditions, concurrency — are left to Chapter 9: Getting the Work to Actually Loop — The Agent Loop; tool calling and execution boundaries are in Chapter 6: From "Can Say" to "Can Do" — Function Calling and Chapter 8: Putting Handles on Capability — Tool.

Aligning the Names First

Term	Full English name or abbreviation	Working definition in this book	What it is not
Agent	agent	a task working unit made of model decision-making, tool execution, and loop control	a one-off question-and-answer request
Agent loop	Agent Loop	the control flow that requests the model, handles tool results, updates state, and repeats	the model automatically holding the right to execute
Autonomy	autonomy	how far the system pushes a task forward without waiting for step-by-step human instruction	unsupervised and unconstrained
Workflow	workflow	a fixed pre-orchestrated sequence of steps; the decision points live in code	the opposite of an Agent
Host	host	the program that holds state, executes tools, and decides the stop conditions	the model itself
Orchestration	orchestration	coordinating the model, tools, state, and human approvals into one task	stacking more models

7.1 The Definition: A Working Unit That Can Loop

"So what exactly counts as an Agent?" Xiao A asked.

"First, a working definition you can use: an Agent is a task working unit that can repeatedly call the model, execute tools, observe the results, and decide the next step." Lao Z said. "The key is not how strong the model is. It is whether the thing can loop. One question and answer does not make an Agent. Only looping does."

"Who has the say inside the loop?"

"The model proposes the next step. The host decides whether to execute it and when to stop. The model is the proposer of decisions; the host is the approver. That division of labor is the starting point of every reliability discussion that follows."

Xiao A pressed: "Why draw the line so sharply? Can't the host just execute whatever the model proposes?"

"Because the model is wrong sometimes." Lao Z said. "It may propose deleting a file that should not be deleted, or calling a tool outside its bounds. If every proposal executes immediately, one mistake lands as a side effect. Splitting it into two steps gives the host a chance to stop it before it lands. That is what lets an Agent take dangerous actions without crashing."

"What does the host approve against?"

"A policy written into the host ahead of time: which actions are allowed, which paths are out of bounds, which ones need approval." Lao Z said. "That policy is not in the model's head; it is in code. So it can be audited, it can be tested, and it does not drift because the model is having a bad round. The model does the thinking; the host keeps the guard."

"Can you give an example of something getting blocked?" Xiao A asked.

"Say the model proposes delete_file with the argument /tmp/old.log." Lao Z said. "The host checks the policy: files in that directory may be deleted, but only if they appear in the manifest this project generated. old.log is not in it. So the host refuses, and writes the refusal reason into the result handed back to the model. Reading that, the model switches to another path, or drops the deletion entirely. The proposal did not land, so there is still room to negotiate. If proposals executed instantly, negotiating would turn into incident response."

"Does that check by the host slow the loop down?" Xiao A asked.

"Yes, and that is a trade made on purpose." Lao Z said. "The model's proposal costs one generation. The host checks the policy, makes a decision, and keeps a record — and that happens only when an action is truly about to land. Separating thinking from approving means the expensive part gets paid only when it is necessary. The alternative — letting the model decide every round whether this may be deleted, whether it needs approval — puts the unreliable part on every path."

Stage	Who owns it	What guarantees it	What happens when it fails
Propose	the model	context and tool declarations	an unreasonable proposal, but nothing has landed yet
Approve	the host	the policy in code	a missed approval lets it through, and the side effect lands
Execute	the host / the tool	the executor implementation	the result may fail or be polluted
Record	the host	logs and state	no way to trace back; debugging becomes guesswork

"What if the host's approval logic itself has a bug?" Xiao A asked.

"Then record that separately too." Lao Z said. "The model proposing wrong is a decision-quality problem. The host approving wrong is a policy-configuration problem. You fix them in completely different places: the first may need a change to the tool declarations or the context; the second needs a code change. Recording them together puts two responsible parties inside one black box, and when something goes wrong both just share the blame."

"So the host's policy is just writing every rule you can think of as an if — and the more you pile up, the safer?"

"Pile it up until nobody can read it, and the policy itself becomes the new source of unreliability." Lao Z said. "A policy must stay simple enough to review, and the critical paths must have tests. The boundary exists to catch the model's unreliability. Don't let it turn around and become a new source of complexity in the system."

Key points

• The core of an Agent is the loop, not the model
• The model proposes; the host approves and executes
• A single question-and-answer is not an Agent

7.2 Going from "Saying" to "Doing" Runs on the Loop

Xiao A drew a circle on paper:

the model generates the next request
    ↓
the host validates and executes the tool
    ↓
put the result back into the context
    ↓
ask the model again for the next step (or to stop)

"Isn't this just a loop?" Xiao A asked.

"It is. But watch the responsibility of each stage." Lao Z said. "The model owns 'what to do next'. The tools own 'turning an action into a real result'. The context owns 'letting the model see what happened last step'. Miss any one of them and the task stops somewhere, or gets distorted."

"Once a tool result comes back, does the model know what to do?"

"Not necessarily. Whatever the tool returns is what the model sees." Lao Z said. "If the tool returns nothing, or returns an error, or returns polluted content, the model can only keep going on bad information. The quality of the loop depends on whether the information at every stage is true."

"Which case is worse? No return, or a wrong return?"

"The wrong return is worse." Lao Z said. "With no return, the model at least knows 'this step did not happen' and can try another path. A wrong return with a normal-looking format is taken as fact, and the model keeps reasoning forward from it, so the error compounds round after round. A silent failure stops things in place. A failure disguised as success starts a landslide."

"What can the host do?" Xiao A asked.

"Validate the shape of the result before it enters the context." Lao Z said. "For example, the tool says 'success', but the result is missing fields that a success must have. The host should mark it as an anomaly before handing it back to the model, not pass it through as-is."

Xiao A thought for a moment. "So a loop is not better the faster it runs. Slowing down to validate is actually steadier."

"That's the right direction. Speed bought by skipping validation gets paid back in later rounds, as distorted decisions. The value of a loop is credible progress, not how fast one round runs."

"That's a bit like a relay race." Xiao A said. "If each handoff is slightly crooked, everything after it is crooked."

"Right, and the crookedness accumulates." Lao Z said. "In round one the model decides based on a half-true result. In round two it treats that decision as fact and builds on it. The error doesn't just sit where it started; every round feeds it back in as input."

"What does validation actually check?"

"At least three things: whether the result arrived, whether the fields that must be there are there, and whether the status matches what it claims." Lao Z said. "If the tool says 'success', there must be output that a success should have. If it says 'failure', it must carry the reason."

"Where does this round of validation stand? I keep worrying I'll miss it."

"It stands exactly at 'before the result goes back to the model'." Lao Z drew a diagram with the checkpoint marked:

model decides next step → proposes action → host validates proposal → runs tool → result
     ↑                                                                                  │
     │                         ←── shape check, status check ──                         │
     └──────────────── the validated result goes back into the context ─────────────────┘
                                               ↑
                 the check sits at the result's entry, not at the tool's exit

"This check belongs to the host, not the model. Run it once, and it blocks several rounds of chained distortion afterward."

Information stage	What the distortion looks like	How the model gets led astray
The tool returns nothing	the result is missing	it assumes the action never happened and retries in place
The tool returns an error	an error result carrying the reason	it knows it failed, and can adjust or give up
The tool returns a fake success	it claims success with no output	it takes that as fact and keeps going; the error grows every round
The context is polluted	irrelevant or stale content mixed in	it plans the next step from a false premise

7.3 Workflow and Agent Are One Spectrum

Xiao A came across another system in the team: customer-support ticket processing, where every step was hard-coded.

"This isn't an Agent, right?" he asked.

"This looks more like a Workflow." Lao Z said. "A Workflow writes its decision points into code: if the condition holds, take this branch; otherwise take that one. It is predictable, easy to test, and its cost is controllable."

"And an Agent?"

"An Agent hands some of its decision points to the model, to be decided at run time." Lao Z said. "The more complex the task, and the harder it is to enumerate the branches in advance, the greater the value of model decisions. The price is that behavior cannot be fully predicted."

"So one doesn't replace the other?"

"No. Most real systems sit in between: an outer Workflow holds the main flow, and an inner layer lets the model decide where it must. Treating the two as either-or makes you miss all the reasonable middle forms. This topic is developed in Chapter 16: When There Is More Than One Route — Workflow & Graph."

"How do I judge whether a stretch of flow should be written as a Workflow or given to the model?" Xiao A asked.

"Look at whether the branches at that decision point can be counted in advance." Lao Z said. "Finite branches with clear conditions: write them in code — stable and cheap. Branches too many to finish writing, or conditions you can only judge by understanding the context: only then consider letting the model choose at run time. The test is 'can I write all the ifs in advance'."

"Mixing the two — is there a cost?" Xiao A added.

"Yes. When something goes wrong you have to tell which layer was wrong: did the Workflow take the wrong branch, or did the model make the wrong decision at its layer. Debugging and auditing get more expensive as the number of mixed layers grows. So mix on demand — more layers is not better."

"At the two ends of the spectrum, is Workflow on the left and Agent on the right?" Xiao A asked again.

"Watch the scale: 'how many decision points are handed to the model'. All of them in code is a pure Workflow. All of them to the model is a high-autonomy Agent. Where your system sits on this axis depends on how much unpredictability the task can tolerate."

Lao Z drew an axis on paper:

all decision points in code                  all decision points to the model
     │                                              │
 pure Workflow ──────────── hybrid ──────────── high-autonomy Agent
     │                                              │
  predictable, cheap                   open, not fully predictable
     │                                              │
  branches countable in advance     branches uncountable; run time decides

"Different positions on the axis carry different costs." Lao Z pointed at the middle:

Dimension	Pure Workflow	Hybrid	High-autonomy Agent
Where the decision points are	in code	main flow in code + local model decisions	in the model at run time
Predictability	high	medium	low
Cost per run	low and stable	medium	can run away
Debugging difficulty	low	rises with the number of mixed layers	high
Typical task	fixed path, countable branches	most real systems	open, changing fast

"So when you choose a position, the question you actually have to answer is 'how much unpredictability can your task tolerate, and how much debugging cost can you pay' — not 'using a model looks more advanced'." Lao Z said.

"Is this spectrum the same thing as the three signals in 7.5?" Xiao A asked.

"You can read them side by side." Lao Z said. "The spectrum picks the position: it tells you which scale to look at. The three signals verify the position: they tell you where the critical thresholds on that scale are, and whether this position can be landed safely."

7.4 Autonomy Is Not the Goal

Xiao A heard "run autonomously" and his eyes lit up: "So if it runs by itself, people don't have to watch it?"

"Do the accounting first." Lao Z said. "Every bit of autonomy you give the model is a bit of predictability, auditability, and cost control you hand over. One task may converge in a few minutes, or burn a large budget without ever stopping."

"If I have to give up all that, what am I buying with it?"

"The ability to handle open tasks whose branches no one can write in advance." Lao Z said. "But for every increase in autonomy you must add, in step, stop conditions, approval points, and failure paths. Autonomy and guardrails come in pairs."

"Can the guardrails wait until the Agent is running?" Xiao A asked.

"No — that order is backwards." Lao Z said. "Once you raise the autonomy, you have already allowed it to push tasks forward with nobody watching. Adding guardrails then is bolting brakes onto a car already in motion: in the rounds you didn't catch, the side effects already landed."

"How do you set how much autonomy is right?" Xiao A asked.

"Reason backwards from the cost of failure." Lao Z said. "The less reversible an action — deleting files, sending mail, making payments — the lower the autonomy, and the more it needs an approval gate. Purely read-only actions that can be redone can have higher autonomy. The autonomy budget is set by 'how much of the damage can be rolled back', not by 'how smart the model is'."

"Put that in a table." Xiao A said.

Action type	Reversibility	Consequence of failure	Autonomy budget
reads, searches	high, no side effects	at worst, some extra tokens	high — you can let go
writing drafts, caches	medium, can be overwritten	data overwritten, but still recoverable	medium — set a budget cap
sending mail, changing config	low, nearly irreversible	real users are affected	low — gate it with approvals
payments, deletions, releases	very low, wide blast radius	incident-grade	very low — confirm each time

Xiao A read the table again. "The guardrails you mention — what exactly are they?"

"At least four: stop conditions, a budget cap, approval points, and failure paths." Lao Z said. "The stop conditions define when it counts as 'done'. The budget cap defines the most it may spend. The approval points define which actions must stop and wait for a person. The failure paths define where to retreat when it stalls or errs. Miss any one of the four and 'letting it run' becomes 'letting it run wild'."

"Isn't a hard-coded budget cap too rigid?"

"You can set one, but then it must trigger an action." Lao Z said. "When the budget runs out, stop, leave the completed part in a resumable state, and hand 'why it stopped' to the next step to deal with — rather than letting the task run bare on budget until it times out or causes an incident. Stop conditions are decisions, not defaults: asking once after each round 'is continuing still worth it' is far cheaper than running to the ceiling and rewinding."

"After it stops, can it resume?"

"If every step kept its state and logs, yes." Lao Z said. "Resuming means restoring the context and carrying on from the last round's output. That requires every intermediate result to be savable and reconstructible. Recording 'how many percent complete' is not enough; you have to record 'which action is done, and what its result was'. A task that cannot leave intermediate state behind can only start over when it stops."

"The same action in a different task — does the budget change too?" Xiao A asked.

"It does. 'Delete' is a routine operation in a cache-clearing task and a high-risk one in a task that deletes a production database." Lao Z said. "The budget must follow the task context; you can't set it by action type alone. A common design in one system: default to a low budget, and only raise it for the part of the task that declares explicit boundaries — directories, accounts, spending limits."

Key points

• The more autonomy, the less predictability and auditability
• Guardrails must be designed alongside autonomy; they cannot be added later
• "Can run autonomously" does not mean "should run autonomously"

7.5 When to Use an Agent

Xiao A listed the candidate tasks and asked Lao Z: "How do I judge whether a task should be built as an Agent?"

"Watch three signals." Lao Z said,

Three signals for using an Agent

1. The task is open: the branches cannot be enumerated in advance and need run-time judgment
2. The environment changes: inputs, state, or tool results become known only during execution
3. Failure is tolerable: there are clear stop conditions, and the worst outcome is within what you can accept

"Conversely, if the task's path is fixed, the inputs are controlled, and the cost of failure is extremely high, an Agent may not be the best answer." Lao Z said. "Ask first 'does this need run-time decisions', then 'is an Agent worth it here'."

"Do all three signals have to hold before using one?" Xiao A asked.

"Not all three, but one is hard: failure must be tolerable." Lao Z said. "The first two decide whether an Agent can add value. The third decides whether it can add that value safely."

"Beyond the three signals, is there a fourth thing worth flagging?"

"There is one: whether the task contains wholesale repetition." Lao Z said. "If the same kind of task shows up dozens of times a day, don't rush to build an autonomous Agent. The cheaper move is usually to build the fixed part as a Workflow or a template first, and leave only the varying part to the model."

"What goes wrong if you force a fixed task into an Agent?" Xiao A asked.

"You pay cost for nothing." Lao Z said. "For that kind of task, a Workflow that just runs the path is cheaper and steadier. Built as an Agent, every run asks the model to make a decision that should have been executed directly — one more call, one more layer of unpredictability, for 'flexibility' you never use."

Lao Z added a chain of judgment:

Can the branches be counted in advance? ── yes ─→ hard-code a Workflow
        │
        no
        ↓
Are inputs only known during execution? ── no ─→ rethink whether you need
                                                 run-time decisions at all
        │
        yes
        ↓
Is the worst failure tolerable? ── no ─→ semi-automatic + approvals
                                         on key steps
        │
        yes
        ↓
design it as an Agent

Xiao A studied the chain for a while. "So the three signals are really two layers?"

"You can read it that way." Lao Z said. "The most dangerous tasks are the ones where the first two hold and the third does not: the model has every reason to run, but if it runs wrong you cannot afford the loss. Those become semi-automatic with step-by-step approvals, never a straight autonomous Agent."

Dimension	Better suited to an Agent	Better suited to a Workflow
Number of branches	cannot be enumerated	finite and explicit
Input certainty	known only during execution	known at the start
Cost of failure	reversible or tolerable	irreversible or extremely high
Cost sensitivity	accepts calls and unpredictability	wants stability and low cost

7.6 What Makes Up One Agent

"What is the minimum set of parts an Agent needs?" Xiao A asked.

Lao Z listed them:

The minimal parts of an Agent

• The model: generates the next request
• The context: carries the system rules, the history, and the tool results
• The tools: turn actions into real-world results
• The loop: chains the three above together and enforces the stop conditions
• The boundaries: permissions, budgets, approvals — the mechanisms that cap the worst outcome

"These five parts are exactly the map of the ten chapters that follow." Lao Z said. "How the model generates, how the context is managed, how tools are designed, how the loop turns, how the boundaries are set — each chapter answers one part's engineering question."

"What do you mean by 'minimal'? Can one of them be missing?" Xiao A asked.

"Missing one and it degrades into something else." Lao Z said. "Without the model, the remaining logic is just dead program. Without the context, the model sees a blank page every round and cannot carry the previous rounds' conclusions forward. Without the tools, it can only output suggestions and never land anything. Without the loop, it turns once and stops, so multi-step tasks never finish. Without the boundaries, there is no limit on how fast it runs, how much it spends, or which files it touches. These five are not configuration options; they are the conditions for being an Agent. A real system adds more to each part, but the base cannot be skipped."

Part	What it owns	How it degrades without it
Model	generates the next request	only program logic left, no decisions
Context	carries rules, history, and results	amnesia every round; the task can't continue
Tools	turn actions into real results	all talk, stuck forever at "saying"
Loop	chains the three and enforces stop conditions	a single Q&A that never converges
Boundaries	cap the worst outcome	uncontrolled; one wrong action crashes it

"Note the division of labor between boundaries and the loop." Lao Z added. "The loop owns 'when to stop'; the boundaries own 'what must not be touched before it stops'. One governs the endpoint in time, the other the scope in space. Neither substitutes for the other. Writing the stop conditions clearly does not mean out-of-bounds actions can't happen. The boundaries have to be set separately."

"Can one task thread all five parts through?" Xiao A asked.

"Use 'run the tests in the repository and fix the failures'." Lao Z said. "The model reads the task and the repository information in the context, and proposes run_test (the model). The host checks the boundaries: is this tool allowed, is the test name in the arguments on the whitelist (the boundaries). Validation passes, the tool executes (the tools), and the test output is written back into the context (the loop). Seeing the failures, the model proposes the next step, and this continues until every test passes or the budget cap is reached (the loop's stop condition). Miss any one of the five and this task stops or runs out of control at some step."

"So when you debug, you also locate the fault by these five parts?" Xiao A asked.

"Exactly. When a task stalls, ask first which stage it stalls at. The model fails to propose anything reasonable: the context is insufficient. It proposes but nothing executes: the host policy, or the tool doesn't exist. It executes but the result is wrong: the tool implementation, or validation is missing. It keeps turning without converging: the loop lacks stop conditions. It touches something it shouldn't: the boundaries weren't set. Five parts, five fault classes, and five classes of logs."

"Among these five parts, which is most likely to be the weak link? I'd guess the model."

"The weak link is whatever the task lacks." Lao Z said. "For open-ended tasks, missing boundaries is the most dangerous: the smarter the model, the more damage it does when it strays. For long-chain tasks, missing context is the most dangerous: conclusions from earlier rounds don't carry forward, so everything after repeats or forgets. For multi-step tasks, a missing loop is the most fatal: it stops after one turn, and the task never completes."

7.7 Leave Generation to the Model, Keep Control in the Host

Xiao A looked at the list and suddenly realized something: "So the model is just a machine that generates text. What makes it able to do work is the loop and the boundaries."

"Precisely." Lao Z said. "The model thinks about the next step. The loop strings the steps together. The boundaries keep it from causing damage. None of these three is something the model can do by itself."

"What if we put control into the model too — let it decide for itself whether to stop, whether to approve?" Xiao A asked.

"That doesn't work, because it makes the supervised party supervise itself." Lao Z said. "If the model both proposes and approves an action, there is no approval step at all. Control stays in the host precisely because the model is unreliable: the reliable part should go to the model, and the unreliable part must stay in code. That is why we started from the model, but do not stop there."

"Spread the list out — which items go to the model, which stay in the host?"

To the model	In the host
proposals for what to do next	whether to execute, and when to stop
which tool to pick, what arguments to fill	policy validation, permission blocking
interpreting results and planning the next step	the state machine, stop conditions, approvals, and records

"This looks like a division-of-labor table, but it is really one boundary line." Lao Z said. "The rows above the line are allowed to go wrong: a bad proposal can be refused, bad arguments can be blocked — the cost has not landed yet. The items below the line, once they err, the cost has already happened. Put the unreliable things above the line; keep the reliable things below. Where this line is drawn decides whether the whole system can safely let go."

"Could this line be drawn in the wrong place?" Xiao A asked.

"Yes, and it moves with the task." Lao Z said. "Add a high-risk tool, change an approval policy, and the line should be redrawn. The act of drawing the line must itself be auditable: who, when, moved which item from below the line to above — all of it should leave a record. Next chapter, we put the first 'handles' on this machine: tools. The model proposes, the host executes, and tools make actions really land."

Summary

An Agent is a task working unit that organizes model decision-making, tool execution, and loop control. It is not a smarter model. The line between it and a single question-and-answer is the loop: it only counts once it loops. The model proposes the next step, the host approves and executes — this division of labor is the starting point of every reliability discussion. Proposing is cheap; approving is expensive, which is why approval happens only when an action is truly about to land. The model has the ideas, the host keeps the boundaries: that is what lets an Agent touch dangerous actions without crashing. Every round of the loop must feed the model verified facts. A failure disguised as success amplifies round after round along the loop, while a silent failure stops things right there.

Workflow and Agent are the two ends of one spectrum: decision points written in code make a Workflow; decision points handed to the model make an Agent. To judge where a stretch of flow belongs on the axis, look at whether its branches can be counted in advance. Most real systems sit between the ends: an outer Workflow holds the main flow, an inner layer lets the model decide where it must. The debugging cost that mixing brings rises with the number of layers, so mix on demand.

Autonomy buys the ability to handle open tasks, at the price of predictability, auditability, and cost control. The ruler for the budget is how much of a failure can be rolled back, and the budget also shifts with the task context: the same action may be routine in one task and high-risk in another. The more autonomy, the more you must add in step: stop conditions, a budget cap, approval points, and failure paths. Guardrails and autonomy are designed together; adding them after it runs means the side effects have already landed. Whether to use an Agent comes down to three signals: is the task open, does the environment change, is failure tolerable. The first two decide whether an Agent can deliver value; the third decides whether it can do so safely.

A minimal Agent consists of five parts — model, context, tools, loop, and boundaries — and they are exactly the map of this book's Part I · Concepts: how the model generates, how the context is managed, how tools are designed, how the loop turns, how the boundaries are set; each chapter answers one part's engineering question. None of the five can be missing. Without the loop it is a single Q&A; without the boundaries it is uncontrolled; without the context it forgets every round. Above all the parts runs one boundary line: what goes to the model, what stays in the host — where the line sits, how it moves, and who changed it must all be answerable. This is the last step that turns a clever typewriter into a trustworthy machine.

Hands-on Verification

Implement a fixed-step task (such as "read the configuration and print it") twice: once as a Workflow, once as an Agent. Run each version 10 times, and record the completion time, the number of failures, and the output at each failure. Watch especially for what decisions the model makes on boundary inputs in Agent mode that "no code ever wrote" — that is exactly where the value and the risk of an Agent both live.