Chapter 1: The Model Is a Component — The Role of the LLM and Choosing One

Xiao A got a new assignment: pick a model foundation for the team's Agent. He opened the model catalog. More than twenty names, parameter counts from a few hundred million to a few hundred billion, prices two orders of magnitude apart. Online, some people said a large model can do anything; others said it only fits specific tasks. The more he read, the more confused he became.

He threw the question at Lao Z: "So how am I supposed to choose?"

Lao Z did not answer directly. "Don't rush to compare parameter counts. First tell me what the model does inside your Agent."

"It generates answers. What else?"

"If it could do everything, Agent engineering would not need to exist," Lao Z said. "This chapter looks at no specific product. We put the LLM back inside the coordinate system of an Agent and get to know its temperament. Once the coordinate system is built, the selection question answers itself."

What this chapter answers

• What the LLM is responsible for inside an Agent, and what it is not
• Where the capability boundary comes from, and why the model cannot be treated as an "all-purpose engine"
• Which constraints selection has to look at, and how evaluation replaces gut feeling
• Why the model should be treated as a swappable component

Not covered: the generation mechanisms — token, autoregression, context window, sampling, hallucination — are left to Chapter 2: The Truth About the Next-Token Machine — LLM. Vendor API differences and their abstraction belong to Chapter 4: How the Translation Bureau Connects to Models — Provider.

A first take on the Agent: for this chapter, you can roughly read "Agent" as a system where "the model proposes the next step, a program executes it, and the program decides whether to continue". The program that executes the action is the host. The full definition is in Chapter 7: A Machine That Relays Automatically — Agent.

Aligning the names first

Term	English / Abbreviation	Working definition in this book	What it is not
Model service	model service / LLM API	A product that exposes model inference through an interface; one call returns text or a stream of events	One call changes the model's parameters
Inference call	inference call	One request an application sends to a model service; billing, latency, and failures are all counted per call	Model training or fine-tuning
Capability boundary	capability boundary	The range of tasks a model completes reliably under the constraints of its training data and inference resources	Everything a vendor advertises as possible
Model selection	model selection	The process of choosing a model under the constraints of task, cost, latency, and availability	Choosing by parameter count or leaderboard rank
Evaluation	evaluation	A means of quantifying model behaviour with reproducible task samples; an evaluation answers only for its samples	One impressionistic trial
Model substitutability	model substitutability	The property that the underlying model can be swapped under the same contract without changing application logic	Output staying predictably identical after a swap

1.1 The model is a component, not an oracle

Xiao A started from the assumption that wiring a model into his system meant hiring a "colleague who knows everything".

Lao Z took the idea apart. "The model does exactly one thing: it generates continuation text from the input. What tools your system has, what permissions it grants, what history it keeps — the model knows none of that. The system assembles everything and hands it over."

Xiao A tried it against his own business. "A user asks 'which stage is my ticket stuck at'. The model turns that sentence into a presentable answer. But for 'where the ticket actually is', the system has to query the business database and tell the model?"

"Exactly. The model does not know where your database is, which table the status field comes from, or whether what it reads may be shown to this user. Its job is to organize the material it receives into text. Where the material comes from, and whether it can be disclosed, are the system's business. Draw that division of labour clearly, and only then do choosing, evaluating, and swapping models have a coordinate system."

"Then does it matter how strong the model is, or not?"

"It matters, but it is not everything," Lao Z said. "Whether one real task gets done depends on three things working together: model capability, tool availability, and system constraints. The model is a replaceable component inside the system, in the same standing as a database or a message queue. It should obey the system's contract, not the other way round."

He drew the three-legged diagram:

Whether the task gets done
   │
   ├── Model capability: can it write the right action
   ├── Tool availability: is there real capability behind the action
   └── System constraints: do permissions, checks, and workflow allow it to run

"If any one leg is short, the task falls over," Lao Z said, pointing at the drawing. "However strong the model, if the tool is not implemented or the permission is not opened, the action stays on paper. The other way round: a mediocre model, but tools and checks hold every step — the task still gets done. Selection only picks one leg for the system. The other two decide success just as much."

Xiao A was not convinced yet. "Suppose the model is strong enough to write tool calls correctly. I hand all the judgment to it. Fine?"

"No. 'Correct once' and 'correct every time' are two different things," Lao Z said. "A model occasionally emits an out-of-bounds argument or a hallucinated path. Whether your system survives that depends on whether you put validation between the model and the real action. Handing all judgment to the model means betting the reliability of the whole chain on a single generation."

He went on with a concrete case. "The model passes one extra argument in a tool call, or leaves out a required field. Forward the call as-is, and the tool may perform an action different from the one you expected. Put a schema check in the middle and the call gets stopped; the system can retry, or hand it to the user for confirmation. The same model error: the first architecture amplifies it, the second absorbs it."

"Then what does 'replaceable' mean exactly?"

"It means the model changes and the rest of the system is not rewritten," Lao Z said. "What you agree with the model is an interface contract: given an input, return text, plus possibly a tool-call structure. As long as the new model keeps that contract, you swap what is underneath and the tools, the loop, and the permissions stay put. But if the tool descriptions, the prompts, and the argument formats are all tuned to the edge for one model's temperament, swapping models means retuning the whole set. That is not a component; that is coupling."

"Can the contract be broken?"

"It can. Two main kinds," Lao Z said. "One: after a model upgrade the output style changes — same input, more wordy or more terse — so downstream parsing has to change too. Two: the model quietly stops honouring some implicit convention, say it always ordered fields a fixed way and now it is random. Anything not written into the contract is not part of the contract. Put your assumptions about model behaviour explicitly into validation and parsing; only then does a breakage become visible when it happens."

Xiao A listed the kinds of breakage he had seen. Lao Z turned them into a more concrete table:

Common contract assumption	Breakage signal	How to make it explicit
Returned field order is stable	Order changes after an upgrade, positional parsing misaligns	Parse by field name, not by position
Tool-call arguments are complete and valid	Occasional missing, extra, or wrong-typed arguments	Schema validation, retry on failure
Wording style is stable	Output turns visibly wordy or terse after an upgrade	Pin the template in the prompt, summarize the output when needed
Implicit ordering or deduplication conventions	Ordering becomes random, results repeat	Write the rules into the prompt, re-verify them in the validation layer

"Note that not one of these entries appears in a vendor's documentation," Lao Z said. "They are unspoken understandings between you and the model. When an understanding becomes a written clause, a breakage is a reportable failure instead of a hard-to-reproduce oddity."

Xiao A then worked through a swap scenario. "Say I switch models and the agreed 'fields come in a fixed order' is gone. Who do I go to?"

"First see which layer notices it," Lao Z said. "If parsing is written positionally, it breaks the moment the model changes — the validation layer notices first. If parsing is by field name but the content is not checked, the fields still arrive complete after the swap and only some conclusion has changed — the evaluation set notices first. The validation layer catches structure; the evaluation set catches content. Missing either layer leaves a blind spot at swap time."

Key points

• The model's job is generation; the system's job is execution and constraint
• Capability, tools, and constraints together decide whether a task gets done
• The model is a replaceable component, not the master of the system; reliability comes from system validation, not from a single generation

1.2 The capability boundary comes from training, not from marketing

Xiao A came across a model description that read "supports conversion between any formats, writes code in any language". He was tempted, and took the wording to Lao Z: how do you judge claims like that?

"Read it as marketing copy, not as a technical specification," Lao Z said. "The capability boundary comes from training data, training objective, and inference resources. In domains the data never covered, it can only continue by analogy. In domains the data did cover, the information may already be out of date."

"What does 'out of date' look like?"

"Training data has a cutoff date, but your repository, your documentation, and your APIs are alive," Lao Z said. "The usage the model learned predates the newer version of your interface, so the call it produces can be syntactically sound and still not run, because it does not know what happened afterwards. The boundary is therefore not only about whether it can do something, but also about whether it kept up with your versions."

"Something continued by analogy also sounds like it 'knows how'."

"That is why you look at whether the move lands," Lao Z said. "In a domain the data covered, the API names, argument names, and file structures it gives usually match the real environment. When it is improvising, the call often parses fine while the objects it references do not exist — calling a function that was never written, or assembling code that looks right against a library nobody has published. The first is memory, the second is the inertia of the completion game. In a short sentence they are hard to tell apart; put them into real execution and one run gives it away."

"Then inside one model, how are the memory zone and the analogy zone distributed roughly? Is there a ratio?"

"No ratio, but there is one regularity," Lao Z said. "Mainstream languages, common frameworks, and the usage of public libraries — it has seen a lot of those, so they usually fall in the memory zone. Private protocols, internal naming, a version released last week — it has not seen them, or has seen few, so they are almost all in the analogy zone. The closer your task is to public corpora, the more likely its claim of 'knowing' is true."

Lao Z drew a capability band:

 Memory zone────────────Ambiguous zone──────────Analogy zone
common frameworks    │  rare but real        │  private protocols
mainstream libraries │  uncommon idioms      │  internal naming
public docs          │  semi-public material │  just-released versions
          │                       │                          │
verify by running       verify by search + run  treat as completion, distrust

"This band also moves," Lao Z said. "A model upgrade shifts it a little. If your system adds a retrieval tool, evidence gets carried into the context, and the parts of the analogy zone where material is obtainable temporarily become a 'lookup zone'. To judge which zone a capability falls in, ignore the model's reputation and probe with your real tasks."

"The probing method, I'd guess, is: let it run in the real environment and see whether what it references exists."

"Yes. Between 'looks right' and 'runs through' sits nothing but the data boundary. The closer you are to that boundary, the more you let execution results speak."

"Then if the references match up, can that prove it 'understands'?"

"It cannot, but it is enough in engineering terms," Lao Z said. "A passing run only says this output landed in the real environment. It does not say the model understands the principle. For a system, a result that runs is acceptable."

"Fine. So how do I actually learn the capability boundary?"

"Only by measuring. The capability boundary is not read out, it is measured. Documentation tells you which protocols and which arguments it supports, but 'which tasks it completes reliably' has to be verified against your real tasks."

"So with a claim like 'code in any language', which word is the problem?"

"'Any'," Lao Z said. "A language appearing somewhere in the training data does not mean the model can reliably produce usable code in it. Obscure languages, just-released library versions, private APIs — those are almost certainly in the analogy zone, not the memory zone. 'Any' is marketing vocabulary. In engineering you replace it with 'a reproducible success rate on a stated sample set'."

Xiao A nodded. "The real question is: for the handful I care about, which falls in the memory zone and which in the analogy zone."

"Yes, and break it down further," Lao Z said. "Both called 'can write code', but 'add a few lines to an existing function' and 'design a module from scratch' are two different capability shapes. 'Read the comment and change an argument' and 'wire up a service against your internal API docs' are not the same thing either. Marketing says one blanket 'can'. Engineering has to split it into pieces: 'on what input, producing what'."

1.3 Selection looks at four things

Xiao A filled every candidate model into one comparison table: parameters, price, latency, context length. When he finished he asked Lao Z: "So many columns. Which should I look at first?"

"No fixed order, but fixed dimensions." Lao Z circled four words on the table:

The four constraints of model selection

1. Task fit: is your task a shape its training covered
2. Cost: the per-call price, context length, and the total after concurrency multiplies them
3. Latency and throughput: how fast one call must be, and how much work it gets through per unit time
4. Availability: region, quota, data residency, and long-term stability

"These four constraints often fight each other," Lao Z said. "Fast and expensive, cheap but slow, strong but with a tight quota. Selection is a trade-off across the four, not a search for the 'best' model."

He also listed when each constraint jumps to the front, and what it costs when it fails:

Constraint	When it jumps to the front	The cost when it fails
Task fit	Frequent errors on your core tasks	Retries and manual clean-up; the hidden cost far exceeds the price gap
Availability	Region, data residency, or quota is restricted	You cannot ship at all; the other three never get a turn
Latency and throughput	Interaction has a hard target, or a batch has a deadline	Users will not wait, or the batch cannot finish in time
Cost	The budget has a firm ceiling	You overshoot the day you launch and are forced to swap models mid-flight

"No table will order these for you," Lao Z said. "The key is to find first which item is a hard constraint — 'unmet means dead' — then sort the rest by cost, largest first. There can be more than one hard constraint: if data must not leave the country and a single reply must come back inside one second, availability and latency both sit at the front together."

"How do I tell whether a constraint is really hard?"

"Say the assumption out loud. Ask: if this condition is not met, does the task still get done?" Lao Z said. "'Data must stay onshore' and the answer is 'then we change the whole approach' — that one is hard. The answer is 'a compromise is acceptable, say an extra approval step' — it is soft. List 'I want' separately from 'I must have', or during selection a soft constraint will rule out a model that actually fits."

"So should I just pick the cheapest?"

"Look at the worst case first. The cheapest model fails often on your core tasks; add retries and manual clean-up and the cost has long exceeded the price gap. Measure task fit first, then talk cost."

"Is there an algorithm for whether the price gap is worth saving?"

"No general formula, but you can estimate per scenario," Lao Z said. "Multiply the difference in failure probability by the number of core calls. The extra retries, retry-on-timeout, and manual review are pure cost. You save on unit price and spend on failure — add the two ledgers together."

"Then I flip it around and pick the strongest. At least I won't lose because the model was not strong enough."

"It will fail a different way," Lao Z said. "The strongest model is usually the priciest, with the tightest quota and the highest latency. In an interactive scenario, one slower step and the experience collapses. In a batch scenario, the quota caps you and the job never finishes. 'Not strong enough' and 'too slow and too dear' both fail the task; they just fail at different stages."

"Same four constraints — would the order differ between interactive and batch?"

"Very different." Lao Z drew two lines:

Interactive (user waiting):  latency and throughput → availability → task fit → cost
Batch (offline to finish):   cost → quota and throughput → task fit → latency

"In an interactive scenario, slow is deadlier than dear — the user will not wait through three retries. In a batch scenario, dear and never-finishing are deadlier than slow: results an hour late is usually fine, unit price doubled hurts. The same model can lead to completely different choices in the two scenarios."

"So the whole pipeline can only use one model?"

"It need not. One system can use several models at once and dispatch by task," Lao Z said. "Hand routing and classification, which are simple judgments, to a cheap small model; hand core generation or complex reasoning to a strong one. You can also give a single task a primary model and a fallback model, and switch when the primary times out or fails. Selection is not 'pick one'; it is 'pick a suitable one for each stage'. Only, the more stages you have, the more combinations you must test, and the evaluation set costs more to maintain."

"When the four constraints fight, is there a general priority?"

"There is a rough order. It is not absolute." Lao Z sketched a decision order beside the table:

Illustration: a common order for the selection trade-off

Task fit (does it get the job right, reliably)
       ↓ pass
Availability (can I use it here, and keep using it)
       ↓ pass
Latency and throughput (fast enough, and enough volume)
       ↓ pass
Cost (is the total bill acceptable)

Task fit comes first because nothing in the other three can rescue it: a model that cannot do the task saves you nothing however cheap. But if data residency, quota, or latency is a hard constraint, that one jumps ahead of task fit.

1.4 Replace gut feeling with evaluation

"How do I measure task fit?" Xiao A asked.

"By evaluation," Lao Z said. "Evaluation is not running a public benchmark and reading the score. It is using your own real task samples to quantify the model's behaviour on the things you care about."

Illustration: if your task is "extract the test command from a project's file list", the evaluation samples can be configuration files from ten different projects. After each run, record: the share of commands identified correctly, the share of file paths read wrongly, and the share that needed several extra rounds before being corrected. The score answers only for this sample set.

Lao Z drew how an evaluation set comes into existence:

Pull task samples from real requests
   │
   ├─ Label the "expected behaviour" by hand first: correct / evidence insufficient / should refuse
   │
   ├─ Classify by behaviour: happy path, boundary input, failure scenario
   │
   └─ Run the model → record the output → score against the labels
          └─ Keep the results as the regression set for future model swaps

"Is the labelling a lot of work?"

"It is, and it pays," Lao Z said. "Someone has to decide the expected behaviour for every sample first: what counts as correct, what counts as acceptable, what counts as a required refusal. If labelers disagree, the score means nothing. So rule one is to write the labelling guide before anything else: is the model's output for a human to consult, or does it trigger actions directly? The definition of 'passing' is completely different. An evaluation set is measured into existence and labelled into existence. Label it sloppily and every measurement is wasted."

"High score means ready to use?"

"The score is only a starting point. Evaluation has to cover three behaviours: the happy path, boundary input, and failure scenarios. For instance, when there is no configuration file at all, does it invent a command? A model that shines on ordinary samples but fabricates answers on boundary samples, and a model with average scores that honestly admits it does not know — put them into an Agent and the behaviour may be entirely different."

He listed what each of the three sample types probes:

Sample type	What it looks like concretely	What it tests
Happy path	Configuration complete, question typical	Ordinary completeness and format stability
Boundary input	Empty configuration, extremely long input, missing fields	Whether it forces an answer, whether it falls apart
Failure scenario	Retrieval returns nothing, permission denied, information out of date	Whether it fabricates, whether it has a way out

"Whichever type is missing, the evaluation misses that class of risk," Lao Z said. "A high score on the happy path only says the routine work is steady. What trips an Agent up most is exactly the odd corners it has not seen."

"What about public leaderboards? Surely they have some reference value."

"They do, but only for screening," Lao Z said. "A leaderboard measures public tasks; yours is not on it. It tells you roughly what a model is capable of on tasks of that type, so you can cut twenty-odd candidates down to a few. But the gap between a leaderboard score and your task can come from data distribution, task shape, or the way the evaluation is run — any of the three. A leaderboard's job is 'do not start from zero', not 'settle the decision'. Running your own samples once is more convincing than citing any leaderboard."

"Could the evaluation I run myself be biased too?"

"Yes. Two places mainly," Lao Z said. "Sample bias: you picked only the projects that felt convenient, so the model scores well on that batch and falls apart on a different repository. Metric bias: you only count 'was it answered correctly', not 'when it was wrong, did it say it did not know, or did it confidently invent something'. Those two errors have very different consequences inside an Agent. Bias cannot be removed. You can only know where it sits and cross-check with another batch of samples."

"Can you make the metric bias more concrete?"

"Take the single label 'answered wrong'," Lao Z said. "If it is wrong and honestly says 'I am not sure about this', the system can route the question to a human and the loss stays controlled. If it is wrong and hands back a path that does not exist, the user follows it and the loss is large. Merge 'fabrication' and 'uncertainty' into one bucket and the evaluation cannot separate the two outcomes."

"How many samples do I need?"

"Enough to show a trend. A handful of samples tell you whether it can do this at all; only tens of them tell you how often it fails. Distinguishing those two questions matters more than piling up counts."

"Once the evaluation set is built, is it done for good?"

"It has to follow the task," Lao Z said. "If the workflow changes, a tool gets renamed, or the acceptance criteria are updated, and the evaluation set does not, the score is measuring a task that no longer exists. Use it as a regression set when you swap models, and update it in step when the task changes. It is an asset you maintain, not a one-time scorecard."

1.5 Cost can be designed

Xiao A worked out the total and got a shock: "Just stuffing project files into the context already costs a lot in a month."

"Cost is not something you take passively; it is something you design," Lao Z said. "Every piece of content you put into the context costs you twice: the input cost of that call, plus latency — the longer the context, the slower the processing. Trimming history, caching a stable prefix, turning long documents into on-demand retrieval: those reduce exactly those two expenses."

"In what order do I apply them?"

"Sort by the content's characteristics, not by the technique's name." Lao Z drew a table:

Context content	Characteristic	The fitting cost measure
System prompt, tool list	A stable prefix identical every turn	Prompt caching, to avoid repeated processing
Chat history	Grows turn by turn, highly repetitive	Trimming, summary compaction, rolling inside the window
Long documents, large repositories	Bulky, most of it unused	On-demand retrieval, carry only the matched fragments
Long-term facts across sessions	Need to persist	Store them externally and read them with a tool

"The order is: first fix what can be fixed, then cut what changes," Lao Z said. "Stable prefixes go to cache; changing parts go to trimming or retrieval. Reverse it and the cache never hits while retrieval saves no money either."

"So the less context, the better?"

"No. Over-trim and the model loses the evidence it needs to finish the task. The goal of cost design is 'exactly enough to finish the task', not 'as little as possible'. Chapter 10: What to Remember, What to Forget — Memory develops this."

"How do I count the latency bill? It does not appear as a charge."

"It charges experience and throughput," Lao Z said. "In an interactive scenario, one second slower per turn and the user feels it stick. In a batch scenario, half a second more per turn, multiplied by your concurrency and total turns, is real machine time. Which ledger it belongs in depends on how sensitive your scenario is to 'slow', not on whether it appears on an invoice."

"When I build a budget, how do I get the numbers right?"

"Count in tokens, not characters," Lao Z said. "Different languages and different content types cut into very different token counts; a passage of Chinese can take several times the tokens of the same-length English. For an estimate, run a few real passages through the target API's counting endpoint and multiply what comes back. Leave some margin: failed retries are a cost too."

"Where does latency mostly come from?"

"At least three places," Lao Z said. "Input length — with the same model, a longer context means slower processing per call. Output length — generation is step by step, so long text is inherently slower than a short reply. Server-side queueing — at peak, no model is fast. The first two can be designed away with trimming and output limits; the third leaves you only off-peak scheduling, lower concurrency, or a bigger quota."

"Model services come streaming and non-streaming. How do I choose?"

"It depends on whether you want to see part of the answer early," Lao Z said. "Interactive scenarios usually pick streaming: display while it generates, and the user's first reaction is much faster. If you must collect everything before structured parsing, non-streaming is less trouble. This is not a difference in quality, it is a difference in arrival: the content waited for is the same, the only question is whether you see it in installments."

"Does streaming chop the content to pieces?"

"It depends on how the consumer downstream uses it," Lao Z said. "For a human to read, fragments flowing past are fine. To parse structure and extract fields, streaming can put a field name several frames away from its value, so the parsing logic has to wait until the full structure appears. The test is the consumer: comfortable for a human, streaming is friendly; if a program needs the whole thing, you usually collect it on the server and process it in one go."

1.6 Put the model back in the system's place

Xiao A took a while to digest it, and finally moved the model down from its throne as an oracle.

"So in one sentence: the model is the component inside the system that owns generation. How strong it is has to be measured, how expensive has to be computed, and whether it can be swapped depends on the interface you left."

"Yes. Those three things strung together are the whole chapter. Measure, so you know where the boundary is. Compute, so you know where the cost is. Leave an interface, so the model does not hold the system hostage."

Lao Z put the three into a table:

Design action	The question it answers	What it looks like in practice
Measure	What can this model do reliably on my tasks	An evaluation set of task samples, covering all three behaviour classes
Compute	What is the total bill for using it	Context cost, caching, and failure rate estimated together
Leave an interface	Does swapping it force a rewrite	One call abstraction, schema validation, prompt templates

"Add the last column and this table is the checklist for a model swap," Lao Z said. "The evaluation set becomes the regression: run the new model once and see which samples changed. The call abstraction guarantees the application code does not move. The templates and validation guarantee the output structure is still the old one. With all three in place, swapping models is an A/B test. Miss one, and swapping models is a refactor."

Xiao A thought a moment. "Then what Agent engineering does is build a scaffold of measure, compute, and leave-an-interface between the model and the real task?"

"Precisely. In this chapter we do not open the model. We deal with it as 'a component that generates text'. To really understand why it is this difficult to handle, we have to start from how it generates text, and that is later. For now, remember three things: measure, compute, leave an interface."

Lao Z drew the three as a checklist and told Xiao A to stick it up:

Before taking a new task or swapping a model, go through each line:
  [ ] Task fit: did I run it on my own samples? all three behaviour classes?
  [ ] Hard constraints: data residency / quota / latency — which one means dead if unmet?
  [ ] Cost: did I estimate the total bill in tokens? retries and clean-up included?
  [ ] Contract: does parsing go by field name? are my assumptions about the model in the checks?
  [ ] Regression set: after this change, does the evaluation set need updating?

Summary

This chapter took the LLM down from its throne as an oracle and put it back at the component layer of an Agent system. It only generates continuation text from the input; tools, permissions, and history are assembled by the system. The capability boundary comes from training data and inference resources. Marketing copy is not a technical specification, so the only way to know it is to measure against real tasks. Whether one instance of "it can" lands in the memory zone or the analogy zone is settled by execution results in the real environment, not by reading the promotional page.

Selection has no "best", only trade-offs. Task fit, cost, latency, and availability fight each other; measure task fit first, then discuss cost. Cost can be designed — trimming, caching, and on-demand retrieval all reduce overhead, but the target is "exactly enough to finish the task", not "as little as possible". Evaluation has to be read across three sample types: the happy path, boundary input, and failure scenarios. At the boundary, whether a model honestly admits it does not know or confidently fabricates corresponds to completely different risk levels. Treat the model as a replaceable component, and swapping what is underneath requires no rewrite of the rest: it obeys the system's contract, not the other way round.

Compress the chapter into a pocket list of three and neither leaderboards nor marketing will carry you off during selection: measure, so you know where the boundary is; compute, so you know where the cost is; leave an interface, so you know how much of the system moves when the model changes. Of the three, measuring is the entry point: once an evaluation set exists it becomes the regression tool for every future swap, and that is its most underrated long-term value. The model is a component, and "component" means: it has a temperament, it has a price, it has a boundary, and it can be replaced — provided you have measured it, costed it, and coupled it loosely.

Hands-on Verification

Pick two models and run each on 5 real task samples. Record only three metrics: the share completed correctly, the share that fabricated unverifiable information, and the number of cases that needed several rounds of interaction to complete. Then cut the context length in half, run it again, and note how completion quality and per-call latency or cost change. Do not read the result as "one model is better than the other across the board".