Chapter 3: Leaving a Scratch Paper for Thinking — CoT

Xiao A's reporting task went wrong. The task was to aggregate three months of sales data by region, compute the month-over-month change, and then flag the anomalies. The model did it in one shot and returned a single number that refused to reconcile, however he checked it. On a hunch, Xiao A appended one line to the prompt — "Let's think step by step" — reran the task, and the numbers matched.

He pasted the two outputs side by side on his screen, and the longer he looked, the more confused he became.

"Did I just... happen to write the right prompt?" he asked Lao Z.

Lao Z did not answer directly. He typed the same question twice: once asking for a direct answer, once asking for the steps to be listed. Then he asked, "What exactly did the line you added make the model do more?"

"Think more?" Xiao A ventured.

"Yes — that's the key," Lao Z said. "But there are many kinds of 'thinking more', and not every kind is worth paying for. This chapter discusses reasoning prompts, thinking budgets, and stop reasons. It does not mistake visible reasoning text for a complete disclosure of the model's internal process."

Align the names first

Term	English full name or abbreviation	Working definition in this book	What it is not
Reasoning	reasoning	the intermediate computation, search, or control process in a task with several constraints	an answer that has been proven correct
Chain-of-Thought	Chain-of-Thought, CoT	a method of putting intermediate reasoning steps in the output or the prompt	a complete transcript of the model's internal state
Reasoning intensity / budget	reasoning effort / budget	a control item in specific APIs for reasoning resources or output budget	a quality scale comparable across vendors
Stop reason	stop reason / finish reason	the state offered when a response ends, from which the caller chooses the control flow	a quality score for the answer
Host	host	the application that calls the model, executes the tools, and decides the next step; expanded in Chapter 7	the model itself

Chinese technical writing commonly renders both inference and reasoning with one and the same word. This book uses "model inference" for one generation pass with fixed parameters, and "reasoning" for the intermediate computation and control process this chapter discusses. The two are not at the same level.

3.1 What the reasoning process buys you

Xiao A ran the reporting problem again, and this time he watched every step of the model's output. The model first listed the per-region totals for each of the three months, then computed the month-over-month change, and finally flagged the anomalies. Every step was visible.

"Why does writing out the process make the answer more accurate?" he asked.

"Think of a person doing mental arithmetic," Lao Z said. "Mental arithmetic runs through to the end in one go: misplace one middle step and everything is wrong. On scratch paper you write down each step, do only a small piece at a time, and see at a glance which step broke. Research calls the practice of making a model generate intermediate steps Chain-of-Thought (CoT)."

Metaphor: scratch paper

Mental arithmetic versus writing out the columns. Scratch paper makes every step checkable, so errors stop snowballing; the same logic applies when a model writes out its intermediate steps.

"Then can I require the model to output all of its thinking, to guarantee correctness?" Xiao A asked.

"No." Lao Z shook his head. "The original CoT paper (Wei et al., 2022) reported that generating intermediate steps from worked examples improved performance on the arithmetic, commonsense, and symbolic reasoning tasks it tested. That is a fact about a method inside one set of experiments; it does not mean every task should ask for a long chain of thought. Visible steps can also be post-hoc rationalization, or can contain errors; some services keep the reasoning hidden and return only a summary or the final answer. Correctness still has to be judged by the answer, the evidence, and external verification."

"Why does writing steps reduce errors?"

"The key is turning one long error into short ones," Lao Z said. "Compute to the end in one breath, and a mistake in the middle means every later step builds on a wrong base. Write the steps down, and each step carries only a small piece of the computation, so the wrong step is easier to catch on its own. What has engineering value is not the number of steps but whether each step has checkable inputs and outputs. A pile of steps whose intermediate values cannot be verified is no different from doing it in one shot."

"Does writing steps have side effects?"

"It does. Output gets longer, and tokens and time grow with it; the same task may also take different intermediate paths on different runs, some direct, some roundabout," Lao Z said. "CoT improves average performance on multi-constraint tasks; it does not guarantee being right every time. It moves an error from hiding inside the result to showing up at a specific step, which makes it easier to locate — but locating is not fixing. The structural comparison looks like this:"

One shot : input ──> the final number directly (no step left to inspect)
Stepwise : input ──> totals ──> MoM ──> anomalies ──> conclusion (each step checkable)

"Let me guess: a task with many steps should keep scratch notes?"

"That is half right. The second point is whether the intermediate steps are verifiable," Lao Z said. "A task with many steps where each step has clear inputs and outputs — multi-round computation, debugging code, breaking down a plan — suits a step-by-step treatment. A task with few steps whose answer sits inside common sense gains nothing from steps but longer output. Whether there is a payoff, one run over a fixed sample tells you; you don't have to settle it by feel."

"And if the model itself doesn't support emitting thinking text?"

"Then don't ask it to 'think'," Lao Z said. "Some models keep the reasoning internal and give only the final answer or a short summary. In that case put the verification on the answer, instead of trying to force intermediate steps out from the outside. CoT depends on the model's output capability, and not every model cooperates."

"Does step-by-step output make an answer harder to read?"

"It can. With more steps, the conclusion gets buried at the end," Lao Z said. "So a prompt usually states the format explicitly — conclusion first, steps after — so the reader doesn't have to dig. Scratch paper serves execution; presentation order serves the reader. The two can differ."

Key point

CoT is a method, not a guarantee. It improves performance on some tasks, but visible thinking is not the same as correct thinking.

3.2 Three ways to leave a scratch paper

With the reporting task wrapped up, Xiao A was tidying his troubleshooting notes when he found he had actually used several different ways to make a model think more. He spread the notebook open for Lao Z.

"Three are common in engineering. Pick by scenario, don't mix them," Lao Z said, and laid out a table on the screen:

Approach	Mechanism	Fits	Main cost
Explicit prompting	ask in the instruction for decomposition, checks, or listed assumptions	the process needs to be visible to a person	output grows, and may expose irrelevant detail
Structured decomposition	the program splits the task into plan, execute, and verify stages	the steps and the acceptance criteria are clear	orchestration gets complex, and information is lost between stages
Reasoning-model budget	the API offers a control item for reasoning tokens or effort	difficulty varies widely and cost needs governing	provider semantics differ, and cost and latency increase

"Do these three rank from basic to advanced as you go down?" Xiao A asked, looking at the table.

"No. This is an engineering taxonomy, not a capability ladder," Lao Z said, closing the screen. "Whether a given model or API supports reasoning controls, and how those controls are named, has to be checked in the official documentation for that version."

"Then where does each one most easily go wrong?" Xiao A asked.

"Each has one typical trap," Lao Z said, and typed out another table:

Approach	Typical failure	How to recognize it
Explicit prompting	output grows after you ask it to "think step by step", but the steps are padded with irrelevant confession	many steps, few checkable intermediate values
Structured decomposition	information is lost in the handoff between stages; the execute stage forgets the verify stage's premises	plan, execute, and verify each speak for themselves
Reasoning-model budget	field names and semantics change with the vendor, so accounts don't reconcile across vendors	the same "high" setting, but the two vendors consume and produce very differently

"So is mixing them out of the question?"

"If you mix them, watch one thing: who is in charge within the same turn," Lao Z said. "Stack explicit prompting and structured decomposition in one turn, and the model may follow the text instruction and the program's stages at the same time, with two sets of instructions fighting. Keep the program's stage split and the prompt's wording apart: the program owns the flow, the prompt owns the expression. Don't issue two sets of instructions in one message."

"When should structured decomposition come first?" Xiao A went on.

"When each stage's acceptance condition is deterministic in itself," Lao Z said. "Take 'pick the data source, then generate the query, then run it, then reconcile'. Each stage produces a checkable artifact: the query text, the number of rows returned, the reconciliation difference. In tasks like that, writing the stage split into the program is steadier than letting the model improvise, because every link can be validated in code."

"And explicit prompting?"

"It fits cases where the process has to be shown to a person," Lao Z said. "A troubleshooting report a colleague has to review, for example: steps written into the answer are easier to follow than steps hidden in internal state. The cost is longer output and possibly exposed irrelevant detail. What you keep for people is a process they can read; what you keep for the program is an artifact they can check."

"What about the reasoning-model budget?"

"That one hands the decision back to the API's implementation," Lao Z said. "You only pick a level or a budget ceiling; the model decides internally how to spend it, and that process is invisible to the application and not controlled by the prompt either. It fits tasks whose difficulty swings widely, where you want to govern by cost but don't want to split the steps yourself. The cost is the black box: you cannot see the internal allocation, and can only infer it indirectly from the output and the usage."

"Can the three be nested?"

"They can, but keep them layered," Lao Z said. "For instance the outer level uses structured decomposition to define the stages, and one inner stage uses explicit prompting to ask for a step breakdown. Each layer manages its own business and they don't conflict. Nesting usually fails when you stack things in the same layer: inside one stage you ask for 'step-by-step thinking' and also paste in a fixed template."

3.3 A thinking budget is a resource constraint

That afternoon, Xiao A casually added "Let's think step by step" to the production code for the reporting task as well. Lao Z walked past, took one look, and frowned. "Do you know what that costs?"

"Costs?" Xiao A didn't get it.

"'Thinking a bit more' usually means more generation or more internal computation, which raises time, token usage, or both," Lao Z said. "Think of a thinking budget as an allowance of scratch paper: the larger the allowance, the more complex the calculation you can do, but the paper costs more too."

"So who gets more, and who gets less?" Xiao A asked.

"The budget should follow the risk of the task," Lao Z said. "Hand deterministic work — format conversion, simple lookup — to the program first. A higher budget for the model is worth it when the task involves multi-constraint reasoning, code changes, or a safety judgment, and you confirm the gain by verifying the result. One practical strategy is to set a total budget: maximum output per turn, total number of turns, the longest wait, and the degrade path after failure. Raising only the per-call thinking intensity while leaving the loop count unlimited can still let a task consume resources without end."

"What happens if the budget is set too high? Besides being expensive."

"Expense is the visible layer," Lao Z said. "The other is overthinking: the model lists a pile of assumptions on a simple task, output grows, latency rises, and a conclusion that was clear gets packaged with hedging. A budget is not steadier the more you give; it has to match the task's complexity. Too little isn't enough, too much is waste, and the line in between comes from measurement."

"And too little?"

"Too little usually gives you 'the right line of thought, unfinished'," Lao Z said. "The model may be cut off at the last step: the intermediate steps are written, the conclusion is not. That output is a half-product, and handling it is harder than handling a plainly wrong answer. Keep the incomplete state and continue, or rerun with a larger budget? Those are two different branches, and you must not store them as one result."

"Is there a difference between a per-call budget and a task-level budget?"

"There is, and they're easy to conflate," Lao Z said. "A per-call budget covers one model call; a task-level budget covers the whole multi-step flow. If one CoT run consumes most of it, the later steps have to downshift or wrap up early. Set only a per-call budget and no task-level budget, and a multi-step task can still overrun the total. This is the same idea as the total-turn limit in the Agent Loop, which Chapter 9 Making the Work Actually Loop — Agent Loop covers."

Xiao A wrote the two columns, per-call and task-level, in his notebook, then asked, "Once you overrun, where do you degrade to?"

"Down to the floor you can accept," Lao Z said. "Rerun at a lower level, switch to a deterministic program, or stop and ask a person — that depends on whether the task can tolerate worse output. Define the degrade path before the budget runs out; don't decide it on the spot once it's gone."

"Are a thinking budget and output length the same thing?"

"They are not. The output limit governs the visible text; a thinking budget governs the reasoning resources the model spends, inside or outside the answer," Lao Z said. "In some APIs the two are separate fields; in others internal reasoning tokens are counted separately. Don't use an output cap to limit thinking: set the thinking cap low and the model may not finish writing its steps; set the output cap low and the conclusion may be cut off. Set each one, and check each one."

Key point

Thinking is a resource, not a free lunch. The budget follows task risk, and a total budget has to back it up, or consumption becomes unbounded.

3.4 Stop reasons are control-flow signals

The next day, while debugging, Xiao A came across the raw fields the model returns, and one of them was stop_reason. He stared at that field for a long time, then went to Lao Z. "What is this thing for?"

"Model responses usually carry a stop reason, and the name differs by API," Lao Z said. "An application must not simply equate 'received text' with 'finished successfully'. Common semantics include: normal end, tool call, length limit, content or policy limit, and abort or timeout."

Xiao A turned it over for a while. "So is a stop reason a verdict of right or wrong?"

"A stop reason is not a quality score," Lao Z said. "An answer that ended normally can still be wrong, and a length-truncated one should not be stitched together and passed off as a complete answer. Its job is to let the caller choose to continue, repair, degrade, or exit with failure."

"What does 'tool call' as a stop reason mean?" Xiao A asked.

"It means this turn's output contains one or more tool-call requests, and the model is waiting for the host to execute them and bring the results back," Lao Z said. "That is not 'the answer ended', it is 'the turn handed over' — the loop still has to continue. Treat 'tool call' as 'done' and the loop stops at exactly the wrong moment, when the model made its request, with the task half finished."

Stop reason	Meaning	What the caller usually does next
Normal end	the model produced final text	go to completion handling (the content still needs verification)
Tool call	the model requests tool execution	execute the tools, and carry the results into the next turn
Length limit	the output reached the cap and was truncated	mark it incomplete; never treat it as a complete answer
Content / policy limit	content filtering was triggered	degrade, or rephrase the task
Abort / timeout	an external cancellation or a timeout	mark it interrupted and keep the partial result

Xiao A copied stop and length onto a sticky note. "How do you tell a length limit from a normal end?"

"Read the stop reason field; don't guess," Lao Z said. "Some APIs return stop on a normal end and length or max_tokens on truncation. The two values differ, so the handling differs too. A truncated output cannot be concatenated and stored as success, because it may break in the middle of a sentence and its meaning is incomplete. The UI should mark it as unfinished, so the user knows this answer is not complete."

"What if the program ignores the stop reason?"

"Each way of ignoring it has a concrete consequence," Lao Z said, and counted off four:

Ignored stop reason	Consequence
treating length as a normal end	half a sentence stored as a complete answer, and every later check is off
treating tool_use as completion	the loop stops early, and nobody executes the tool requests
treating error as a normal stop	the failure is swallowed silently and the task sticks in an error state
treating a normal end as necessarily correct	verification is skipped and wrong conclusions flow straight downstream

"So once the program has the stop reason, it's doing a dispatch?"

"Exactly. The stop reason goes into a dispatcher, and each reason maps to one branch," Lao Z said, and drew a diagram:

Response ends ── stop ───────────────> verify content → completion handling
              ├─ tool_use ───────────> execute tool → write results back → next turn
              ├─ length ─────────────> mark incomplete → re-collect or degrade
              ├─ content limit ──────> rephrase → retry or stop
              └─ aborted/timeout ────> keep partial result → report status

"Which layer does this dispatcher live in?"

"The host application layer, not the model layer," Lao Z said. "The model only returns a termination signal. How to respond to it — continue, repair, degrade, or exit — the host decides from the current task context. A stop reason carries no verdict of right or wrong. It only tells the host how this turn ended; the remaining branch decisions belong to the host."

Xiao A scrolled further down the debug log. "Is there one stop reason per task?"

"No. A stop reason is the end signal for this one response, not the conclusion for the whole task," Lao Z said. "In a multi-turn task, every response has its own stop reason: the earlier turns may all be tool_use, and only the last one is stop. Reading 'the last response was stop' as 'the whole task succeeded' loses what happened in the earlier turns. To judge the task-level result, read every turn's stop reason as a chain."

"What if the stop reason is missing or empty?"

"Treat it as unknown; don't treat it as a normal end," Lao Z said. "An empty field may be a protocol difference, a version difference, or a server-side problem. Filling in a default value and letting it slip through puts the unknown into the decision chain. An unknown stop reason must take an explicit 'cannot determine' branch instead of being silently classified as success."

"Should the raw stop reason be passed through to the UI?"

"Pass it through, but make it readable to the user," Lao Z said. "A raw value like length belongs in the log; in the interface, 'this answer was truncated' is clearer. Store the raw value and the interpreted state separately. With both layers present, neither troubleshooting nor presentation lacks material."

Key point

A stop reason is a control-flow signal: it tells the program what to do next, and does not answer whether the answer is right.

3.5 Quality, cost, and verification

Before the reporting task went live, Xiao A came to Lao Z with a page full of numbers. "I ran the tests. Making the model think more does raise accuracy, but every run is twice as slow and three times as expensive. So where should the budget be set?"

"Don't assume higher is better. Write it down as an explicit trade-off," Lao Z said, and typed out an illustration:

Task is high-risk and verifiable   → raise the reasoning budget, and run verification
Task is simple and checkable       → cap the budget, prefer the deterministic program
Result cannot be verified          → lower automation privileges, require stated uncertainty

"Is this something I can just copy from someone else?" Xiao A looked at his own project.

"No, and this is an engineering opinion," Lao Z said. "The real thresholds should come from your latency target, your prices, your cost of failure, and offline evaluation. Don't infer from a single nice example that some budget level is better for every task."

"How do I test so it isn't just one nice example?" Xiao A asked.

"Fix the task sample, then score along dimensions," Lao Z said. "Split it into at least three layers: whether the result is right (tests pass, numbers agree, citations exist), how much it costs (tokens, latency), and what failure looks like (which stop reason, which assumption was wrong). Run a set of samples to get a distribution instead of running one to get a conclusion. If you raise the budget one level and none of the three dimensions improves noticeably, that level is not worth paying for."

"Are verification methods ranked by strength too?"

"They are. The strongest is deterministic checking: tests pass, numbers match, a command returns exit code 0. The middle tier is evidence verification: a citation to a source, traceable back to the raw data. The weakest is model self-assessment, where it says 'I think it's fine'," Lao Z said. "Self-assessment can serve as a self-check step in a prompt; it cannot serve as an acceptance basis. How strong the acceptance is decides how high automation may run: only tasks whose results admit deterministic checking suit high automation, and tasks that rest on self-assessment need their privileges lowered."

"Can you lay those three verifications out as a comparison?"

"Sure," Lao Z said, and sorted them from strongest to weakest:

Verification method	Strength	Fits	Limitation
Deterministic checking	strong	tests passing, numeric reconciliation, command exit codes	needs an executable definition
Evidence verification	medium	traceable citations, checkable sources	requires maintaining an evidence index
Model self-assessment	weak	internal self-checks, a last fallback	cannot serve as an acceptance basis

"Who records this data?"

"The host records it, not the model's report," Lao Z said. "Token usage, latency, and stop reasons come from the response metadata; correctness comes from the checker. Letting the model report 'I think I got it right' only mixes verification into generation."

"How often should the evaluation run? Once per change?"

"It follows the change," Lao Z said. "One line changed in a prompt, one level swapped in the budget, one model version upgrade — each is worth rerunning the same sample set. Shipping without a rerun is deciding on last time's impression."

3.6 Wiring "thinking" into a verifiable process

Xiao A decided to rebuild the reporting task from the ground up. He stopped asking the model to "think it through properly" and made every step produce something checkable: list the assumptions first, then pick the data, then compute, then reconcile.

"The steadier approach is not to demand unbounded long reasoning, but to give every stage a checkable artifact: assumptions, a plan, execution results, and a verification conclusion," Lao Z said after reading the new design. "The model can generate them, or a deterministic program can. The point is that a later step must never treat an earlier stage's guess as a fact."

Task → write assumptions → pick evidence or tool → get results → check acceptance criteria → conclusion
                           └── unavailable / timeout ──> degrade, clarify, or stop

"Then should everything on the scratch paper be stored? What about the stretch of reasoning the model wrote freely?"

"Not necessarily — it depends on the audit requirement," Lao Z said. "Visible intermediate text has review value, but it also leaks unnecessary content, occupies context, and easily leads later steps to anchor on a wrong premise. Long free-form reasoning especially should not be stored by default: it consumes storage, may leak information, and later steps tend to treat 'it thought about it' as 'it verified it'. For a system that needs auditing, recording which evidence was used, which tools were called, and why it stopped is usually more useful than saving verbose free-text reasoning. Storing what happened is closer to fact than storing what it thought: the first is verifiable, the second is the model's own account."

Illustration: when troubleshooting a test failure, first list "a missing environment variable" as an assumption, then read the configuration and run the related tests. If neither piece of evidence supports it, drop the assumption instead of going on editing code to keep the story coherent.

"How do you tell a draft from a conclusion inside the flow?" Xiao A asked.

"By whether a later step consumes it," Lao Z said. "A draft is a candidate anyone may overturn; a conclusion is something the evidence supports and that moves on to the next step or into the final output. The flow needs one explicit state promotion: an assumption rises from candidate to adopted only if evidence supported it. An assumption without evidence stays in the candidate region and does not enter the conclusions. If the model alone decides that promotion, you have mixed verification into generation again."

"And after verification fails?"

"A verification failure is a kind of result, not an accident," Lao Z said. "It means the current assumption does not hold, so the flow returns to 'pick the next assumption' or 'escalate to a person', and keeps the failure record. Next time the same class of problem appears, the known-bad path can be skipped outright. Treat failure as a branch in the flow instead of letting the loop spin in place, and verification means something."

"Then which checkable artifacts exactly should be stored?"

"Enough to answer three questions is enough," Lao Z said. "Which assumptions were used, which tools were called, and why it stopped — those three map to the draft, the evidence, and the stop reason. Store them, and afterwards you can answer 'what was this conclusion based on'. Store only the final answer, and you see the conclusion and nothing of the process."

"What is 'escalate to a person' in the flow?"

"A normal transition," Lao Z said. "When the assumptions run out, the evidence contradicts itself, or a judgment needs authority, the flow stops and hands the draft, the evidence, and the candidate assumptions to a person. Like a verification failure, it is a branch of the flow, not an exception. The more branches there are, the less the system needs to guess when it lacks information."

"How do you version the drafts and the evidence?"

"Follow the flow," Lao Z said. "A draft itself may be messy, but the version line must keep running: an assumption edited, overturned, or adopted leaves a record each time. Each evidence reference names which file or which tool return it came from. Then when a conclusion is challenged, you can answer 'this assumption became what it is at this point, based on that'."

3.7 Budget is part of the control flow

The last thing before shipping, Xiao A wanted to configure thinking intensity across vendors, only to find every vendor names its parameter differently. Flipping through the documentation, he asked Lao Z, helplessly: "If I set reasoning_effort, doesn't that unify things across vendors?"

"No. The field names and visibility of reasoning_effort, thinking tokens, or an internal budget are each vendor's own: OpenAI's reasoning models use levels like reasoning_effort, Anthropic's extended thinking uses a token budget like budget_tokens, and the units and semantics are not the same. They are capabilities of particular interfaces, not a general protocol."

Receive task → pick budget by risk → call the model
  → normal end: verify the answer and the evidence
  → length: keep the incomplete state, then continue or degrade
  → timeout / aborted: stop this turn and report the status
  → tool call: execute, return the results, then decide the next turn

Xiao A thought it over. "So the budget is deliberate allocation, not luck?"

"Right," Lao Z nodded. "The normal path is a verifiable conclusion reached within the budget. The two counter-examples are output reaching the length limit, and a high effort still resting on a wrong assumption: the first must not be disguised as a complete answer, the second shows that a budget only changes resource allocation. In engineering you should concentrate the budget on the steps that are hard and verifiable. The cost is that you need evaluation and monitoring, and unverifiable high-risk conclusions should have lower privilege to execute automatically."

"Then can one vendor's 'high' be compared directly with another's? Do you have to reconfigure when you migrate?" Xiao A asked.

"Not directly, and migrating almost always means reconfiguring," Lao Z said. "Vendor A's high effort may mean more internal reasoning tokens, while vendor B's same-named parameter may only change how the answer is worded. Field names, visibility, and cost structures do not line up. Copy A's configuration onto B and most of the time it does nothing, gets ignored, or means something entirely different. What you can compare across vendors is measured results: run both vendors' 'high' on the same task set with the same scoring, and look at the gap in output. The baseline for a migration works the same way — the measured results on that same evaluation set. When you log a request, store the actual field name, the model version, and the usage together; only then do you have a comparison baseline after a vendor switch."

"How does the budget show up in the request?"

"At least three layers: request parameters (effort or a thinking budget), limits (maximum output, maximum turns), and the final usage receipt," Lao Z said. "The first two are configurable before the request goes out; the third is known only when the response comes back. Keep the configured budget and the actual consumption in separate books. When they don't reconcile, degradation, retries, or cache hits are usually at work — and that gives you a lead."

"Should budget configuration live in code or in the prompt?"

"In code," Lao Z said. "A budget is a commitment about a resource ceiling, and that belongs to the host's control. Put it in the prompt and you let the model decide its own spending ceiling. Keep budget fields in the request-assembly layer; the prompt expresses task intent only. Keep the two layered, and problems are easier to locate."

"Should thinking intensity be bound to the task type?"

"You can bind it, but leave room for exceptions," Lao Z said. "Route by task type to a default level, for instance: raise it for complex tasks, lower it for simple ones. But when verification fails at the default level, allow the next turn to retry at a higher one. What you bind is the starting point, not the ceiling: a fixed default plus a recorded retry is steadier than one hard-coded level."

"And if a vendor only offers three levels — low, medium, high — how do you pick the default?"

"Run the three levels over an offline task set and see which level lands on the intersection of cost and correctness," Lao Z said. "A level is only the entry point; the real basis is the measured distribution on your own task set."

Summary

Chain-of-thought makes the model think before it answers: it turns implicit intermediate reasoning into explicit text steps, so a multi-constraint task is handled piece by piece instead of concluded in one breath. The value of the method is that the reasoning process becomes auditable — you can see which assumption the model made at which step, and where it went down a side path. Parameters like reasoning intensity or reasoning budget control how much "thinking resource" the model is willing to spend. Raising one can genuinely help the model handle harder problems, but it can equally be slower and dearer without being more accurate; a budget buys an opportunity, and correctness still has to be guaranteed by verification on top.

Two things need separating here. First, the intermediate process is only working material: the steps the model "thinks" are still probabilistically generated, so it can reason smoothly and still be wrong. The reasoning process therefore needs external verification and cannot be treated as a verified conclusion. Second, the stop reason is an input to control flow: the model ended because of stop, was cut off by length, stopped to request a tool_use, or failed with error — each calls for a completely different next step. There is also a trap specific to Chinese: one word covers both inference (predicting with fixed parameters) and reasoning (the intermediate computation process). In English these are different words, and this chapter is entirely about the second.

There are three ways to leave a scratch paper — explicit prompting, structured decomposition, and the reasoning-model budget. They form an engineering taxonomy, and each has its typical failure: explicit prompting may make output longer without adding checkable intermediate values; structured decomposition may lose information between stages; a reasoning budget is constrained by each vendor's field semantics, so accounts don't reconcile across vendors. When you mix them, watch who is in charge within the same turn, so two sets of instructions don't fight. The budget follows task risk and needs a total budget behind it: too much invites overthinking, too little gets cut off at the last step. A truncated intermediate artifact must keep its incomplete state and must not be stored together with a complete answer.

In the end the stop reason lands in one dispatcher in the host: each reason maps to one branch, and whether to continue, repair, degrade, or exit is the host's call based on the task context — the stop reason itself carries no verdict. How strong the verification is decides how high automation may run: deterministic checking, evidence verification, and model self-assessment weaken in that order, and only tasks whose results admit deterministic checking suit high automation. Keep the budget's books separate too — the configured budget, the limits, and the actual usage receipt are three layers of information, and cross-vendor comparison rests on measured results over one shared task set. Store the draft, the verification, the stop reason, and the actual consumption each in its place, and a CoT flow becomes something you can both see into and settle accounts on.

Hands-on Verification

Pick a small task with a definite answer, and run it twice: once asking for a direct answer, once asking for the assumptions to be listed first. Write down the scoring rules beforehand — for example whether tests pass, whether the calculation is correct, whether citations exist. Record output length, elapsed time, and score, rather than choosing a prompt on subjective impression.