Chapter 6: From "Can Say" to "Can Do" — Function Calling
Xiao A built a demo feature: he asked the model to "read the config file for me." The interface neatly showed "the model has read config.ts," but when he flipped through the logs he froze — there was no record of any file being read.
"Didn't it say it read it?" he asked Lao Z.
"It only 'said' it did," Lao Z said. "Front-loaded rules can constrain how a model expresses itself, but they can't make text produce an external effect. The 'I have read the file' a model writes is just a piece of text, not a file access."
"So to make it actually read a file, what do I have to do?"
"That's where Function Calling comes in: it separates 'what the model says' from 'what the program does'," Lao Z said.
This chapter explains how Function Calling separates a model's request from actual execution. It does not discuss the permission design of any specific tool.
Align the names first
| Term | English / abbreviation | Working definition in this book | What it is not |
|---|---|---|---|
| function calling / tool use | function calling / tool use | the interface capability by which a model outputs a structured call request | the host has already run the action |
| tool call | tool call | an unexecuted request carrying a name, arguments, and a call ID | a verified user intention |
| JSON Schema | JSON Schema | constraints describing the data shape of arguments | permissions, path safety, or business authorization |
| tool result | tool result | the structured state an executor returns against a call ID | the model claiming it finished |
| parallel calls | parallel calls | the protocol possibility of several call requests in one round | safe to run concurrently |
6.1 A Model Request Is Not a Program Action
Xiao A read the glossary from top to bottom twice, then pointed at the first row and asked Lao Z: "The act of 'reading a file' — whose job is it actually?"
"Think of the model as the person ordering the food; the program is the kitchen," Lao Z said. "The person ordering writes out an order — 'read_file, argument path=src/config.ts' — but an order is not the dish arriving at the table. Function Calling (also often called tool calling) is a protocol capability: in its response the model returns a structured call request that states what it wants to call and with what arguments, and whether the call runs is up to the host program. The names, the fields, and whether parallel calls are supported all vary by API; this one boundary stays the same: the model outputs only a request."
// Illustration: a request that has not run
{ "name": "read_file", "arguments": { "path": "src/config.ts" } }"So this JSON by itself doesn't read the file?"
"Right. This JSON neither reads the file nor proves the path is allowed to be read. The authority to execute always lies with the host program."
"Why does the model sometimes act as if it 'has already done it'?"
"Because generating text means continuing from the context. 'I have read the file' is the same kind of thing as it continuing into a snippet of code or an API name," Lao Z said. "Without function calling, the model can only describe in natural language what it wants to do. With function calling, that 'what I want to do' is written as a structured request."
"And if I don't use it, and force myself to parse 'read file' out of natural language?"
"Try it and you'll see," Lao Z said. "The model might write 'please read config.ts,' or 'could you take a look at the config file,' or bury it inside some other sentence. You'd have to parse out the action, extract the arguments, and also guard against it inventing its own steps. Guessing an action out of text means treating a parser as another model. What function calling does is let the model express 'what I want to call, with what arguments' directly at the protocol layer, so an application can parse it, validate it, and reject it — turning that step from 'parsing prose' into 'reading structured fields.'"
"And the arguments it gives — surely those are tidy, at least?"
"Tidy, yes," Lao Z said. "But the trustworthiness is unchanged: the text it produces is just as unreliable as before. Structure only changes how you process it."
Metaphor: ordering a dish
The model is the person ordering; the program is the kitchen. However neatly the order is written, the kitchen still has to actually cook the dish; the model "saying" it read a file is not the file having been read.
| Aspect | Natural-language description | Structured call request |
|---|---|---|
| How the model expresses it | uses a sentence to "say what it wants to do" | uses fields to "request what to call" |
| How the app extracts info | parses prose, easy to get wrong | reads structured fields, validatable |
| Rejectability | whether to reject rests on human judgment | the host can reject outright and give a reason |
| Execution guarantee | none | still none: the protocol does not run anything |
6.2 The Loop from Declaration to Result
Having got the metaphor, Xiao A asked about the flow: "One complete 'order to serving' — how many steps?"
"A full round has at least four steps," Lao Z said:
- The application declares to the model the callable capabilities and their argument schemas;
- the model returns one or more call requests;
- the application validates each request and then executes it, or refuses to execute it;
- the application associates the structured result back to the original call, and lets the model decide the next step.
Lao Z drew the four steps as one diagram:
① declare tools + schema ──────> ② model returns call requests (id, name, args)
^ │
│ ↓
④ result back to model by id <── ③ validate → execute / reject
│
structured result"What about the declaration step? How does the model know which tools exist?"
"The application attaches a tool list in the request — usually names, descriptions, and argument schemas," Lao Z said. "The model reads that list, weighs it against the user's task, and decides whether to call, which one to call, and what arguments to fill. If the declaration omits a tool you actually need, the model can only improvise with another one, or tell you in natural language that it can't do it."
"And if the descriptions are written badly?"
"The model will pick the wrong tool or pass wrong arguments — that's the problem Chapter 8: Fitting Handles onto Capabilities — Tool expands on," Lao Z said. "This chapter follows only the closed loop of the call protocol itself."
"What if one step is missing?"
"Look at step ④ first. If the result isn't returned, the model has only the front half of the information: it knows 'what I wanted to do' but never learns 'whether it worked.' It's like calling someone up and nobody answers — the other side never hears your next line," Lao Z said. "And if the validation in step three is skipped, the model has stepped past the permission boundary. Only when the loop closes and the result goes back can the model enter its next round of decisions."
"Back to step ③ — what exactly does validation check?"
"At least two layers: whether the argument shape is right, and whether this call should be allowed at all," Lao Z said. "If the shape is wrong, reject it outright and say why. Whether to allow it depends on the host policy — section 6.3 covers that layer in detail. What validation must do is stop errors before they land."
Key point
The full loop = declare → request → validate & execute → return the result. No return, and the model is "blind." No validation, and the model "oversteps."
6.3 Schema Constrains Shape, It Does Not Guarantee Intent
Xiao A circled the JSON Schema row in the glossary and breathed out: "With this constraint, won't the arguments simply stop being wrong?"
"No. An argument schema can require path to be a string, line an integer, and mode a member of an enum," Lao Z said. "That reduces format errors and helps you generate UI and validation messages. But a schema cannot prove that path lies inside the workspace, that a request matches the user's purpose, or that a command is safe. Those get checked after the schema, as business and security validation."
model arguments → schema validation → permission / path / policy validation → execute → normalized result"It'll still pass wrong values, won't it?"
"It will. A missing required field, a wrong type, an out-of-range enum value, even a wholly invalid chunk of JSON, or simply the wrong tool chosen — all count," Lao Z said. "An executor treats these as normal error paths: it returns the validation failure, the reason for it, and the shape it expected to the model, lets the model correct its arguments and try once more, and the task still moves on."
"Is there an example of schema-valid but intent-wrong?"
"There is. The schema requires path to be a string; the model fills in path: "/etc/passwd" — valid format, out-of-bounds path," Lao Z said. "Or the schema requires recursive: boolean; the model fills in true — valid format, but semantically you may not want a recursive delete. A schema is a shape constraint. Intent is judged by the permission layer and the business layer, and those two layers cannot be merged into one."
| Validation layer | What it checks | What it can't catch |
|---|---|---|
| schema validation | field types, required fields, enum values | out-of-bounds paths, dangerous commands, wrong intent |
| permission / path validation | whether inside the workspace, whether a write is allowed | whether the business meaning is correct |
| business / approval | whether it matches the user's intent for this run | — (the last line of defense) |
Xiao A scanned the table top to bottom: "These three layers run in series, and not one of them can be cut?"
"Right. Whenever one layer passes something through, the next layer has to catch it itself," Lao Z said. "Schema passing doesn't mean the path is safe; a safe path doesn't mean the intent is right. Cut any layer and you hand that layer's risk to the layer below — or to the incident."
"What usually causes these errors?"
"A missing field is often because the task description didn't give enough information; a wrong type is the model filling in a 'string' as an array; a wrong tool is the description in the declaration being unclear," Lao Z said. "All three causes are worth feeding back to the model, so its next call has a better basis."
| Common error | Example | What goes back to the model |
|---|---|---|
| missing required field | path absent | say which field is missing and ask for it |
| wrong type | path given as an array | state the expected type and why this one is invalid |
| out-of-range enum | mode given a value outside the list | list the allowed enum values |
| wrong tool chosen | read_file on a config that doesn't exist | explain why the call failed and allow switching tools |
"Are all four of these schema-layer errors?"
"The first three are; the fourth is more of a tool-selection problem," Lao Z said. "But you handle them the same way: tell the model, in structured form, 'this call doesn't hold, and why.' Don't leave it facing an empty result and guessing on its own."
"And the schema declaration itself — is it better written in detail or kept rough?"
"Enough detail to constrain the shape, but not so much that it turns into prose," Lao Z said. "A schema acts as a guardrail: state required fields, types, and enums clearly, and the model fills accordingly. The business background is for the tool's developer — you don't pile it into the description."
6.4 The Boundaries of Streaming and Parallelism
Streaming's turn. Xiao A worried first for the kitchen: "Won't streaming output tear the order to pieces?"
"It will. In a streaming protocol, the arguments may arrive across several chunks," Lao Z said. "Until you receive the completion marker, any partial JSON is good only for a preview or a progress display, not as grounds for execution."
"What does something mid-stream look like?"
"For instance, the arguments field arrives half-way as { "path": "src/con — you can tell it wants to read some file under src/, but you don't know the full path," Lao Z said. "Run that half and either it errors or you read the wrong file. The right move is to wait for the completion event, assemble the full arguments, and only then do schema validation. Streaming is for showing progress in the UI."
Lao Z added a second streaming sketch:
time → t1 t2 t3 t4
{ "name": "read_ file", (completion event)
{ "path": "src/co nfig.ts" } ← parseable only now
└───────────────────────────────────┘
any frame before this is just an unfinished fragment"At any frame t1, t2, or t3 you hold incomplete arguments," Lao Z said. "Some implementations use an incremental delta field to say 'here's another piece of this,' others just hand you slices; the field names vary by API, but it's the same everywhere: only the content after the completion event is eligible for validation and execution."
"That 'parallel calls' row in the glossary — can one round really come back with several requests?"
"Yes — some services allow several calls in the same response," Lao Z said. "But when the model writes it that way, it's only saying 'I want all of these.' Who runs first, and whether they can run at the same time, is for the host to decide from resource dependencies; how many calls the model asked for in one round says nothing about whether parallelism is safe. Two reads of different files can run in parallel; one read and one write of the same file have to run in sequence; writes to the same file, the same database record, or the same external order likewise have to run in sequence or take a lock."
"How exactly do you sort the resource relationships?"
"Look at three kinds," Lao Z said. "Read–read has no conflict: two read-only calls are the safest thing to parallelize. Write–write conflicts: two calls that write the same resource run in sequence or take a lock. Read–write dependency: one call's input comes from another call's output, so the second must wait for the first. Parallelism is an optimization; sequence is the baseline. First make sure there's no conflict, then talk about speed."
| Resource relation | Example | How to run |
|---|---|---|
| read–read | read A, read B | can parallelize |
| write–write | both write the same file | in sequence, or with a lock |
| read–write dependency | read the config, then create based on it | the latter waits for the former |
"What extra cost does parallelism bring?" Xiao A asked.
"The cost of reconciling results," Lao Z said. "After parallel execution, the results come back as a batch. Without call IDs, you can't tell which result answers which request. Even with IDs, you still have to handle partial success: three calls, one fails, two succeed — whether the rest of the task proceeds, and how, is for the host to decide. Parallelism saves time and leaves the ordering and reconciliation cost to the host."
6.5 Failure, Retry, and Idempotency
Having covered parallelism, Xiao A turned to the other end: "What if execution fails?"
"A tool result has to distinguish success, recoverable failure, rejection, timeout, and cancellation, and give the model enough machine-readable information," Lao Z said. "Retrying a model call or a stream is not naturally idempotent. If a tool sends a message, makes a payment, deletes, or writes, a repeat run can produce a duplicate side effect. That's why you need call IDs, idempotency keys, execution records, or human approval — you can't rely only on telling the model 'don't repeat.'"
"When a failure is returned to the model, how does it handle it?"
"It depends on the failure type," Lao Z said. "A recoverable failure (a file temporarily locked, say), the model may wait and retry. A rejection (an out-of-bounds path, say), the model may change the path or give up. A timeout (you don't know whether the remote took effect), the model must not retry blindly — it has to check the execution record or ask for human confirmation."
| Failure type | Meaning | Reasonable next step for the model |
|---|---|---|
| recoverable failure | a transient fault (a lock, a rate limit) | wait, then retry |
| rejection | permissions or the path don't allow it | change the arguments, or give up |
| timeout | the remote state is unknown | check records, use an idempotency key, or escalate to a human |
| cancelled | an external signal aborted it | stop the current round |
Xiao A thought of another case: "What counts as a 'silent success'?"
"That's the worst," Lao Z said. "A tool execution fails but nothing is returned, or it's returned as 'success.' The model then proceeds on wrong information and produces results that look plausible but are actually wrong. Failure must be explicit, classifiable, and carry context."
Xiao A asked again: "At minimum, what should a failure result carry?"
"Three pieces, none droppable: a status code or status name, the failure type, and a machine-readable reason," Lao Z said. "Just the word 'failed' leaves the model unable to tell whether to retry, change arguments, or give up. And a reason alone isn't enough either — without the status, the program can't branch automatically. What you give the model must let it tell apart 'this step can still be salvaged' from 'this step is hopeless.'"
"And retries? Trying a few more times can't hurt, surely — it's just reading a file," Xiao A guessed.
"A file read really can't be broken by repeating; ten tries read the same thing," Lao Z said. "But send one message or make one charge, and one extra retry is one extra side effect. Whether a retry is safe depends on whether the operation can be run repeatedly. So the rule is: a reentrant call — no problem retrying after a failure; a non-reentrant call — a retry must carry an idempotency key, or first check the execution record before you decide."
| Safeguard | What it prevents | When to use it |
|---|---|---|
| call ID | results that don't match requests | parallel, streaming, multi-round calls |
| idempotency key | a repeat run producing a duplicate side effect | remote create, payment, sending a message |
| execution record | not being able to tell whether the last run took effect | timeout, disconnect, recovery |
| human approval | an irreversible action being blindly replayed | deletion, publishing, a large payment |
Xiao A wrote the term "idempotency key" in his notebook: "What does that key look like?"
"It's a unique identifier for this operation, generated by the caller, and the remote remembers 'this key has already been handled'," Lao Z said. "Carry the same key on a retry, and when the remote sees the duplicate it returns the first result directly, without running it again. The decision logic lives on the remote; all the caller has to do is two things: guarantee the key is unique, and keep it unchanged on a retry."
"If there's no idempotency key, what's the safest alternative?"
"Check the execution record," Lao Z said. "After a timeout, first check 'did this call actually take effect?' If it did, leave it alone; only if it didn't, retry. If you can't find a record, escalate to human confirmation. All three measures turn 'unknown' into 'known' — they just cost different amounts."
"And the retry count and interval — who decides?"
"The host sets the ceiling. The model may take part in deciding whether to retry, but the count and the interval are policy," Lao Z said. "The model says 'try once more'; the host checks the retry budget — how many left, how long since the last — and only lets it through when there's room. Letting the model retry without limit means handing cost control to the unreliable side."
6.6 Call Correlation and Compensation
Back to the reconciliation mess from parallelism, Xiao A asked Lao Z: "When two tools run at the same time, how do you keep the results from crossing wires?"
"Every call should have a stable call ID; the tool result must carry the same ID back, so that in parallel you never hand A's result to B," Lao Z said. "An execution record distinguishes at least 'requested, approved, running, succeeded, failed, rejected, cancelled.' That is the basis for both recovery and audit."
Lao Z ran one call from start to finish along a single line:
model request(id) → validate → approve → execute → result(id, status) → write back
└────── reject / timeout ───────────────────────┘"Why does the call ID matter so much?"
"Because of parallelism," Lao Z said. "In one round the model requested three calls: read A, read B, write C. When the three results come back, without ID correlation you might mistake C's write failure for A's read failure — so the model goes off to fix A's path while the real trouble, C, is left for nobody. The call ID is the wiring between result and request; lose that wire and parallelism turns to soup."
"Is the ID generated by the model, or by the host?"
"Usually the model carries it in the request and the host echoes it back verbatim in the result," Lao Z said. "The exact field name varies by API, but the requirement — that request and result can be paired — never changes. If the model doesn't carry an ID, the host has to allocate one itself and maintain the mapping. Either way, somebody must be responsible for the pairing; don't assume the results line up on their own."
The more Xiao A listened, the more he felt this wire couldn't be lost: "What happens when pairing fails?"
"You get crossed wires," Lao Z said. "Take two parallel calls, A and B. If B's result is written back labelled as A's, the model thinks 'reading A succeeded' when it actually got B's contents. The error doesn't necessarily surface at once — it takes effect quietly inside the model's next decision. An ID-pairing failure is more insidious than a result failure, because it looks like success."
"In the execution record, what action does each state correspond to?"
"Only success lets you keep going; a failure must be written back with a reason; a rejection must state why it was rejected; a cancellation must stop the current round," Lao Z said. "The coarser you split the states, the more you have to guess during recovery."
"And how do these states relate to auditing?" Xiao A asked.
"Audit is the complete replay of the states," Lao Z said. "Without states, there's nothing to audit. When something goes wrong, all you can do is rummage through memory to reconstruct events — and memory, that log, is the least reliable one. So each call records its ID, who initiated it, when, a summary of the arguments, and the state transitions. Then when something goes wrong you can answer 'who initiated this action, when, through how many retries, and where it finally stopped.'"
"What's an argument summary? Do you store the full arguments?"
"Store a summary, not the sensitive content," Lao Z said. "Paths and commands — the kind of thing that helps you troubleshoot — you can store. Keys and tokens — once they hit the database they become a new leak surface. What goes in the log depends on two things: whether you can retrieve it when it's gone, and whether storing it causes trouble."
Illustration: after a network timeout on
create_issue, the caller doesn't know whether the remote has already created it. Retrying directly may yield two Issues. Safer designs are to use an idempotency key the service supports, query a correlatable record, or hand it to a human. Not every side effect can be compensated — especially sending a notification, making a payment, or deleting data.
6.7 A Call Is a Small Transaction, Not a JSON Fragment
Having heard the whole chapter, Xiao A wanted first a master diagram: "Can all of this be drawn as a flowchart?"
"It can, and it should," Lao Z said, and pulled a main line across the paper:
declare ToolSpec → model tool call(id) → schema validation
→ policy / authorization validation → execute or reject → tool result(id, status) → write into the next round"Apart from the happy path, what's the easiest trap to fall into?"
"Two counter-examples," Lao Z said. "Counter-example one: the arguments satisfy the schema but point at a file outside the workspace — the execution boundary must reject it. Counter-example two: a remote create times out, and the caller doesn't know whether it took effect; retrying directly duplicates the side effect, so you must check the execution record, use the service's idempotency key, or ask for human confirmation."
"Are these fields industry standards?"
"No. Each vendor's tool-calling docs describe only its own interface's events and result returns: OpenAI Chat Completions puts the calls in tool_calls and returns results as role: "tool" messages; Anthropic Messages starts a call with a tool_use content block and returns it with a tool_result content block. There is no unified standard for the specific fields," Lao Z said. "This book's engineering choice is to model schema, permissions, concurrency, and results separately. It's testable and auditable; the price is that you must maintain stable IDs, a state machine, and a compensation / confirmation path."
"Does that trade-off get drawn into the diagram too?"
"It can." This time Lao Z marked the side branches as well:
┌────────────────────── the transaction boundary of one complete call ──────────────────────┐
│ │
user task → declare → request(id) → schema → policy → execute → result(id) → write back
│ │ │
shape error deny timeout
│ │ │
return reason return reason check records / idempotency key
│
unconfirmed → human confirmation
└───────────────────────────────────────────────────────────────────────────────────────────┘"Every cell in this diagram is testable," Lao Z said. "Shape validation, policy validation, execution, write-back — each layer has fixed inputs and outputs, so you can write a test for it alone. Design a call as a small transaction, and tests can cover it, recovery has a basis, and audit can trace it."
"In a real project, does this whole flow have to run for every call?"
"It depends on the call type," Lao Z said. "A read-only, purely local call whose failure has no side effect can take the lean path; it's the calls that write outside, send messages, or make charges that are worth laying out the full transaction state machine."
Xiao A grabbed the last thing he still felt unsure about: "Between 'the model says it's done' and 'the executor returns success' — where exactly is the difference?"
"The first is language; the second is fact," Lao Z said. "Structured calls make this uncertainty visible, but they don't dissolve it automatically. The model saying 'I've created the Issue' is just a sentence; the executor returning status: success + issue_id is evidence you can take and check against."
Summary
Function Calling carries the model from "can say" to "can do," and it does so by teaching it to file a request: output a structured request carrying a tool name, arguments, and a call ID, while the one that acts is still the host program. The request itself executes nothing; the authority to execute always rests with the host program, and this distinction is the foundation of the whole reliability chain. Structure makes a request parseable, validatable, and rejectable, but the trustworthiness of the content does not change for it; guessing an action out of natural language means treating a parser as another model. A complete tool round-trip has four steps: declare the tools so the model knows what's available, validate the arguments the model gives, execute the calls that pass validation, and return the results to the model by call ID. If the loop never closes, the model is left only with "what I want to do" and never learns "whether it worked." JSON Schema describes the shape of the arguments; whether the arguments match the user's intent is guarded by runtime validation after the schema. A model may produce arguments that are schema-valid but semantically wrong, so this validation cannot be dropped.
Validation itself has three layers: schema owns shape, the permission layer owns scope, the business layer owns intent. Whenever any layer passes something through, the layer below must catch it itself. A validation failure isn't a task failure either; it's only the model getting one chance to correct itself — feeding the reason back to the model in structured form is worth far more than leaving it facing an empty result and guessing. The real risk hides in the result-return step: a tool execution failing is itself a result. Return the failure clearly to the model, and the model gets a chance to adjust its strategy, retry, or give up. Swallow the failure silently, and the model keeps fabricating on missing information. Failure must be explicit, classifiable, and carry context: a recoverable failure waits for a retry, a rejection changes the arguments, a timeout must not retry blindly but checks records or uses an idempotency key. What you give the model must let it tell apart "this step can still be salvaged" from "this step is hopeless."
One round may produce several parallel call requests, but "parallel requests" is not "parallel execution." Whether it's truly parallel is decided by the host's execution policy; the safety of parallelism comes from resource analysis, not from how many calls the model asked for at once. The call ID is the wiring between request and result; lose that wire and parallelism turns to soup. A pairing failure is more insidious than a result failure, because it looks like success. Stringing all this together, one tool call should be designed as a small transaction: a stable call ID, a well-defined set of states, a compensable failure path, and a strict distinction between "the model says it's done" and "the executor returns success."
The thickness of the safeguards follows the size of the side effect: read-only, side-effect-free calls take the lean path; it's the calls that write outside, send messages, or make charges that are worth laying out the full state machine. Skipping any one of the four steps — validation above all — produces side effects you can't trace. Once tools multiply and actions start touching external systems, "how to make an action actually land" stops being a protocol question and becomes a question of the tool's own interface design.
Hands-on Verification
Implement a read-only read_file mock executor. Feed it, in turn, a call request with a missing path, a request with an out-of-workspace path, and a request with a valid path, and confirm each result carries a different status and that no unauthorized file gets read. Then simulate an unfinished chunk of JSON and confirm it never reaches the executor.