Chapter 4: How the Translation Bureau Connects to Models — Provider
The team decided to switch its model service from Vendor A to Vendor B. At kickoff the product manager thumped his chest: "Aren't all the APIs basically the same? It's a one-line config change." The task landed on Xiao A.
He agreed at once, turned around, and went to read the docs. Three days later he showed up at Lao Z's desk with two dark circles under his eyes and two API documents spread out in front of him.
"I was wrong," he said. "Vendor A takes messages as an array; Vendor B wants system passed separately. Vendor A's streaming is data: line by line; Vendor B's is event: plus data:. Even authentication differs — one uses Bearer, the other x-api-key. That isn't changing one line. That is rewriting the whole thing."
"Are you sure you're switching only the vendor, and not the business logic?" Lao Z asked.
Xiao A's voice trailed off. "But aren't the interfaces the same? You send messages, you get replies..."
"That 'basically the same' is exactly the problem," Lao Z said. "Once you actually wire both up, you find authentication, model names, streaming events, tool formats, rate limits, and error codes are all different. What this chapter solves is precisely that: how to block the differences without pretending they don't exist. We are not ranking specific products. We discuss the Provider abstraction itself."
"But where do the differences come from? Why doesn't anyone just standardize?" Xiao A pressed.
"They come from each vendor growing inside its own protocol, model architecture, and product line," Lao Z said. "Vendor A designed for chat early on, so messages are an array and the role field hangs off each one. Vendor B entered from completion, so system is its own field. The streaming events, error codes, and rate-limit headers are all products of separate histories. No vendor designed against a cross-vendor standard from scratch. Expecting vendors to converge on one protocol is like expecting history to run backwards so someone can rewrite it."
Align the names first
| Term | English / abbreviation | Working definition in this book | What it is not |
|---|---|---|---|
| Provider | Provider | The service boundary for authentication, the model catalog, and request behaviour | a single HTTP SDK |
| adapter | adapter | The conversion implementation between the unified contract and a vendor's protocol | erasing every capability difference |
| unified message | unified message | The business representation of roles, content blocks, calls, and results | the union of all native fields |
| streaming | streaming | Incremental events with a lifecycle | just appending strings forever |
| idempotency | idempotency | The property that repeating an operation does not further change the target state | "retries are usually fine" |
4.1 The object of abstraction is difference
Xiao A spread the two documents out and found more differences than he had imagined. Even "what the model is called" differed. He asked Lao Z: "So does the Provider layer have to smooth all of this away?"
"Quite the opposite," Lao Z said. "Different model services differ in authentication, model identity, request fields, streaming events, tool formats, rate limits, and error encoding. The Provider layer's job is not to pretend those differences don't exist. It is to define the shared semantics the business needs as a stable interface, and to expose the capabilities that cannot be unified explicitly."
"Wait — which differences really can't be unified?" Xiao A asked.
"The ones tied to model capability itself," Lao Z said. "If Vendor A accepts image input and Vendor B doesn't, that is not a field-format problem. It is a model-architecture problem, and no clever Provider layer can conjure the capability. But 'how an image is transported' can be unified into one field — except that when Vendor B receives that field, it has to return 'not supported' instead of quietly dropping it. Field formats are the surface; capability differences are the substrate. A Provider can smooth the surface; it cannot fill the substrate."
Metaphor: the translation bureau
A translator does not pretend every language is the same. They render what you mean accurately, and when something cannot be rendered they tell you "this cannot be translated literally." A Provider is to vendors what a translation bureau is to countries.
// Illustration: the business layer depends on its own stable request and event types
type ModelClient = {
stream(request: UnifiedRequest, signal?: AbortSignal): AsyncIterable<UnifiedEvent>
}This is only an illustration, not the real interface of any SDK.
4.2 Unified messages and capability negotiation
Xiao A decided to define one unified message format for both vendors first. He listed fields for half a day, then took the result to Lao Z. Lao Z read it once and asked: "What about images? Vendor A supports images and Vendor B doesn't. What are you going to do?"
Xiao A froze.
"A unified message should express roles, text and content blocks, tool calls, tool results, and the correlation IDs you need," Lao Z said. "Images, audio, reasoning controls, and structured output are not supported by every service. The interface has to be able to declare capabilities rather than quietly drop content."
"Then does a unified interface mean you never touch code when you switch services?" Xiao A asked.
"No. Abstraction lowers migration cost, but different semantics still show through," Lao Z said. "If a service supports neither images nor parallel tool calls, the caller has to choose: degrade, refuse, or switch to a compatible model. Turning 'not supported' into a visible error is more reliable than producing a request that looks successful but is missing content."
"Degrade or refuse — how do you choose?" Xiao A pressed.
"Look at whether the task can tolerate degradation," Lao Z said. "If the task is 'describe this image', degrading to 'read the alt text only' may be enough when image capability is missing. If the task is 'find the defect in the image', degrading is the same as failing the task, so refuse outright and suggest another model. Degrade visibly: the caller must know 'this result is a degraded one', instead of being left to assume it got the full capability."
"How does capability negotiation actually land? I'd guess you read the vendor docs before connecting?"
"Docs work as a first filter only. Declaration and confirmation usually happen at initialization," Lao Z said. "At connection setup or on the first request, the Provider queries which capabilities the target service supports — image, parallel calls, structured output, and the rest — or takes what is already known, and records them as a capability list. Upper layers decide what to send from that list. Remember: the capability list is runtime fact, not a promise in the docs. Docs saying 'supported' does not mean supported for your account, your version, and your current quota."
"Which layer maintains that list?"
"The Provider itself," Lao Z said. "It caches a capability list at initialization, and upper layers read that cache to decide how to send requests. But the list can go stale: quota runs out, a model gets retired, and the runtime fact changes. The steadier approach is to re-confirm the capabilities that matter right before the request, instead of trusting the startup snapshot for the whole run."
Lao Z laid the differences that can be smoothed and those that cannot out in a table:
| Capability difference | Smoothable? | Handling |
|---|---|---|
| Field naming (is system in the array or its own field) | Yes | adapter translates |
| Error code naming (401 vs invalid_api_key) | Yes | adapter maps |
| Whether the model accepts image input | No | declared in the capability list; report an error when unsupported |
| Parallel tool-call ceiling | No | declared in the capability list; report an error when exceeded |
| Rate-limit policy (per token or per RPM) | Partly | adapter maps to readable fields; the policy itself still differs per vendor |
"Anything the list doesn't mention, I'll assume is supported and catch it if it breaks?"
"The opposite. Anything unstated counts as unsupported," Lao Z said. "Vendor capability docs are usually incomplete. For a capability that never appears on the list, the safest assumption is that it is unsupported: put it in the request and you get an error or a silent drop. Treating undeclared capability as unsupported is safer than treating it as supported — the former costs at most one extra confirmation, the latter may send the request out only to reveal a truncated result. Once testing confirms the capability, record it on the list."
4.3 Streaming must keep its boundaries
Halfway through the migration, Xiao A got stuck again while integration-testing streaming output. He found Vendor B emits tool arguments in several chunks, and the first two do not form a complete JSON.
"Is streaming just receiving strings nonstop?" he asked Lao Z.
"Not that simple," Lao Z said. "The upper layer has to distinguish at least text deltas, tool-argument deltas, usage, completion, and error events. Tool arguments in a stream may be temporarily incomplete JSON; only after the completion event can they serve as executable input."
"Why can't tool arguments execute as they arrive?" Xiao A pressed.
"Because they may need several chunks before they are complete," Lao Z said. "Vendor B can split one tool call's arguments into {"path":"src/, core/ut, and ils.ts"} arriving as three chunks. At the first chunk the string isn't even closed, so you cannot parse it and cannot validate it. Try to execute at that point and you get a syntax error, or a truncated path that goes to read a file that doesn't exist. Text deltas are for people to read — a truncated one still displays. Tool arguments are for machines to execute — a truncated one is bad input."
request → auth and send → provider events
↓
Provider parse / map
↓
text delta | tool delta | usage | completed | error"So I can just append text deltas straight to the UI?"
"You can, and usually should," Lao Z said. "The word-by-word effect a user sees is exactly text deltas being appended in real time."
"What about usage and completion events?"
"Usage usually arrives at the end and carries the token counts for that turn," Lao Z said. "The completion event is the signal that 'this stream is over': it seals the final result and attaches the stop reason. Without a completion event, everything in your hands is intermediate state. Even if you have received a lot of text, you cannot audit or persist it as a complete model response."
"What if the stream breaks halfway and usage never arrives?"
"Then there is no usage for that turn. Leave the record blank instead of filling in an estimate," Lao Z said. "When a stream breaks, the model may already have emitted part of the answer, but the server never got to send the usage event. That account can only be settled against what you actually received. A missing usage is better than a wrong one: put an estimate into the cost report and it will never match the real bill, which makes cost debugging harder, not easier."
"What if the user stops wanting to wait?" Xiao A asked.
"The application must support cancellation: when the caller cancels, propagate an AbortSignal or an equivalent mechanism. The Provider is responsible for closing the transport and marking the partial result it already has as aborted, not as completed."
"And the tool arguments already emitted — do those execute too?"
"Discard them. Do not execute," Lao Z said. "The cancellation semantics are 'this turn is void'. Even if the tool arguments are fully collected, as long as no completion event arrived, or the cancel signal arrived first, they must not go to the executor. The nastiest case is when the cancel signal and the completion event arrive almost simultaneously. Then you need an explicit priority. Usually cancel wins: mark the stream aborted rather than completed, so half-finished arguments never leave as a complete call."
"Also, can events arrive out of order — the completion event first, with text deltas still in flight?"
"Event order is a contract too; each vendor just signs a slightly different one," Lao Z said. "Deltas normally arrive before the completion event, but vendors chunk very differently: some split one stretch of text into many small pieces delivered one by one, others send one big block at a time. An adapter must not assume 'one delta is exactly one character', and must not assume deltas arrive at a fixed granularity, and the UI must not compute progress from the number of deltas. Completed, failed, and aborted are all terminal states. Receive any one of them and the stream is over; after that you should expect no more deltas. If the UI is still waiting for 'the last character', the application's state machine isn't treating a terminal state as terminal — it isn't that the model hasn't finished."
4.4 Authentication, timeouts, and retries
At eleven that night, Xiao A found in the logs that Vendor B had printed its error message verbatim, and there in plain sight was an API key. He broke out in a cold sweat, deleted the logs overnight, and went to Lao Z first thing the next morning.
"Where do keys like that belong?" he asked.
"Keys, OAuth credentials, and endpoint configuration are the Provider's or its runtime environment's responsibility. They must never be interpolated into user messages or logs," Lao Z said. "Set a timeout on every request, and separate retryable transient failures — some network errors, a temporarily unavailable server — from non-retryable configuration, authentication, and argument errors."
"Then what can a log actually record?" Xiao A asked.
"Record the model name, the endpoint hostname, a request summary, the response status code, and the duration. That is enough to debug the problem and leaks no credentials," Lao Z said. "Never print request headers or raw response bodies verbatim, because an error message may echo the authentication field you sent. Pass logs through redaction first — replace fields like Authorization and x-api-key with placeholders — and only then write them to disk. Otherwise you turn one local debugging session into a bigger leak."
Lao Z split the failures worth resending from the ones where resending does nothing:
| Failure type | Usually retry? | Reason |
|---|---|---|
| Network jitter, connection reset | Yes (bounded backoff) | Transient; resending most likely recovers |
| Temporary server 5xx unavailability | Yes (bounded backoff) | It may be restarting |
| 429 rate limit | Yes (read Retry-After) | Waiting usually restores service |
| 401 authentication failure | No | Resending only repeats the failure |
| 400 argument error | No | The input is wrong; resending fixes nothing |
| 404 invalid model name | No | A configuration error; resending is useless |
"Then 'just resend when a request fails' can't be wrong, can it?" Xiao A said, taking it for granted.
"There is a boundary people often miss," Lao Z said, his tone turning serious. "A generation request is usually not free, and not naturally idempotent. Even with identical input it may produce different output and bill additional usage. If a request triggers an external action, the upper layer must constrain that action with a request ID, an approval, or an idempotency key. You cannot treat 'retry the model request' as a safe replay."
"Will the Provider handle idempotency for me?"
"No. That is the caller's responsibility," Lao Z said. "The Provider only sends the request and brings the response back. It doesn't know whether this generation will create a ticket, write a file, or send an email. The idempotency key has to hang on the execution boundary that produces side effects — for example, attach a request ID when calling a tool, and let the tool side check whether that ID has already run. A Provider retry is just 'ask the model again'. The model may answer differently, and that new answer then triggers the action again, so the side effect may well repeat."
"What about error classification? Both vendors return 429 — can I treat them as one kind of error?"
"No. A status code is only an entry point; classify by semantics," Lao Z said. "Even one vendor's 429 can mean two things: 'you are requesting too often, wait a moment' and 'your quota is exhausted, a human has to act'. Retrying helps the first and only makes you wait longer in the second. Conversely, another vendor may express the same class of error with a completely different status code, or use one status code for problems at different stages. Misclassification is sneakier than a wrong retry: it doesn't fail on the spot. It lets the whole retry policy quietly push in the wrong direction. The logs show nothing unusual; only the cost grows."
For cases where looking only at the status code misclassifies, Lao Z gave three examples:
| Status code alone | Actual semantics | Correct classification |
|---|---|---|
| 429 → retry as rate limiting | Quota exhausted | Not retryable; escalate to a human or wait for quota to recover |
| 400 → fix the arguments | The model name was retired | Configuration error; check configuration instead of editing arguments |
| 401 → resend with the credentials | The credentials expired | Refresh the credentials, don't resend the same ones |
Key point
Retry only transient failures. Resending a generation request = regenerating = paying again, and it may trigger the external action twice.
4.5 The gains and costs of the design
The migration finally ran end to end, and Xiao A slumped into his chair. He did the arithmetic: over four hundred lines of adapter code for this migration, tests not counted. He asked Lao Z: "Was this abstraction worth it at all?"
"The Provider abstraction buys centralized governance, unified observability, and lower switching cost. Its price is wider types, a least-common-capability limit, and extension fields that exist in order to preserve advanced capabilities," Lao Z said. "Define the semantics you actually use first, then add adapters. Reaching for full vendor coverage too early usually turns the business layer back into vendor conditionals. A translation bureau is a fine thing, but staffing one for just two vendors is a bad deal."
"Wider types — I'd guess you mean the unified message keeps collecting fields?" Xiao A asked.
"Right. It's about the unified message's fields: they must hold the union of all vendors," Lao Z said. "Vendor A's message has role; Vendor B's has role plus name. The unified message has to carry name as an optional field even though Vendor A will never use it. The more fields, the wider the type, and the harder it becomes for a caller to see at a glance which fields actually take effect on the current vendor. The cost of wider types is blurrier boundaries."
"How do you add extension fields without letting the unified interface decay back into vendor branches?" Xiao A pressed.
"With an explicit namespace and a visibility convention," Lao Z said. "A unified field is semantics every vendor has. Vendor-specific fields go in a prefixed namespace like provider_extra, and the caller has to fetch it explicitly, never mistake it for a general field. The test: after deleting every provider_extra field, the business logic should still run across vendors. If it breaks, you have been treating a vendor-specific capability as a general one."
Lao Z grouped these costs and their countermeasures into a table:
| Design cost | How it shows up | Countermeasure |
|---|---|---|
| Wider types | More and more fields, most of them nullable | Separate core fields from the extension namespace |
| Least common capability | Cutting advanced capabilities to stay compatible with the weakest vendor | Keep advanced capabilities in extension fields, enabled by the capability list |
| Bloated adapters | One translation layer per vendor | Connect only the vendors you actually use; don't pre-build roads |
| Test matrix explosion | N vendors × M capabilities | Test the core paths first; verify extension capabilities on demand |
"What does test matrix explosion look like? A concrete example?" Xiao A asked.
"Yes, and nearly every migration hits it," Lao Z said. "Three vendors and five capabilities make fifteen combinations in the full cross product, and most of them never occur together in the real business. The test matrix is not the full cross product; it is the combinations that actually happen. Circle the core combinations along real paths and test those first, then add extension capabilities when they get enabled. A full cross product sounds rigorous, but it spends the budget on paths that never run."
4.6 The event contract of the adapter layer
The second week, Xiao A started on Vendor B's streaming event parsing. He found the easiest place to get it wrong: handling "events still in flight" and "events already finished" as one pile.
"The unified interface distorts most easily in streaming," Lao Z said. "Design internal events as records with a lifecycle, not just strings: text_delta only appends for display; tool_call_delta only accumulates arguments; usage is for observability; completed seals the result; failed and aborted terminate but keep what already arrived."
open → text_delta* / tool_call_delta* → completed
↘ failed | aborted"Which layer does retry live in?" Xiao A asked.
"By layer as well," Lao Z said. "A connection that never got established, a transient service failure, or an explicit rate-limit signal may suit bounded backoff. An authentication failure, an invalid model name, and argument validation failure should usually surface immediately."
"If the connection drops and reconnects, can the deltas I received repeat?" Xiao A pressed.
"They can, so consuming deltas has to tolerate duplicates," Lao Z said. "After a reconnect, some services re-send from the beginning and some resume from the break. Either way the deltas an adapter sees may overlap what it got before the break. A repeated text delta costs the UI at most one or two extra characters — harmless. A repeated tool-argument delta adds half a field when you concatenate, and the whole JSON fails to parse. Split delta handling by type: display text deltas directly and tolerate repeats; collect tool arguments, then validate them as a whole, and treat a parse failure as an error event. Tracking duplicates segment by segment is state you then have to maintain — don't take that on lightly."
"So the display side and the parse side tolerate different amounts?" Xiao A confirmed.
"Right — that is where the two lines fork," Lao Z said. "The display line wants real time: it allows duplicates and half-finished output. The parse line wants completeness: it would rather wait for the completion event than gamble on a half-product. Keep the two lines apart, and duplicates and reordering cannot hit both at once."
Illustration: when
{"path":"src/arrives, the adapter layer may display "tool arguments forming", but it must not parse them and execute. Only when the call ID, tool name, and complete arguments have all finished and passed validation may they go to the executor. Event names and fields are not uniform across APIs, so take the target Provider's official docs as the contract.
4.7 Write the request lifecycle as states
On the last day of the migration, Xiao A drew the whole call flow as a state diagram, taped it beside his desk, and asked Lao Z: "Is this right?"
"It is. And you should write it as a state machine, not a pile of ifs," Lao Z said after reading it:
prepare unified message → adapter converts request → send
→ rate limit: read the service hint or backoff policy, retry a bounded number of times
→ timeout: cancel the underlying request, mark it incomplete
→ streaming: accumulate text / tool deltas, never execute half-formed arguments
→ completed: seal the final message, usage, stop reason
→ failed / aborted: keep the partial result and a diagnosable error"Do these states travel across vendors?" Xiao A asked.
"No. rate limit, timeout, error objects, and maximum wait time have no shared semantics across providers, so the adapter must map them," Lao Z said. "We covered the rules for retry. One point often missed: a client-side timeout does not prove the remote side never received the request. Resending then may produce a second generation and a second charge."
"Any rule of thumb for how many retries and how long a backoff?"
"No cross-vendor value, only principles to follow," Lao Z said. "Keep the retry count finite and the backoff increasing: wait a short time first, then stretch it each round. If the vendor's docs give a retry hint — Retry-After, say — use its value; otherwise use conservative increasing backoff. Keep the retry window shorter than the time a single failure can last. Past the limit, fail outright or escalate to a human. Don't let one transient fault become a ten-minute retry loop."
4.8 Migration is not a config change, it is re-verifying the contract
"Now that the migration is done, does anything differ from your original 'one-line config change'?" Lao Z asked.
"Completely," Xiao A said with a wry smile. "I thought changing a base URL would be the end of it. In practice I re-verified authentication, message format, stream events, and error codes — all of it."
"That is what migration really looks like," Lao Z said. "The Provider abstraction lowers the cost of re-implementing, not the cost of checking the contract. Switching vendors means re-checking everything below, and not one item on memory:"
| Check item | What to confirm | Consequence of skipping |
|---|---|---|
| Authentication method | Bearer? API key? OAuth? | Every request 401s, and the logs leak keys |
| Message roles | system sent separately or merged into the array? | Role semantics shift and the rules stop applying |
| Streaming events | event names, fields, completion marker | Half-finished arguments get executed as a complete call |
| Stop reason | enum values, and whether the semantics match | Truncation read as a normal completion |
| Tool format | field names of call requests and results | Results cannot be linked back to their requests |
| Rate limits and error codes | Retry-After on 429, the meaning of 5xx | The retry policy breaks or retries the wrong things |
| Usage and billing | token field names, the billing unit | Cost reporting is wrong |
"Of these seven, which buries the worst landmine?" Xiao A asked.
"Stop reason," Lao Z said. "Two vendors can differ in both the field name and the values. OpenAI Chat Completions uses a finish_reason field, and a normal end is stop; Anthropic Messages uses stop_reason, and a normal end is end_turn. Same meaning, different spelling, so the adapter must map it. Miss the mapping and the downstream loop reads 'ended normally' as 'tool call' or 'truncated', and the whole Agent behaves wrong. This kind of bug is the hardest to chase: the model raised no error, the call didn't fail — only the stop semantics don't line up, and the task quietly stops in the middle."
"How do you verify every mine is cleared?"
"Run the same test suite you used against the old vendor, once each, and compare behaviour," Lao Z said. "You don't require identical output character for character. You require the structural results to match: stop reason, number of tool calls, error classification. The acceptance criterion for a migration is 'the same tests pass on the new vendor', not 'the new vendor looks usable'."
4.9 Make every request debuggable
"The biggest lesson from days of debugging is that with most problems you cannot tell at first glance which layer they are in," Xiao A said.
"So the last thing the Provider layer has to do is leave every request a debuggable record," Lao Z said. "A model request from start to finish has at least four moments worth recording: when the request goes out, when the first byte arrives, when the stream ends, and when an error happens. What you write at those four moments decides how far back you can trace when something goes wrong."
| Recording moment | What to record | What question it answers |
|---|---|---|
| Request sent | model name, estimated input tokens, request ID | who sent this request and what it cost |
| First byte | time to first token (TTFT) | is it stuck in the network or in server-side queueing |
| Stream end | total tokens, stop reason, duration | was the output truncated, did it end normally |
| Error | error code, a redactable message, context | which class of failure, and should it be retried |
"How do these records work with the redaction you mentioned earlier?"
"Redact before recording; write to disk only after redaction," Lao Z said. "Model name, endpoint, token counts, status code — those can be recorded in full. Request headers, raw response bodies, and authentication fields must be replaced or truncated. A good log lets you reproduce 'what happened at the time' without leaking 'what key was carried at the time'."
"And where does it get stored?"
"Layer it by what debugging needs," Lao Z said. "Structured fields — model name, tokens, duration — go to metrics or structured logs, so they aggregate. The raw event stream goes to a detailed log with a retention period, so you can replay one request. Store the two layers apart: read metrics day to day, open the details when something breaks."
"When something actually breaks, how do these records localize the problem?" Xiao A pressed.
"Sort by who holds the evidence," Lao Z said. "If the first byte never arrives and TTFT looks wrong, the problem is in the network or in server-side queueing — check the network layer and service status. If the stream breaks mid-way with text already received, the problem is in the connection layer — check the cause of the disconnect. If the stream ends cleanly but the stop reason is wrong, the problem is in the adapter layer or in how the caller handles stop semantics. Each class of problem has its evidence in a different place; go where the evidence lands instead of guessing from the top down."
"Isn't all this recording a cost in itself?" Xiao A asked.
"It is, and a continuing one," Lao Z said. "Recording all four moments in full costs storage and costs querying. Skip the key moments and you have no evidence when things break. The usual trade-off: record the structured fields — model name, duration, status code — in full, because they are cheap; give the raw event stream a retention period and clean it up automatically when it expires. Logs are data with a lifecycle, not a permanent archive. Set no retention period and logs will eventually fill the storage. Then the only move left is to delete everything, and one purge takes your debugging ability with it."
"Any last piece of advice?"
"Yes," Lao Z said. "The Provider is the layer standing in front of vendor differences, but it is not super glue. It lowers coupling; it does not lower complexity. Every abstraction brings its own rules: how errors get classified, how retries back off, how logs get redacted. Those rules need maintenance just as vendor differences do. Forget to maintain them, and the abstraction itself becomes the next debugging problem."
Summary
The core problem a Provider solves is that the upper application should not care which model service sits behind it. It does two things: it translates a request in unified form into a service request one vendor can understand, and it translates the streaming response back into a unified event sequence. This translation layer wraps authentication, the model catalog, and request behaviour, so the upper layers always face the same interface and switching vendors means switching only the Provider. Streaming is not returning the whole result at once; it is an event stream with a lifecycle — start, deltas, end — and each part has its own semantics. Timeouts and retries are also the Provider's job, but they are explicit controls: a timeout cancels an unfinished request, and a retry targets a clearly transient failure with a finite attempt count, not retrying forever until success.
But a Provider reduces coupling; it cannot abolish capability differences. Change vendors and the scope of capability negotiation, the semantics of error codes, and the rate-limit policy may all differ. Those differences must be handled explicitly, and an adapter may lose vendor-specific information while converting. Two traps are easy to fall into. First, error classification must be consistent across the timeout and retry paths: if one place treats a network error as success and another treats it as failure, the failure semantics of the whole chain drift. Second, idempotency is the precondition for safe retry: if running an operation twice charges twice or sends the email twice, blind retry manufactures an incident.
Hands-on Verification
Write a fake implementation of a Provider interface that emits, in order, a text delta, an incomplete tool argument, a completion event, and an error event. Confirm the UI does not treat half a JSON as call arguments. Then cancel a request and confirm the business layer receives "aborted", not "success".