Part II Chapter 9: Shipping and Customs Clearance — pi-protocol
Chapter guide: Stop ⑧ on the source route (foundation layer). Switch to the
pi-protocolpackage in the foundation layer and readcodec.ts,framing.ts, andprotocol.ts. What this stop has to take apart is why Xiao A's line "just turn the object into JSON and throw it over" does not work: a byte stream has no message boundary, and it takes four layers — schema, CBOR, frame — to catch the problem one layer at a time.
Xiao A wrote a remote tool, and the first step was to send a message to the other side. He typed socket.write(JSON.stringify(msg)), then turned around, pleased with himself: "Is this not enough? Just turn the object into JSON and throw it over."
"There is no boundary for 'one message' in a byte stream." Lao Z shook his head. "What the peer receives is a run of bytes. How does it know where your JSON ends? If a message gets split into two chunks halfway through, how does it know the receiving is not done? A remote session cannot hand TypeScript objects straight to the socket. We start reading from the schema in pi-protocol and keep going until incremental bytes become a verified message."
This chapter does not discuss transport listeners or how business commands execute. All source evidence comes from upstream pi at tag v1.0.0 (commit a13d35a7, packages/protocol/src/, package version 1.0.0).
The four-layer chain
"The write path in packages/protocol/src/codec.ts is: schema validation → CBOR encoding → length frame. The read path is the reverse: FrameDecoder cuts out a complete payload, decodeCbor decodes it, and parseClientMessage or parseServerMessage validates it with the TypeBox Check."
schema → encodeCbor → 4-byte length frame
network chunk → FrameDecoder → decodeCbor → schema check → messageMetaphor: shipping and customs clearance
Sending a message is like shipping goods: inspect first (schema), then pack (CBOR), then stick on a length label (frame). The receiver reads the label to cut the package, unpacks, and inspects last. If any step is wrong, the whole shipment goes back.
"The strict objects in protocol.ts disable additional properties, and PROTOCOL_VERSION is 8. At v1.0.0 the protocol defines only the envelope; it no longer defines business commands: the client sends hello, request, and cancel, and the server answers with hello, hello_error, response, service_update, and attachment. A request carries one target, which either holds only serverId or adds sessionId and attachmentId. The real business content sits inside call, result, and update, and for those the schema only writes Type.Unknown(). codec.ts additionally requires the whole message to be a legal JSON value and rejects circular references, non-plain objects and the like — so 'CBOR can decode it' does not mean the protocol accepts it."
Xiao A: "The business content is not validated at all?"
"The protocol layer only guarantees that it is a JSON value. The meaning of the business fields is validated by the service above." Lao Z said. "Upstream commit 48dd1e2f moved the experimental client/server into pi-durable, so the protocol retreated into a general-purpose RPC envelope. This chapter reads only the envelope and the bytes, not the business."
Xiao A pressed: "Why can't the four layers be merged into one? Isn't it enough for the schema to just reject things?"
"Because each layer blocks a different class of bad input, and merging them leaves gaps." Lao Z went layer by layer. "The schema (TypeBox Check) blocks 'the structure is right but a field is wrong': type misspelled, serverId not in UUID form. chord's isJsonValue blocks 'the runtime value has the wrong shape': circular references, non-plain objects such as Map / Date — TypeBox cannot see those before serialization. The CBOR layer blocks 'malformed byte encoding': a truncated payload, illegal UTF-8, nesting too deep. The frame layer blocks 'lost boundaries': half a frame, an over-long declaration. If only the schema remained, an object containing a circular reference would blow up during CBOR encoding, and the error would land in the byte-generation stage instead of the validation stage — hard to locate."
"The other way round, the four layers each have a cost too." Lao Z said. "Every message goes through two validations (a parse on the write side, a parse on the read side), one CBOR encode/decode, and one frame read/write. But input across a process or the network is untrusted to begin with, and what this cost buys is: malformed bytes never become a message that business code executes."
"The four layers are not the only way to cut it either. It is a position taken between two opposing demands." Lao Z laid the possible shortcuts out in a table. "Drop any layer, and you postpone one class of error to a later stage. Only the overhead you save differs."
| Alternative | Overhead saved | New risk in exchange | Where the failure surfaces |
|---|---|---|---|
| schema only | CBOR encode/decode, frame read/write | runtime shape errors such as circular references surface only at encoding; the byte stream still has no message boundary | the encoding stage — late and imprecise |
| schema + CBOR, no frame | frame prefix read/write | the peer can only know a message ended after reading the whole thing; a half frame cannot be recovered safely | the read side — undecidable across chunks |
| schema + frame, no CBOR | the CBOR layer | over-deep nesting and very long single strings go straight into parse; decode resources can be exhausted by malformed input | the read-side parse |
| all four layers (current) | none | every message goes through two validations, one encode/decode, one frame read/write | a unified ProtocolValidationError boundary |
"Read it together with the call chain: each handoff point accepts only the product of the layer above it." Lao Z drew a second handoff table. "What the upstream hands over and what the downstream needs are pinned in this table."
| Handoff point | Input | Output | Failure shape |
|---|---|---|---|
| business object → parse | an unknown value | a validated union-type message | ProtocolValidationError |
| message → encodeCbor | a validated message | a payload with no length prefix | CborError over the limit, then wrapped |
| payload → encodeFrame | Uint8Array | 4-byte prefix + payload | over uint32, or not a Uint8Array |
| chunk → FrameDecoder | a block of arbitrary bytes | zero to many payloads | over the limit, half frame, failed state |
| payload → decodeCbor | a payload | an unknown value | CborError (truncation, too deep, illegal encoding) |
| unknown → parse | the decoded value | a union-type message | schema / structure failure |
Validate the object first, then encode it
Xiao A jumped in with an answer: "I'll guess — validate while encoding, one pass?"
"No. Validate first, encode after." Lao Z opened parseClientMessage in codec.ts:
export function parseClientMessage(value: unknown): ClientMessage {
if (!Check(ClientMessageSchema, value) || !isJsonValue(value)) {
throw new ProtocolValidationError("Invalid client protocol message");
}
return value;
}"TypeBox Check first validates the union type, the field ranges, and additionalProperties: false. isJsonValue comes from @earendil-works/chord and recursively accepts JSON scalars, arrays, and plain objects while rejecting circular references, non-plain objects, and non-finite numbers. The two steps cannot replace each other: in the envelope, call, result, and update are Unknown in the schema, and only isJsonValue looks inside them. Conversely, isJsonValue has no idea whether this is one particular kind of client message."
The write side's encodeProtocolMessage finishes like this:
const validated = parse(value);
try {
const maxFrameLength = options?.maxFrameLength ?? DEFAULT_MAX_FRAME_LENGTH;
return encodeFrame(encodeCbor(validated, { maxByteLength: maxFrameLength }));
} catch (error) { /* wrapped into a ProtocolValidationError */ }"Reject the invalid object first, then bound the CBOR output, and append the frame last. Exceptions get wrapped into ProtocolValidationError, and the error text is truncated by boundedErrorMessage, so a peer cannot construct an over-long error and pollute the logs."
Xiao A stared at the encodeCbor line: "The frame length limit comes from configuration, but CBOR's actual output length is known only after encoding. What if it goes over?"
"The maxByteLength that encodeCbor receives is exactly maxFrameLength, and it does not only check a single string." Lao Z said. "The writer in cbor/encoder.ts checks the number of bytes it needs in total before every reallocation and throws CborError once the limit is passed. So a message built from many small fields whose total goes over the limit also fails while the bytes are being generated. Older versions kept a second assertCompleteFrame check after framing; v1.0.0 removed it from framing.ts. From that we can infer that the write side's length contract is now carried entirely by the CBOR writer."
Xiao A caught another detail: "The error text is cut to 500 characters — does the stack information get truncated away too?"
"It does, but deliberately." Lao Z said. "Error text travels back over the protocol or goes into logs, and the peer is untrusted. If it constructs an over-long input that makes the schema fail, then without truncation the error text itself becomes the vehicle of an amplification attack. 500 characters is enough to locate 'which field, which type error'. Stack information belongs in local logs; it should not ride along in the error envelope."
Xiao A flipped back: "What is that ancestors Set in isJsonValue for? Isn't it there to check circular references?"
"It is cycle detection." Lao Z opened packages/chord/src/json.ts. "check recurses with an ancestors set: it adds before entering an array or object and deletes in the finally when it leaves. If the same object appears again on the recursion stack, ancestors.has(value) decides false outright — this blocks direct self-reference like a.self = a and the indirect cycle a.x = b; b.y = a alike. There are also a few lines that are easy to miss: an object whose prototype is neither Object.prototype nor null fails; Date, Map, Set, and class instances cannot pass; a property with a symbol key, or a getter, fails; numbers must be finite; an array must additionally have no holes and no extra properties. A property whose value is undefined also fails, so an 'optional field' has to be genuinely absent — you cannot write it as undefined."
"The write side and the read side run the same pair of checks, in opposite directions: the read side adds one CBOR decode, the write side adds one frame assembly. Draw out the whole write path and it looks like this."
object value ──Check──> hits a union member? ──isJsonValue──> the whole value a JSON value?
│ │ │
│ no: bad field/type │ no: cycle / Map / Date / class │ yes
▼ ▼ ▼
ProtocolValidationError ProtocolValidationError encodeCbor ──cumulative over limit?──> CborError → ProtocolValidationError
│ within the limit
▼
encodeFrame ──> frameHow a chunk becomes a message
Xiao A: "What about network chunks? One message might get split across several packets, right?"
"It can also be several messages in one packet. That is why FrameDecoder keeps the unfinished tail; only a complete payload goes to the CBOR decoder, and only after CBOR succeeds does it go to parse." Lao Z opened ValidatedMessageDecoder.push:
for (const frame of this.frames.push(chunk)) {
messages.push(this.parse(decodeCbor(frame, { maxByteLength: this.maxFrameLength })));
}
return messages;"Every complete frame inside one chunk goes to the same parser, in order. If any one of them fails, it falls into the catch, and you never get a partial array of 'the first few succeeded'. Callers should treat one push as a batch boundary."
Xiao A still had a question about half frames: "How exactly does it accumulate them inside the decoder?"
"Look at the state machine in FrameDecoder.push." Lao Z turned to framing.ts. "It has four state variables: headerLength (how many length-prefix bytes have arrived), expectedPayloadLength (how much payload to receive, known only once the prefix is read), payloadLength (how many payload bytes have arrived), and payloadBlocks (the payload array accumulated in 64KB blocks). Once a chunk comes in, the loop fills the header first: if the chunk has 2 bytes left and the header still needs 4, it fills 2, continues, and waits for the next push. When the header is full it reads frameLength, checks whether it exceeds maxFrameLength (over the limit means fail at once), and then enters the payload accumulation stage. Payload works the same way: one chunk may contain the tail of the previous message and the head of the next, and the loop cuts them apart into different frames."
"Two boundaries deserve special attention." Lao Z said. "First, frameLength is validated the moment it is read, before any payload memory is allocated. Otherwise the peer declares a length of 4GB, and the decoder tries to allocate first and fails after — the memory has already been eaten. Second, at end(), if headerLength !== 0 or expectedPayloadLength !== undefined, a half frame is still stuck at the end of the stream; it has to report Truncated frame at end of stream. A half frame cannot be swallowed as a normal EOF, or the caller will think all the messages arrived."
"Draw the decoder's progression and it is two loops in a row." Lao Z drew a state diagram:
chunk ────────────────────────────────────────────────────────┐
│ │
v │
[header phase] headerLength < 4 ──chunk runs out──> continue, wait for the next push
│ headerLength == 4
v
read frameLength ──> over maxFrameLength ──> fail (before any payload is allocated)
│ frameLength == 0 ──> emit an empty payload, back to the header phase
│ frameLength > 0
v
[payload phase] accumulate in 64KB blocks until payloadLength == frameLength
│
v
single block emitted directly / several blocks merged → complete payload → back to the header phase
│
v
end() ── headerLength != 0 or expectedPayloadLength still set ──> "Truncated frame at end of stream""Note that frameLength === 0 is an explicit branch: an empty-payload frame is emitted directly, instead of entering payload accumulation and sticking there." Lao Z pointed at the diagram and added a line. "That path is uncommon, but the decoder handles a zero-length declaration explicitly, which shows the edge cases were enumerated one by one. As for what happens when an empty payload reaches decodeCbor, that is the CBOR layer's business: not one byte can be read, so it reports truncation. The frame layer does not intercept it."
| Stage | Input / output | Normal semantics | Failure semantics |
|---|---|---|---|
| FrameDecoder | chunk → payload list | may accumulate across chunks | over the limit, or an incomplete tail frame, raises an error |
| decodeCbor | payload → unknown | restores the CBOR value | illegal encoding or a limit trips an exception |
| parseClient / ServerMessage | unknown → union type | yields a usable message | schema or structure failure |
| ValidatedMessageDecoder | chunk → message list | emits zero to many messages in order | permanently failed after a failure |
The boundaries of the frame and the decoder
"framing.ts uses an unsigned 32-bit big-endian length prefix, and the default limit for a single frame is 16 * 1024 * 1024 bytes — configurable, but it may not exceed uint32. The decoder can accumulate across multiple network chunks, and it also rejects incomplete tail frames and over-long frames."
Xiao A: "Does that length prefix carry a type and a checksum?"
"It does not." Lao Z pointed at encodeFrame:
const length = payload.byteLength;
frame[0] = length >>> 24;
frame[1] = length >>> 16;
frame[2] = length >>> 8;
frame[3] = length;"This is a big-endian uint32 length. It carries no type, no version, and no checksum. Type and version are the schema's job; integrity is the transport's. The decode options in cbor/decoder.ts bound byte length and nesting structure, so one malformed input cannot occupy memory or recursion depth without limit. The exact limits have to be checked against that file's options and the call arguments — do not treat the default frame limit as the single limit for every CBOR structure."
Xiao A went on: "So what limits does the CBOR layer have of its own?"
"cbor/options.ts gives three independent defaults." Lao Z read them off one by one. "maxByteLength defaults to 16MB (aligned with the frame limit) and bounds total bytes and any single byte or text string; maxContainerLength defaults to 1,000,000 and bounds array element counts and map entry counts; maxDepth defaults to 64 (configurable up to 512) and bounds nesting depth. The three are independent: a 100KB message can hold a flat array of a million elements and trip the container limit, or a 70-deep nest of small objects and trip the depth limit, and neither gets a pass just because total bytes stayed under 16MB. decodeCbor also rejects indefinite-length, tags (major type 6), break markers, non-finite floats, integers outside the safe integer range, non-string map keys, duplicate map keys, and extra bytes trailing after the payload. What it accepts is a restricted subset, but it does not check whether an encoding is the shortest one, so you cannot say it accepts only the deterministic encoding of RFC 8949."
Xiao A: "Why a length prefix, instead of newline-delimited like JSON lines, or just a magic byte in front of every frame?"
"This is the classic multiple-choice question of transport protocols." Lao Z said. "Newline separation looks simple, but then a newline inside the message content must be escaped, and the boundary is really 'read until you hit a newline': the length cannot be validated in advance, and the whole line has to be buffered in memory first. Magic bytes plus a length would help resynchronize after a bad frame, but magic costs bytes, and the current protocol did not choose it — it is a bare 4-byte length prefix with no synchronization marker at all. That is exactly the root reason the decoder cannot skip bad bytes and recover: with no marker to scan for, it cannot realign at the start of the next frame. This choice makes 'rebuild the connection' the only safe recovery point. What the length prefix buys is O(1) boundary detection and pre-allocation validation; what it costs is 4 bytes of overhead and 'one bad frame spoils the stream'."
Failure invalidates the decoder
Xiao A: "So if one chunk is bad and good bytes arrive afterwards?"
"After ValidatedMessageDecoder.push catches a frame, CBOR, or schema exception, it sets failed = true; any later push throws 'decoder has failed'. The error is again wrapped into ProtocolValidationError, and its text is truncated the same way. The caller cannot count on automatic recovery over the same corrupted byte stream; normally it should close the connection and re-establish the protocol state."
Key point: stricter failure semantics
The
faileddesign discards later bytes that may well have been correct, and the caller has to rebuild the decoder and the handshake state. The payoff is that nothing ever guesses at recovery on a stream that has lost its frame boundaries. The cost is that every message must be encoded and validated, but input from across a process or the network should not skip validation in pursuit of throughput.
"Protocol validation failures also have to be separated from business errors: in response, the error that comes with ok: false is { code, message }, where code only has to be a non-empty string and the actual values are up to the service above; a version mismatch goes through hello_error. These are all validated envelopes: the client can display them and keep processing. Bad CBOR or a bad schema is not a recoverable business response."
"These failure classes differ completely in severity. Laid out in a table it is clearer." Lao Z listed a classification table:
| Failure class | Typical trigger | Decoder state | What the caller does |
|---|---|---|---|
| frame level | over-long declaration, half frame at EOF, bad length | failed, connection ends | rebuild the decoder and handshake state |
| CBOR level | truncated payload, illegal UTF-8, over-deep nesting, tag / indefinite | failed, connection ends | rebuild the decoder and handshake state |
| schema level | type misspelled, extra field, serverId format wrong | failed, connection ends | rebuild the decoder and handshake state |
| business level | a response with ok: false, hello_error | decoder still healthy | display the error; whether to keep the connection is up to the layer above |
Xiao A: "Is there any way to sacrifice just one frame instead of throwing the whole connection away?"
"There is, but every option changes the protocol." Lao Z said. "One: put a synchronization marker in the frame header, and after a bad frame scan for the next magic and realign. Two: number every request, and after a lost frame fall back to a timeout and resend. Three: do not declare death at the decoder layer — drop the bad chunk and report the 'suspicious frame' upward as a whole. All three either widen the frame header or change session semantics, and both are outside the current protocol's trade-off. What the current protocol picks is 'wrong, so cut': the simplest way to guarantee that no fake message is ever produced. Flow control and error correction are bet on the transport layer; recovery is bet on reconnection."
Xiao A would not let go: "What about letting the decoder skip the bad bytes and carry on reading from the next frame?"
"Because once the frame boundary is lost, you have no idea how far the bad bytes extend." Lao Z went back to FrameDecoder.fail. "It clears all the state: headerLength, payloadBlocks, and expectedPayloadLength are all reset, then it sets state = "failed". Consider a 'skip and recover' scheme: you receive a frame that declares 100 bytes, but only 50 arrive before the stream cuts out. The decoder skips it and keeps reading — but is the next stretch of bytes the header of a new frame, or the residue of the bad one? As I said, the length prefix carries no synchronization marker, so there is no way to tell. The only safe recovery point is still to rebuild the connection."
"Put that trade-off against different traffic and the result differs." Lao Z said. "For low-frequency, high-integrity messages like requests and responses, paying the cost is worth it. Pushes like service_update, which may be high-frequency, land on the strict side; but pi-protocol multiplexes them onto the same frame stream, so strict semantics have to apply uniformly."
"end() follows the same failed semantics." Lao Z wrapped up. "ValidatedMessageDecoder.end() throws 'decoder has failed' again when the FrameDecoder is already failed, instead of returning silently, so the caller never mistakes it for a normal end of stream."
Summary
Remote communication cannot throw objects straight at a socket, because a byte stream has no natural boundary for "one message". pi-protocol rebuilds that boundary with four layers: the schema defines which objects the protocol accepts and strictly forbids extra fields; CBOR encodes a validated object into compact bytes; the frame splits the byte stream with a four-byte big-endian length prefix; the validated decoder strings these layers back together, from chunk to frame to CBOR to message. Each layer has its own failure semantics: an over-long frame is rejected before allocation, a half frame errors at EOF instead of pretending to be a normal end, CBOR caps nesting depth so that malformed input cannot exhaust memory. One chunk may span several messages or contain only half of one; the decoder keeps the unfinished tail and hands only complete frames to the stages behind it.
The most memorable part of this design is the failure semantics: after a failure at any of the frame, CBOR, or schema layers, the decoder enters a permanent failed state, every later push throws, and it accepts no more input. That trades stricter semantics for clearer boundaries; the implementation never guesses at recovery on a stream that has lost its frame boundaries. Skipping bad bytes is not possible because the length prefix carries no synchronization marker, and how far a bad frame extends cannot be determined. To keep communicating, the caller rebuilds the protocol state according to its connection-level policy. Protocol validation failure and business error must also be separated: hello_error and a response with ok: false are validated envelopes the client can display; bad CBOR or a bad schema is a connection-level fatal error.
Layering is a choice with a price. Every message goes through two validations, one CBOR encode/decode, and one frame read/write; four decision points interlock: the frame format (length prefix vs. newline vs. magic), the CBOR limits (byte, container, depth — three independent axes), the split of validation (Check for the envelope schema, isJsonValue for the runtime shape), and the failure semantics (cut on error vs. skip and recover). Removing any layer postpones one class of bad input to a later stage, and what you get in exchange is more overhead and harder localization. For a low-frequency, high-integrity session protocol this trade-off is worth it; for a high-frequency, loss-tolerant stream it lands on the strict side. But pi-protocol multiplexes requests, responses, and service pushes onto the same frame stream, and uniform strict semantics leave the protocol with only one failure model to understand.
Source Walkthrough
- Read
ClientMessageSchema,ServerMessageSchema, andPROTOCOL_VERSIONinprotocol.ts, and list which fields areUnknown. - Follow from
encodeClientMessageincodec.tsdown toencodeFrame, then read backwards fromClientMessageDecoder.push. - Mark the length- and depth-related limits in
framing.tsandcbor/decoder.ts, and construct an over-long frame to verify the error path. pushone frame in two pieces, and confirm the first can return an empty array and only the second yields a message.- Feed the decoder illegal CBOR and
pushagain, and confirm the failed state does not keep consuming bytes.