Module data_stream 

Draft-20 data stream header encoding and decoding.

Byte-for-byte identical to draft-19. The field list, the field order, the widths and the set of valid Type values are all unchanged; what draft-20 changed is the receive path.

Type became Type Flags, and the rules left the figure

Draft-19 enumerated the legal leading values inside the figure — Type (vi64) = 0x10..0x15 / 0x18..0x1D / … for a subgroup header — and draft-20 declares the field a set of flags and states the invalid combinations as prose after it. Computing draft-20’s rules gives back draft-19’s enumeration byte for byte, on both carriers, so no valid frame changed hands. A decoder that validated by range membership was already right; one that validated loosely, by masking off bits it did not know, becomes wrong.

The two carriers do not share a rule set, and neither can be derived from the other. Section 11.4.2 gives SUBGROUP_HEADER three conditions and Section 11.3.1 gives OBJECT_DATAGRAM three of its own, and they disagree at two points:

• bit 4 (0x10) MUST be 1 on a subgroup header and MUST be 0 on a datagram, where it is reserved — the same bit, opposite requirements, and it is what tells the two carriers apart in the stream-type registry;
• the datagram has an “a bit set whose meaning is not specified” condition and the subgroup header has none, because bits 0 through 6 are all specified for a subgroup. The subgroup gets an explicit “values of 128 or greater” condition instead, which the datagram does not have — there the unspecified-bit rule catches the same values, since 128 sets bit 7.

Non-minimal Type Flags are accepted on receive and never emitted

Section 1.4.1 permits a value to be encoded in more bytes than it needs, and Section 11.4.2 words its third condition as “values of 128 or greater (i.e., any value that requires more than a one-byte variable-length integer encoding)”. Those two clauses are not equivalent under that allowance: the value 0x14 written as the two-byte 0x8014 is a legal varint whose value is below 128 and whose encoding is two bytes. This codec reads the rule as a bound on the value, so it decodes such a header, and it always emits the minimal one-byte form. The draft does not settle which clause governs; rejecting a legal-but-non-minimal encoding risks failing a conformant peer and emitting one risks tripping a stricter peer, so the asymmetry is the safe default. Draft-19’s decoder refused every wide spelling outright, which is the behaviour this replaces.

The payload rule

Unchanged from draft-19, and worth restating because it is the one place where the encoder refuses an object draft-18 would have reinterpreted. Draft-18 said every Object with a status other than Normal has an empty payload; draft-20 Section 11.2.1.1 says instead that an Object has an empty payload unless its status is registered as permitting one, and Section 15.9 puts that permission in the Object Status registry. Both encodings here keep their draft-18 framing — a subgroup object carries a status exactly when its Object Payload Length is zero, and a datagram carries one exactly when its type sets the STATUS bit, both stated that way by the draft — so no frame these decoders can read is able to state a status and a payload at once. The registry rule therefore bites where a caller can hold both: SubgroupObjectReader::write_object consults the status’s payload permission and refuses an object whose status forbids the payload handed with it, rather than dropping the status and writing the bytes as a Normal object. On the way back out, SubgroupObject and DatagramHeader answer the same question from the registry, so a reader never has to recover it from a length.

Subgroup Type Flags: form 0b0XX1XXXX, so bit 4 is set and bit 7 is clear; the ranges are 0x10..0x1F, 0x30..0x3F, 0x50..0x5F, 0x70..0x7F.

• bit 0 (0x01): PROPERTIES
• bits 1-2 (0x06): SUBGROUP_ID_MODE (0=zero, 1=first_obj, 2=explicit, 3=reserved)
• bit 3 (0x08): END_OF_GROUP
• bit 5 (0x20): DEFAULT_PRIORITY (no priority byte)
• bit 6 (0x40): FIRST_OBJECT

Datagram Type Flags: 0b00X0XXXX, so bits 4, 6 and 7 are clear; the ranges are 0x00..0x0F and 0x20..0x2F.

• bit 0 (0x01): PROPERTIES
• bit 1 (0x02): END_OF_GROUP
• bit 2 (0x04): ZERO_OBJECT_ID (object_id=0, field omitted)
• bit 3 (0x08): DEFAULT_PRIORITY (no priority byte)
• bit 5 (0x20): STATUS (status byte replaces payload)

Neither range is fully assigned, and draft-20 spells out which values in them an endpoint must refuse rather than decode, closing the session with a PROTOCOL_VIOLATION. Section 11.4.2 excludes the subgroup Types whose SUBGROUP_ID_MODE is the reserved 0b11 — 0x16, 0x17, 0x1E, 0x1F and the same four offsets in each higher range — because that mode does not say whether a Subgroup ID field follows the Group ID, so a decoder would have to guess and a wrong guess shifts every later field by the width of that varint. Section 11.3.1 excludes the datagram Types setting both STATUS (0x20) and END_OF_GROUP (0x02) — 0x22, 0x23, 0x26, 0x27, 0x2A, 0x2B, 0x2E and 0x2F — because an object status message cannot signal end of group. SubgroupHeader::decode and DatagramHeader::decode refuse both lists, and SubgroupHeader::encode_checked and DatagramHeader::encode_checked refuse to write them.

Fetch header: stream type 0x05 + request_id, then per-object records whose leading Serialization Flags varint says which of the object’s fields are on the wire at all; FetchObjectHeader decodes and encodes one such record. Draft-20 adds a third End of Range marker to Table 7, FetchEndOfRange::TimedOut (0x20C), for the Objects a relay abandoned when its FILL_TIMEOUT budget ran out — what draft-19 reported as an Unknown range.

Structs

DatagramHeader

FetchHeader

The head of a fetch stream: the stream type 0x05 and a Request ID.

FetchObject

One frame from a FETCH stream with its delta-encoded fields resolved.

FetchObjectHeader

One Object on a draft-20 fetch stream, up to but not including its payload.

FetchObjectLayout 🔒

Which optional fields a Serialization Flags value puts on the wire.

FetchObjectReader

Resolves the delta-encoded fields of the frames on one FETCH stream.

FetchObjectWriter

Re-encodes resolved fetch frames onto one FETCH stream.

SubgroupHeader

SubgroupObject

One object within a draft-20 subgroup stream. Object IDs are delta-encoded; whether a per-object “properties” block (the draft-20 equivalent of extension headers) is present depends on the PROPERTIES bit on the enclosing SubgroupHeader. Use SubgroupObjectReader to encode/decode.

SubgroupObjectMeta

The framing of one draft-20 subgroup object, without its payload.

SubgroupObjectReader

Enums

FetchEndOfRange

What an End of Range indicator on a fetch stream asserts about the Locations it covers, from draft-20 Section 11.4.4.2.

GroupOrder

The order a FETCH response’s Groups arrive in, which decides how a Group ID Delta is applied.

Constants

DATAGRAM_DEFAULT_PRIORITY_BIT 🔒

DATAGRAM_END_OF_GROUP_BIT 🔒

DATAGRAM_PROPERTIES_BIT 🔒

DATAGRAM_RESERVED_BIT 🔒

Bit 4. Draft-20 Section 11.3.1: “This bit is reserved and MUST be zero.” The opposite of Section 11.4.2, where the same bit MUST be 1.

DATAGRAM_SPECIFIED_BITS 🔒

Every bit draft-20 Section 11.3.1 gives a datagram Type Flags value a meaning for: PROPERTIES, END_OF_GROUP, ZERO_OBJECT_ID, DEFAULT_PRIORITY and STATUS. Bit 4 is reserved and bits 6 and up are unspecified, so both fail the mask.

DATAGRAM_STATUS_BIT 🔒

DATAGRAM_ZERO_OBJECT_ID_BIT 🔒

FETCH_DATAGRAM_BIT 🔒

Table 9 flag: the Object’s Forwarding Preference is Datagram, so it has no Subgroup ID and the two low bits are to be ignored.

FETCH_END_OF_NON_EXISTENT_RANGE 🔒

Table 7: End of Non-Existent Range.

FETCH_END_OF_TIMED_OUT_RANGE 🔒

Table 7: End of Timed-Out Range. New in draft-20.

FETCH_END_OF_UNKNOWN_RANGE 🔒

Table 7: End of Unknown Range.

FETCH_FLAGS_MAX 🔒

The largest Serialization Flags value whose bits are flags. Draft-20 Section 11.4.4: “When less than 128, the bits represent flags”.

FETCH_GROUP_ID_DELTA_BIT 🔒

Table 9 flag: a Group ID Delta field is present.

FETCH_OBJECT_ID_DELTA_BIT 🔒

Table 9 flag: an Object ID Delta field is present.

FETCH_PRIORITY_BIT 🔒

Table 9 flag: a Publisher Priority field is present.

FETCH_PROPERTIES_BIT 🔒

Table 9 flag: a Properties field is present.

FETCH_STREAM_TYPE 🔒

FETCH_SUBGROUP_ID_EXPLICIT 🔒

Subgroup ID mode 0b11: an explicit Subgroup ID field is on the wire.

FETCH_SUBGROUP_ID_MODE_MASK 🔒

Serialization Flags bits 0-1, the Subgroup ID encoding (draft-20 Section 11.4.4.1, Table 8).

PADDING_DATAGRAM_TYPE

Datagram type for padding, draft-20 Section 11.5.2: “An endpoint MAY send a datagram with a type of 0x132B3E29 to send padding data. The datagram contains the type followed by zero or more bytes that MUST all be set to zero.”

PADDING_STREAM_TYPE

Unidirectional stream type for padding, draft-20 Section 11.5.1: “An endpoint MAY open a unidirectional stream with a stream type of 0x132B3E28 to send padding data. The stream begins with the stream type, followed by zero or more bytes that MUST all be set to zero.”

SETUP_STREAM_TYPE 🔒

The unidirectional stream Type draft-20 Section 10.3 gives the control stream.

SUBGROUP_BASE_BIT 🔒

SUBGROUP_DEFAULT_PRIORITY_BIT 🔒

SUBGROUP_END_OF_GROUP_BIT 🔒

SUBGROUP_FIRST_OBJECT_BIT 🔒

SUBGROUP_ID_MODE_MASK 🔒

SUBGROUP_ID_MODE_RESERVED 🔒

The SUBGROUP_ID_MODE value draft-20 reserves, once the mask is applied and the field shifted down.

SUBGROUP_PROPERTIES_BIT 🔒

Functions

datagram_type_error 🔒

Which failure a leading datagram Type Flags value that is not one a reader wants is.

datagram_type_is_valid 🔒

Whether raw is a datagram Type Flags value draft-20 admits.

decoded_status 🔒

Turn a wire Object Status code into an ObjectStatus, refusing one draft-20 does not assign.

skip 🔒

Advance buf past len bytes without copying them.

stream_type_error 🔒

Which failure a leading unidirectional stream Type that is not the one a reader wants is.

subgroup_type_is_valid 🔒

Whether raw is a subgroup Type Flags value draft-20 admits: below 128, bit 4 set, and not the reserved SUBGROUP_ID_MODE.

validate_datagram_type 🔒

Refuse a datagram Type Flags value draft-20 Section 11.3.1 lists as invalid.

validate_subgroup_type 🔒

Refuse a subgroup header Type Flags value draft-20 Section 11.4.2 lists as invalid.