Expand description
Per-class shaping statistics: the atomic storage and its snapshot.
The split mirrors instrument exactly — a
ShapeRecorder of atomics that the forwarding tasks write, and a
plain-value ShapeStats that a reader takes. It is a sibling of
Counters, not an extension of it: interest_none.rs compares
Counters whole against Counters::default() and actions_*.rs
compare it by value, so a field added there would have to be threaded
through every one of those assertions, and Counters would lose Copy
for a Vec that is empty on all but shaped sessions.
§Why a pre-sized Vec, indexed by position
The class list is fixed for a session’s lifetime — live reconfigure is
expected to build a new recorder rather than mutate this one. So the
rows are allocated once at session construction, in
ShapeProfile::classes order, and every increment on
the data path is one relaxed fetch_add at a known index. No lock, no
name lookup, no hot-path allocation — the cost model instrument.rs
already established.
Snapshot order is therefore the configured order, deterministically, and
a class that never saw a unit is present with a zero row rather than
absent. A reader asking “what did video do?” gets an answer whether or
not video did anything, which is the difference between a starved
class and a mis-typed one.
§What has a producer today
Two session totals — objects_seen and bytes_shaped — are written at the
one place a shaped session’s framed unit becomes visible, and they exist
here so that an unshaped session shaped nothing is falsifiable rather than
vacuous: something has to move when the path is entered, or an all-zero
snapshot cannot tell “not armed” from armed and did nothing.
bytes_shaped additionally counts what no rule could see — a stream
header, an oversized object’s passthrough chunk, a bypassed stream’s tail —
because the unshapeable row is a term of the conservation identity and a
term with nothing on the other side of the equals sign is not a term.
objects_seen does not: it is the classifier’s count, and a header is not
an object.
Admission adds the three rows a policy can move without a clock:
objects_dropped / bytes_dropped (Overflow::DropTail),
blocked_episodes (Overflow::Block) and streams_reset_by_shaping
(Overflow::ResetStream). Every one of them is charged to the class the
classifier resolved, so a per-class figure is an answer about a rule
and not about a stream.
Release adds the rest: bytes_delivered / objects_delivered on every
granted unit, tokens_exhausted_episodes when a class’s own bucket is
dry, starved_behind_other_class when a different class’s unit was in
the way, objects_expired on the Expiry::ResetStream arm, and
streams_with_mixed_classes once per stream that carried two classes.
§Session totals carry a direction
Every session total is stored twice, once per leg: uplink is the client’s traffic on its way to the relay, downlink is the relay’s on its way to the client. One recorder serves every forwarding task of a session, so without the split an author shaping both legs reads a single figure and cannot tell an uplink stall from a downlink one — a bidirectional run reports downlink starvation as though the uplink class had caused it.
The flat totals stay, and they are derived: bytes_shaped is
uplink.bytes_shaped + downlink.bytes_shaped, summed in
ShapeRecorder::snapshot rather than accumulated in a third atomic.
The data path therefore costs exactly what it did — one relaxed
fetch_add into the arriving leg’s row — and the aggregate cannot drift
from its parts, which is not a property any pair of independently
written counters has. What that leaves falsifiable is the attribution:
charging every byte to one leg keeps the sum right and both legs wrong,
which is what the tests below and egress.rs’s two-leg test aim at.
Per-class rows are deliberately not split. An author who wants a
class figure per leg writes two classes and keys each matcher on
Matcher::side; the totals are the ones no
configuration could separate, which is why they are the ones that carry
the direction themselves.
§The proxy-wide aggregate, and why it is not a sum over live sessions
ProxyRecorder is the second recorder in this module: one per
TransparentProxy, held by its control
plane, and charged by the same writers that charge the session
recorder — every note_* below forwards, so no call site in session.rs
or egress.rs knows it exists and no figure can be charged to one
recorder and missed by the other.
Summing the sessions a control plane lists instead would be wrong three
ways, and only the first is fixable. The registry holds no recorder at
all, so there is nothing to sum. The registration is released by a
Drop, so a total taken over it would go down when a client
disconnected — a statistic that falls under normal operation cannot be
alerted on. And the list is a snapshot of a proxy that keeps moving, so a
sum walked across it is a consistent read of nothing. A recorder that
outlives every session has none of those problems: a session whose whole
future is dropped mid-flight still contributed at the instant each unit
was charged.
§Where a proxy-level byte is charged: the measurement point
ProxyStats::per_leg is a 2×2 — two legs, two directions — and the
two axes are orthogonal, which is exactly what makes the shape worth
having and exactly what makes it easy to fill in wrongly. A proxy holds
two connections; a byte crosses both, read on one leg and written on
the other. So the cell is chosen by where the measurement is taken, not
by a label copied off the arriving side:
per_leg[Client].uplink— bytes read from the client.per_leg[Upstream].uplink— bytes written to the relay.per_leg[Upstream].downlink— bytes read from the relay.per_leg[Client].downlink— bytes written to the client.
The alternative — deriving a leg from the side a counting site holds — looks
equivalent and carries no information at all. Every hook site is handed
ClientToProxy or RelayToProxy and nothing else
(ShapeProfile::try_new refuses a rule keyed on an egress side), and over
those two values leg and direction are the same partition: the client row
would be a verbatim copy of the uplink row, the upstream row of the downlink
row, and two of the four cells would be identically zero. Four numbers
carrying two numbers’ worth of information, with a reader who took
per_leg[Upstream] for what this proxy sent upstream getting the figure
for what it received from the client.
Charging by measurement point makes the difference between the two uplink cells the shaper’s own retention — bytes it read from the client and did not write to the relay — which is a number that can be non-zero and therefore a number that can be asserted.
Only the flow of units is measured twice. The three event figures on a
DirectionStats — expiries, streams a policy gave up on, streams that
carried two classes — are decisions taken over traffic that arrived,
so they are charged to the arrival cell and the departure cell reports
zero for them. That is stated on LegStats rather than left for a
reader to infer from a zero.
ProxyStats::sessions is the flat rollup, and it sums the two
arrival cells — the two places a byte enters this proxy, where each
byte is counted exactly once. Summing all four would count every byte
twice, once read and once written.
Every figure on this page has a producer. Worth stating, because it was not always true: five fields here snapshotted as a constant zero for a long time, each documenting its own emptiness, and what settled them was not writing five producers but noticing that none of the five belonged here.
Everything on this page is gated on a configured ShapeProfile — a
proxy running without one reports ProxyStats::default() however many
gigabytes it forwards. Two of the five counted what a hook does,
which needs no profile at all, so in this type they could only ever have
been a partial count that read zero for every unprofiled session that
delayed or truncated a thousand objects. They are
Counters::units_delayed
and
Counters::objects_truncated
now, beside
Counters::objects_elided,
which had already settled where a hook’s decision is counted.
The other three were Duration totals, and a sum of wall-clock time that
the machine’s scheduler moves as much as this code does is reportable and
never assertable. The timing dimension is measured instead by
Counters::release_errors,
which reports a distribution — p50, p95 and an exact maximum — rather
than a total nobody can calibrate.
With release and the unshapeable row wired, the conservation identity
Σ classes(delivered + dropped) + default + unshapeable == bytes_shaped
holds for a stream that ran to completion. Both sides come from the same
measurement — raw.len() for a framed object, Pending::len() for a
unit no rule saw — taken at the see-point and charged again, unchanged,
at release, so it is an identity over one number and not an agreement
between two.
Two things break it, both named rather than papered over. A
STOP_SENDING-driven teardown, where propagate_stop clears the queue
rather than draining it — those bytes are neither delivered nor dropped,
and Impairment{QueuedBytesAtTeardown} is what accounts for them. And a
hook action that changes a unit’s size after it was seen: Replace,
ReplacePayload and Truncate are all charged in full on the left and
by what they actually wrote on the right. Both are properties of the
run, not of the recorder, which is why the fixture that asserts
the identity takes no hook action and runs its streams to completion
before it reads.
Structs§
- Class
Counters 🔒 - One class’s atomic counters — the storage behind one
ClassStats. - Class
Stats - Per-class shaping statistics, one entry in
ShapeStats. - Direction
Counters 🔒 - One leg’s session totals — the storage behind one
DirectionStats. - Direction
Stats - Five totals over the traffic travelling one way.
- LegStats
- One connection’s shaping statistics, split by which way the traffic was
going — one entry of
ProxyStats::per_leg. - Proxy
Recorder 🔒 - Proxy-scoped shaping-counter storage — the storage behind
ProxyStats. - Proxy
Stats - Shaping statistics for a whole proxy: every session it has accepted, including the ones that have already ended.
- Session
Stats - What every session this proxy has run did, added up — the flat form of
ProxyStats. - Shape
Recorder 🔒 - Session-scoped shaping-counter storage.
- Shape
Stats - Shaping statistics for one session.
Enums§
- Direction 🔒
- Which leg of the proxy a shaped unit is travelling on.
Functions§
- leg_
index 🔒 - This leg’s row in
ProxyStats::per_leg. - opposite 🔒
- The leg a unit read on
legleaves this proxy by. - split 🔒
- The connection a side’s traffic is travelling over, and the way it is
going — the pair
ProxyStats::per_legis indexed by.