Struct SharedDataflow 

pub struct SharedDataflow { /* private fields */ }

The operators of a plan that do not read the session, arranged once for every subscriber.

“Do not read the session” is the sentence §5.3 uses and it is one atom short: what this holds is the operators that are a function of the accumulator alone, so everything downstream of crate::plan::Op::Presence is excluded too. The reason is this type’s version — it is the log’s seq, and a roster moves when seq does not (docs/96 §96.5).

docs/05-tier-lowering.md §5.3:

a thousand connected users of todos.map(filter_by(session.user)) must compile to one shared dataflow whose final per-session operators (filter, project, diff) run per subscriber

crate::plan::Plan has said which nodes those are since the plan existed — per_session is false for exactly the operators reachable from the accumulator without passing through the session. What was missing was somewhere for them to live that is not one subscriber’s engine.

The three choices §24.7 said this design had in it

1. Who advances it. Not the sequencer: that would put view maintenance on the write path and do it for a state nobody is looking at. The first subscriber to render at a new version advances it, under a write lock, and every subscriber that renders at that version afterwards finds it done. So the work happens once per version, is paid by a renderer that was about to do it anyway, and does not happen at all when nobody is subscribed.
2. What a subscriber holds while it renders. A read lock, for the whole of its own render. Readers do not block readers, so a thousand subscribers render concurrently; the only writer is the advance, which is O(δ). The alternative — publishing an immutable snapshot per version — has to copy any arrangement that moved, which is the O(n) this engine exists to remove.
3. What happens to a subscriber that fell behind. It replays the changes it missed, from a bounded history of recent versions (Step). Beyond that history it rebuilds — correct at any lag, because a rebuild reads the current arrangement whole and a rebuild is already contagious downstream (Cell::rebuilt).

What is still not shared

The page is per-session in every corpus program, so what is shared is the prefix below the session, not the render. 24-feed.beck is the case where that prefix is most of the plan and the sketch is the case where it is least; docs/26 has the table.

The lifecycle: who keeps this alive, and for how long

The three choices above say how the dataflow is maintained. They are silent about when it stops being worth maintaining, which 26 §26.9 recorded as two loose ends — arrangements that are never released, and a change history that is a constant rather than a policy. Both are the same missing rule, and it is the reader-frontier discipline of differential dataflow’s shared arrangements: a reader set, a frontier per reader, history compactable up to the minimum frontier, and the trace droppable when the reader set is empty.

So a subscriber engine is counted. SharedDataflow::subscriber enters it in the reader set and its Drop removes it; each render publishes the version it reached; an advance compacts to the oldest frontier and, when the last reader goes, the arrangements are released outright. What the process holds is then a function of who is connected rather than of what has ever connected.

Implementations

impl SharedDataflow

pub fn new(prepared: Arc<Prepared>) -> SharedDataflow

pub fn with_retention( prepared: Arc<Prepared>, retention: Retention, ) -> SharedDataflow

pub fn retention(&self) -> Retention

pub fn subscriber(self: &Arc<Self>) -> Engine

A subscriber’s engine over the same plan: the per-session operators, and nothing else.

The engine is a reader of this dataflow for exactly as long as it lives. It takes an Arc<Self> because that is what makes the second half true: the engine has to be able to say it has gone, and a subscription ends by dropping its engine rather than by calling anything.

pub fn reader(self: &Arc<Self>) -> Reader

A reader of the shared arrangements that renders no page: crate::read’s SQL client.

It is a member of the same reader set as a subscription, and that is the design rather than an implementation convenience. A SQL client holding a connection is a reason to keep the arrangements — it is going to ask again — and a SQL client that has gone is not, which is exactly what the reader set already decides for subscribers (51). The alternative, reading the arrangements without joining the set, has a release racing every query.

Its frontier stays at the unrendered one: a reader that never applies a delta cannot use the change history, so pinning any of it for this reader would retain history nobody reads.

pub fn render( &self, engine: &mut Engine, state: &Value, version: u64, session: &Value, presence: &Value, ) -> Result<(Value, u64), ExecError>

Render one subscriber’s page, maintaining the shared prefix once for all of them.

version identifies the state: two calls with the same version must pass the same state, because the second is served from what the first computed. Returns the page and the version it actually reflects, which may be newer than the one asked for — another subscriber may have advanced the shared side in between, and rendering the newer state is correct where rendering the older one would mean unwinding an arrangement.

That returned version is not a courtesy. A patch frame is labelled with a seq and a resuming client is served the difference from it (§4.3), so a frame labelled with a state the page does not reflect is a wrong DOM after the next reconnect.

pub fn version(&self) -> u64

The version the shared prefix currently reflects.

pub fn advances(&self) -> u64

How many times the shared prefix has been advanced since the process started.

§5.3’s claim is that a thousand subscribers of one view share one dataflow. This is the number that says so: it counts advances, not renders, so it stays flat as subscribers are added and moves only when the fold does.

pub fn releases(&self) -> u64

How many times the arrangements have been given up because nobody was reading them.

The counterpart to SharedDataflow::advances, and the number a deployment weighs against it: every release is a cold start charged to whichever subscriber reconnects first.

pub fn readers(&self) -> usize

How many subscribers are attached right now.

pub fn retained(&self) -> usize

How many versions of change history are being kept.

Bounded above by Retention::depth and below by the oldest attached reader’s frontier, so on a fanout whose subscribers all render at every version it is 1 rather than 64. This is the number that says the frontier discipline is doing something.

pub fn arranged(&self) -> u64

Entries across every shared arrangement — held once, however many subscribers there are.

pub fn footprint(&self, base: &Value) -> Footprint

What the shared prefix retains beyond the accumulator — once, for every subscriber.

pub fn work(&self) -> Work