Module incremental

Module incremental 

Source
Expand description

Which views can be maintained by delta, and which have to be recomputed — and why.

docs/03-type-and-effect-system.md §3.8:

Subscribed views (anything feeding a live page, or marked materialized) compile to incremental dataflow plansremaining updates by ±1 per event, never by recount. … Arbitrary pure code is incrementalized where analysis allows, recomputed where not — beck explain incremental <view> shows which, and why.

20 §20.6 item 3 said the input for this existed (“a view whose row is empty is a pure function of the signal — which is §3.8’s precondition”) and §20.5 said the command was not built, because until the general slicer there was no plan to ask about: an inlined view is one expression, and “which vertices are incremental” is not a question an expression can answer.

§What this is, and what now sits beside it

It is the analysis: a verdict per view, from the shape of what that view computes. When it was written there was nothing behind it — every view was a full recompute per event and the report said so in its first line, because a command called explain incremental that printed “incremental” about a recompute would be the most misleading output in the compiler.

There is now an engine (crate::plan, crate::engine), and the report’s first line changed with it rather than before it. The two answer different questions and the report gives both:

  • this module asks whether a view — a vertex of the signal graph — is a pure function built only from operations with delta rules;
  • crate::plan decomposes what the view does into operators, so a view this module calls recompute because it contains a match may still have its collections maintained around that match.

The plan is the truth about what runs. This is the truth about what a view is, which is the answer a developer needs before writing one that quietly costs a recount per event over a million rows.

§The rule, and where it comes from

Three things have to hold before a vertex can be maintained by delta, and they are checked in this order because that is the order in which the answers are useful:

  1. The row is empty. §3.8’s precondition, and the one Phase 2 already computes. A view that performs an effect is re-evaluated when the effect says so, not when its input changes.
  2. Every operation it applies has a delta rule. list_len after a filter_list updates by ±1; a sort_by maintains a sorted arrangement; arithmetic and record construction are pointwise. A match on the accumulator, or a function this analysis cannot see through, has no rule, and the honest answer is “recompute”.
  3. It is downstream of a durable fold and upstream of a sink. A vertex nothing subscribes to is not a view; §3.8’s scope is “anything feeding a live page, or marked materialized”.

RULES is the table for step 2. Like crate::cost’s numbers it is stated, not measured — each entry is a delta rule the differential-dataflow literature already has, and it is written down so that it can be argued with rather than discovered in a profiler. Nothing in this module claims an implementation exists for any of them.

Structs§

Assessment
One vertex’s assessment.

Enums§

Verdict
What a view engine could do with one vertex.

Constants§

RULES
The operations with a known delta rule, and the rule.

Functions§

assess
Assess every vertex between the durable folds and the sinks.
report
What beck explain incremental prints.
verdicts
A map from vertex label to verdict, for a test that wants the answer rather than the prose.