Expand description
A list’s elements, in one of two layouts.
docs/105-the-ecosystem-answer.md §105.8:
Valueis 16 bytes and a list isList(Arc<Vec<Value>>), so a million doubles is a boxed 16 MB; andFloat(u64)is stored as an order-preserving key rather than asf64bits, which is exactly right for the reason its doc comment gives — a map key and the state digest need a total order agreeing with arithmetic — and exactly wrong for a dense kernel, which pays a bit transform per operation. That is not a defect to fix inValue. It is a second representation to add.
This is that representation, and the sentence that matters most about it is the one it is not: it is not a second kind of list. A Beck program has one list type, one order, one equality, one digest and one wire format, and this module’s whole obligation is that a caller cannot tell which layout it got.
§The two layouts
| Layout | Bytes an element | What it is for |
|---|---|---|
Seq::Boxed | 16 | Anything at all — records, strings, nested lists. What every list has always been |
Seq::Ints | 8 | A list of Int, dense |
Seq::Floats | 8 | A list of Float, dense and as f64 rather than as Value::Float’s order key |
The Floats row is the one with a consequence beyond memory. A kernel — BLAS, an FFT, anything
this project has no business reimplementing (105
§105.8) — takes a *const f64, and so does Apache Arrow: a Float64Array’s values buffer is
a contiguous f64 run. Seq::floats is that pointer, and until it existed there was nothing
in this language to hand either of them.
§What a caller may not be able to tell, stated as the four things
Two Seqs holding the same elements are the same list, and four separate mechanisms have to
agree about that or replay determinism (04
§4.8) fails in a way that depends on how a value happened to be built:
- Equality and order.
OrdandEqare written by hand here, over the logical sequence. A derived one would compare the variant tag first, makingInts([1])andBoxed([Int(1)])two different values and sorting every column before every list. - The digest (
crate::core::digest) hashes a tag, the length and each element, and reaches the elements throughSeq::iter— so it is the same bytes either way by construction rather than by a second implementation agreeing. - The wire format (
crate::repr) does the same. Value’s size. The layout is an enum behind theArc, so aValueis still 16 bytes and a list still costs one pointer. Putting the enum in theValuewould have widened every value in the language to pay for a representation most of them do not use, which is the tradecrate::core::Value’s own doc comment refused forcrate::core::Record.
seq.rs’s own tests assert the first, and beck-cli/tests/records.rs asserts all four against
the layouts a program can actually produce.
§Where a column comes from
Nothing in the language says “make this a column”, and nothing should: the layout is a fact
about the elements, so it is chosen where a list is built. Seq::pack takes the elements a
primitive produced and reads them; Seq::push promotes an empty list on its first element,
which is what makes the accumulator idiom go(i + 1, list_append(done, x)) — how lib/, the
corpus and both SICP chapters build a list (70
§70.6) — produce a column with no program changing a line.
Promotion is only ever O(1): an empty list on its first push, or a pack over elements the
caller had already built. A Boxed list of a million ints is not re-examined on every push,
because that check is what would turn the idiom quadratic.
§The off switch
Choosing a layout is a choice the runtime makes unbidden, so docs/08
§8.3 item 8 applies: set_columns turns it off for the process, and the gate runs both
settings. With it off every list is Seq::Boxed and every answer is the same one — which is
what the switch is for, and what makes “a caller cannot tell” a test rather than a claim.
Two callers reach it without recompiling: beck_rt::AppConfig::columns for a served
application, and BECK_COLUMNS=0 for a beck process that is not one — run, test, bench
and build all build lists, and none of them has an AppConfig.
It is process-wide, which is the one thing about it worth knowing before using it: a list is
built in a hundred places that have no configuration in scope, and a Value may not carry one
— it is 16 bytes on purpose. So two applications in one process share the setting, and a test
binary that flips it has to serialise the tests that do.
Structs§
Enums§
- Seq
- A list’s elements. See the module docs for what a caller may not be able to tell.
Functions§
- built
- How many columns this process has built.
- columns
- Whether the columnar layout is on.
- set_
columns - Turn the columnar layout on or off for this process, and answer what it was.