Expand description
What a source file may contain, before anything tries to read it.
docs/35-standards-landscape.md §35.5 item 2:
“pin the Unicode version per release; add UTS #39’s security profile with conformance vectors”.
docs/08-roadmap.md §8.5.2 classes it R — a retrofit that
becomes expensive “the moment identifiers exist in published packages”, which is before the
registry and therefore now.
§The profile Beck adopts
UTS #39 defines restriction levels for identifiers. Beck is at Level 1, ASCII-Only, and it
is there by construction rather than by filtering: the Python surface’s identifier production is
[A-Za-z_][A-Za-z0-9_]* and always has been. That is the strictest level in the report, and it
makes the two attacks UTS #39 is mostly about — confusables and mixed-script identifiers —
unrepresentable rather than checked. §12.7’s vocabulary for that distinction is
“unrepresentable by construction”, and identifiers.rs is the negative test proving it.
What ASCII-only identifiers do not close is the other half of UTS #39 §4: bidirectional
confusion, where a file renders in an editor differently from how it compiles. That is
Trojan Source (CVE-2021-42574), it works through comments and string literals rather than
identifiers, and no restriction on identifiers touches it. scan is that check.
§Why the version pin is one line
The compiler carries no Unicode tables: an ASCII-only profile needs none, and the character
classes below are stable properties of characters that were assigned decades ago. UNICODE is
therefore a statement of which version these rules were written against rather than a
dependency — and the day Beck accepts a non-ASCII identifier, that constant stops being a note
and starts being a thing with tables behind it.
Constants§
- UNICODE
- The Unicode version this file’s rules are stated against.
Functions§
- is_
non_ ascii_ letter - Whether a character would be an identifier character in some language but is not one in Beck.
- scan
- Check a source file before either surface reads it.