feat: zipped packages, target set, checklists, audit fixes and a stress test
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
142 files changed
+2312
-268
No files matched your search
+9
-8
@@ -52,7 +52,7 @@ The procedure is the same for every file:
|
||||
3.
|
||||
Find the inputs and the expected outputs of the case.
|
||||
The table below says which fields are which, and the chapter it links defines the rule.
|
||||
Only five files, `alignment`, `corrections`, `grading`, `resolve` and `rights`, put the expected outputs under `expect`.
|
||||
Only six files, `alignment`, `corrections`, `grading`, `resolve`, `rights` and `targets`, put the expected outputs under `expect`.
|
||||
In the other fifteen the expected outputs are fields beside the inputs, such as `output`, `sha256`, `canonical` or `parsed`.
|
||||
4.
|
||||
Compute each output from the inputs with your implementation.
|
||||
@@ -72,13 +72,13 @@ Some files change the procedure:
|
||||
Each of its cases lists ids from `projections.jsonl` and names a field in which those cases must be equal (`same`) or unequal (`differ`).
|
||||
Compute the field for each listed case and compare across the group.
|
||||
- `fingerprints.jsonl` needs the download-stamp rules in [registry/stamps/scienting-stamps-1.json](../registry/stamps/scienting-stamps-1.json), which each case names in `rules`.
|
||||
- `resolve.jsonl` and `grading.jsonl` name a package by a path from the repository root.
|
||||
Both files use `fixtures/packages/hale2026harmonic`.
|
||||
- `resolve.jsonl`, `grading.jsonl` and `targets.jsonl` name a package by a path from the repository root.
|
||||
All three use `fixtures/packages/hale2026harmonic`.
|
||||
|
||||
Several files also mix more than one kind of case, told apart by their fields.
|
||||
`ids.jsonl` has minting cases and letter-first cases, `keyed.jsonl` and `signatures.jsonl` have a case for each derivation or record, `spotcheck.jsonl` has sample-size cases and one full sample, and `rights.jsonl` has stage checks (a `stage` field), backend checks (a `backend` field) and one combining case (neither).
|
||||
|
||||
The reference checker in [tools/](../tools/README.md) runs all twenty files, together with the rest of the repository's checks:
|
||||
The reference checker in [tools/](../tools/README.md) runs all twenty-one files, together with the rest of the repository's checks:
|
||||
|
||||
```sh
|
||||
python3 tools/check.py --quick
|
||||
@@ -122,18 +122,19 @@ The table below uses a few terms that the chapters define:
|
||||
| `resolve.jsonl` | Anchors made on the example paper's first reading, and how each resolves after its corrections. | [6.5](../spec/06-anchors.md#65-resolution) |
|
||||
| `rights.jsonl` | Stage checks with policies and expiry, the combining rule, and backend conditions. | [14](../spec/14-rights.md) |
|
||||
| `grading.jsonl` | Exercises, submissions and the grades they earn. | [19.4](../spec/19-exercises.md#194-grading) |
|
||||
| `targets.jsonl` | The target set of targeted F3 for the example paper under each digit rule, given the anchored nodes. | [8.10](../spec/08-fidelity.md#810-targeted-f3) |
|
||||
|
||||
Resolve those paths against the files in that package only.
|
||||
|
||||
## What the vectors prove, and what they do not
|
||||
|
||||
The specification, and the vectors with it, are at version 0.1.0 ([0.3](../spec/00-introduction.md#03-format-identifiers-and-versions)).
|
||||
The specification has 27 formats: 20 are `candidate` and seven are `provisional` in [registry/formats.json](../registry/formats.json).
|
||||
The specification has 29 formats: 18 are `candidate` and 11 are `provisional` in [registry/formats.json](../registry/formats.json).
|
||||
A candidate format changes only with a recorded reason and a new or changed vector ([22.5](../spec/22-conformance.md#225-stability)).
|
||||
|
||||
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `tools/oracle/`.
|
||||
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
|
||||
The vectors hold 203 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` and `signatures.jsonl` two cases each.
|
||||
The vectors hold 221 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` and `signatures.jsonl` two cases each.
|
||||
|
||||
The vectors do not show the following.
|
||||
|
||||
@@ -147,8 +148,8 @@ The vectors do not show the following.
|
||||
- `canonical-json.jsonl` has five cases, none for the repository's rule that a lone surrogate, `NaN` or an infinity has no canonical form.
|
||||
- `merkle.jsonl` has roots for 1, 2, 3, 4, 5 and 7 items.
|
||||
A receipt with no items is invalid ([8.5](../spec/08-fidelity.md#85-merkle-roots)), so no case has an empty list, although the RFCs it follows define a hash for an empty list ([RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162)).
|
||||
- Seven formats are `provisional`, and no case carries their identifiers.
|
||||
Their chapters define values that no vector pins, such as a dataset's content hash ([15](../spec/15-datasets.md)) and the hash that confirms a digitization ([16](../spec/16-digitization.md)).
|
||||
- Eleven formats are `provisional`, and only `grading.jsonl` carries any of their identifiers (exercise, submission and grade).
|
||||
The chapters of the others define values that no vector pins, such as a dataset's content hash ([15](../spec/15-datasets.md)), the hash that confirms a digitization ([16](../spec/16-digitization.md)) and a checklist's content hash ([8.11](../spec/08-fidelity.md#811-checklists)).
|
||||
They are expected to change as implementations gain experience ([0.4](../spec/00-introduction.md#04-status)).
|
||||
- Reading a PDF: how text lines are extracted from a PDF is outside the specification; `fingerprints.jsonl` supplies the lines, so it tests the fingerprint function only ([7.1](../spec/07-geometry.md#71-page-fingerprints)).
|
||||
- Faithful conversion: the vectors check what is computed from text, not whether a converter read the paper correctly.
|
||||
|
||||
Reference in new issue
Block a user