alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
56 lines
4.2 KiB
Markdown
56 lines
4.2 KiB
Markdown
# 22. Conformance
|
|
|
|
## 22.1 The suite
|
|
|
|
| Path | Holds | A conforming implementation |
|
|
| --- | --- | --- |
|
|
| `vectors/*.jsonl` | Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
|
|
| `fixtures/packages/` | Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of [12.2](12-package.md#122-paths). | Accepts each, and recomputes every value it contains. |
|
|
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `tools/load.py --bundle`. | Accepts each. |
|
|
| `fixtures/invalid/` | Invalid packages, each with `expect.json` naming the rule it breaks. | Refuses each, and SHOULD name that rule. |
|
|
| `fixtures/keys/` | The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
|
|
|
|
All fixture text is synthetic and dedicated to the public domain; license fields inside fixtures describe the fictional works.
|
|
Each part's CI runs this suite against its own implementation, so the parts cannot drift apart silently.
|
|
|
|
## 22.2 Validating a package
|
|
|
|
A validator checks, in order, and reports each failure as `path:line: rule: message`:
|
|
|
|
1. The manifest: present, valid, sorted, every listed file present with its hash and size, nothing unlisted, paths safe and no symbolic links, required files present.
|
|
2. Every file in canonical form, byte for byte, and every record against its schema.
|
|
3. The tree ([3](03-nodes.md)), runs and references between nodes, and the counts.
|
|
4. The policy and hashing scheme ([12.5](12-package.md#125-text-policies)).
|
|
5. The five values of every node, the root and `meta_hash`, when the text is present and the key, if any, is known.
|
|
6. The ID table, the corrections, the witnesses, pages, locations and references, and every file's order.
|
|
7. Receipts: IDs, roots, documents, node order, signatures against the package's roster, spot-check keys, seeds and samples, and the fidelity distribution.
|
|
|
|
## 22.3 Schemas
|
|
|
|
Schemas use draft 2020-12, restricted to the keywords `$schema`, `$id`, `$ref`, `$defs`, `$comment`, `title`, `description`, `examples`, `default`, `deprecated`, `type`, `enum`, `const`, `properties`, `required`, `additionalProperties`, `patternProperties`, `propertyNames`, `minProperties`, `maxProperties`, `dependentRequired`, `items`, `prefixItems`, `minItems`, `maxItems`, `uniqueItems`, `contains`, `minLength`, `maxLength`, `pattern`, `format`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `oneOf`, `anyOf`, `allOf`, `not`, `if`, `then` and `else`.
|
|
Patterns avoid `\d`, `\w`, `\s`, `\b` and lookbehind, whose meaning differs between regular-expression dialects.
|
|
A schema checks shape; the chapters' rules, such as the five values, are checked by validators.
|
|
|
|
## 22.4 Rules
|
|
|
|
[registry/rules.json](../registry/rules.json) lists every rule ID with its meaning.
|
|
Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is never reused.
|
|
|
|
| Prefix | Covers |
|
|
| --- | --- |
|
|
| `package.` | The manifest, paths, files and schemas. |
|
|
| `doc.tree.`, `doc.node.`, `doc.run.`, `doc.table.` | The tree, node content and runs. |
|
|
| `doc.hash.` | The five values, the root and the metadata hash. |
|
|
| `doc.id.`, `doc.order.` | The ID table and file order. |
|
|
| `doc.policy.` | Text policies and keyed hashing. |
|
|
| `doc.correction.`, `correction.` | A package's corrections, and the gate. |
|
|
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
|
|
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
|
|
| `grade.` | Grading. |
|
|
|
|
## 22.5 Stability
|
|
|
|
Within a major version of a format, a valid file stays valid, and every value the chapters define is computed the same way.
|
|
Rule IDs, enumeration values and format identifiers are never reused with another meaning.
|
|
A `candidate` format changes only with a recorded reason and a new or changed vector; a `provisional` one may change more freely, and says so in [registry/formats.json](../registry/formats.json).
|