feat: zipped packages, target set, checklists, audit fixes and a stress test
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
142 files changed
+2312
-268
No files matched your search
+3
-3
@@ -427,8 +427,8 @@ Keep using it as a reading and distribution format.
|
||||
|
||||
*Status.*
|
||||
The specification is version 0.1.0.
|
||||
`registry/formats.json` lists 27 formats: 20 are `candidate` (complete, with vectors and fixtures, ready to implement) and seven are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot and draft-ids ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
|
||||
The repository holds 23 chapters, 37 JSON Schemas, 20 vector files, five valid fixture packages and 17 invalid ones, each naming the rule it breaks, and a reference checker, `tools/check.py`.
|
||||
`registry/formats.json` lists 29 formats: 18 are `candidate` (complete, with vectors and fixtures, ready to implement) and 11 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade and checklist ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
|
||||
The repository holds 23 chapters, 39 JSON Schemas, 21 vector files, five valid fixture packages, one of them also zipped, and 28 invalid ones, each naming the rule it breaks, and a reference checker, `tools/check.py`.
|
||||
Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; tools Apache-2.0 ([README.md](../README.md#licenses)).
|
||||
|
||||
*No adoption claim.*
|
||||
@@ -471,7 +471,7 @@ These are the places where the guides and the specification differ, and this pag
|
||||
6. **Package tree:** `docs/concepts.md` omits `suspects.jsonl` from its package tree, and [spec/12 §12.1](../spec/12-package.md#121-files) lists it.
|
||||
7. **RFC 6962 is obsolete:** spec/08 §8.5 cites it; RFC 9162 obsoletes it (see above).
|
||||
8. **Status:** The guides describe the formats as one set.
|
||||
Seven of 27 are `provisional`, including `scienting-dataset/1`, which the datasets paragraphs in `docs/concepts.md` present without that qualifier.
|
||||
Of the 29 formats, 11 are `provisional`, including `scienting-dataset/1`, which the datasets paragraphs in `docs/concepts.md` present without that qualifier.
|
||||
|
||||
*The distinguishing combination, and what it costs.*
|
||||
manuspec puts in one package permanent sentence- and cell-level IDs, five hashes per node, anchors with explicit resolution outcomes and drift, append-only evidence-gated corrections, signed fidelity receipts with spot-check bounds, per-witness rights with AI-use axes, and value-to-cell anchors.
|
||||
|
||||
Reference in new issue
Block a user