feat: zipped packages, target set, checklists, audit fixes and a stress test

Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):

- Zipped packages: the oracle writes and reads `.sct.zip` archives under
  the rules of spec/12 §12.2; validate.py and load.py accept them; a
  byte-identical zip of the tiny package joins the fixtures, and two
  broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
  oracle and pinned by vectors/targets.jsonl under each digit rule; the
  rule now covers cells as well as sentences and the leaves of anchored
  containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
  covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
  WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
  accessible export can find a confirmed figure description; the layer
  fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
  used them.

Audit fixes, each with a vector or a broken package where one was
missing:

- Folder packages: symbolic links are refused on validate and load,
  manifest paths must not collide under case folding, listed files are
  matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
  a lone surrogate or invalid UTF-8 is refused instead of translated;
  integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
  signed; a qa receipt's nodes must be the sample at the original's
  hashes, its scope must name qa.of, its errors must be sampled nodes;
  statuses are computed once, iteratively, so chains of spot-checks
  hundreds deep settle; Base64 and manifest shape errors become refusals
  instead of tracebacks; counts and the fidelity of roster-less packages
  are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
  validated against the node schema and the references between nodes
  are checked, so a set to a bad value, an insert with unknown
  properties or a delete of a referenced header cell is refused; a
  relabel to null drops the label; a reclassify drops foreign
  attributes; a split strips whitespace across runs; ids.jsonl order
  follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
  node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
  ({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
  the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
  "10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
  computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
  revocation may be carried by several packages.

Checking:

- tools/stress.py: seeded randomized property tests over canonical JSON,
  normalization, notation, random correction walks, anchor resolution,
  Merkle roots, spot-check samples, error bounds, fingerprints, mutated
  archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
  the repository, the zipped fixture and crafted archives, the checklist
  coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 20:07:22 -04:00
1 parent 8c8b51cc3f
commit ea0a7efa78
142 files changed
+2312 -268

No files matched your search

+12 -5
View File
@@ -158,9 +158,12 @@ def _follow(state, nid):
def _relabel_reverted_own(state, doc, nid, anchor):
"""The own_hash the node had before the relabel corrections since the anchor, or None."""
rel = [c for c in state.corrections if c["index"] > anchor["rev"] and c.get("target") == nid]
if not rel or any(c["op"] != "relabel" for c in rel):
"""The own_hash the node had before the relabel corrections since the anchor, or None when anything else
changed the node or what is below it since (spec/06 §6.5: its *only* changes were relabel corrections)."""
since = [c for c in state.corrections if c["index"] > anchor["rev"] and (
c.get("target") == nid or nid in (c.get("path") or []) or nid in (c.get("to_path") or []))]
rel = [c for c in since if c.get("target") == nid]
if not rel or any(c["op"] != "relabel" or c.get("target") != nid for c in since):
return None
n = dict(doc.nodes[nid], label=rel[0]["from"])
tmp = Doc(doc.key, [n if x["id"] == nid else x for x in [doc.nodes[i] for i in doc.order]])
@@ -175,7 +178,11 @@ def resolve(state, anchor, threshold=Fraction(1, 4)):
original = anchor["node"]
nid, chain, via = _follow(state, original)
live_kind = doc.nodes[nid]["kind"] if nid else None
kind = kind or live_kind or next((e["node_kind"] for e in state.ids if e["id"] == original), None)
table = {e["id"]: e for e in state.ids}
# The kind falls back to the retired entry the chain reached (spec/06 §6.5); an alias carries no kind.
retired_kind = next((table[x].get("node_kind") for x in reversed(chain)
if table.get(x, {}).get("kind") == "retired"), None)
kind = kind or live_kind or retired_kind
cls = anchor_class(kind) if kind else "leaf"
is_span = "span" in anchor
result = {"node": nid}
@@ -256,7 +263,7 @@ def resolve(state, anchor, threshold=Fraction(1, 4)):
now = set(doc.leaves(nid))
now_match = {values[x]["match_hash"] for x in now}
held = sum(1 for i, m in orig if i in now or m in now_match)
if orig and 2 * held >= len(orig):
if 2 * held >= len(orig):
return out("moved")
return out("stale", reason="the successor holds fewer than half of the original leaves")
if values[nid]["node_hash"] == anchor["node_hash"]: