0 Fork 0

Files

T

feat: zipped packages, target set, checklists, audit fixes and a stress test

Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):

- Zipped packages: the oracle writes and reads `.sct.zip` archives under
  the rules of spec/12 §12.2; validate.py and load.py accept them; a
  byte-identical zip of the tiny package joins the fixtures, and two
  broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
  oracle and pinned by vectors/targets.jsonl under each digit rule; the
  rule now covers cells as well as sentences and the leaves of anchored
  containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
  covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
  WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
  accessible export can find a confirmed figure description; the layer
  fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
  used them.

Audit fixes, each with a vector or a broken package where one was
missing:

- Folder packages: symbolic links are refused on validate and load,
  manifest paths must not collide under case folding, listed files are
  matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
  a lone surrogate or invalid UTF-8 is refused instead of translated;
  integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
  signed; a qa receipt's nodes must be the sample at the original's
  hashes, its scope must name qa.of, its errors must be sampled nodes;
  statuses are computed once, iteratively, so chains of spot-checks
  hundreds deep settle; Base64 and manifest shape errors become refusals
  instead of tracebacks; counts and the fidelity of roster-less packages
  are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
  validated against the node schema and the references between nodes
  are checked, so a set to a bad value, an insert with unknown
  properties or a delete of a referenced header cell is refused; a
  relabel to null drops the label; a reclassify drops foreign
  attributes; a split strips whitespace across runs; ids.jsonl order
  follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
  node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
  ({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
  the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
  "10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
  computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
  revocation may be carried by several packages.

Checking:

- tools/stress.py: seeded randomized property tests over canonical JSON,
  normalization, notation, random correction walks, anchor resolution,
  Merkle roots, spot-check samples, error bounds, fingerprints, mutated
  archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
  the repository, the zipped fixture and crafted archives, the checklist
  coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-10-14 20:07:22 -04:00

4.1 KiB

Raw Blame History

Contributing

This page is for anyone who wants to fix the specification, change a format or add an example. Propose a change as a pull request on the repository at https://git.scient.ing/infra/manuspec.

What a change needs

A change starts in the chapter of the specification that defines the rule, because the chapters are normative. The schemas, SQL, registries, vectors and fixtures then follow the chapter (spec/00 §0.5).

Change the chapter. Say what is new in the chapter's own terms, and link the sections it touches.
Change the schemas, SQL and registries that the rule shapes. A new refusal gets a rule ID in registry/rules.json.
Compute, never type, the expected values. Change the oracle in tools/oracle/ to apply the rule, and add the case that shows it to tools/build.py: a vector, a fixture or a broken package in fixtures/invalid/. Then run pixi run build, which rewrites fixtures/ and vectors/.
Run the checks. pixi run check must end with All checks pass. Without pixi, run python3 tools/check.py, with Python 3.14 and two packages, jsonschema and cryptography installed.
Record the change in CHANGELOG.md.

Versions

A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version of the format, with a migration. Adding an optional property, a node kind, a run kind or an enumeration value is a minor change (spec/00 §0.3). A rebuild that changes any file under vectors/ or any hash in fixtures/ is a computed value changing, so say in the pull request why it must.

New node kinds and run kinds

Propose a kind with the vector or fixture that shows it. A node kind needs its row in chapter 3, the node schema, and its entry in the CHILDREN table of tools/oracle/model.py. A run kind needs its row in section 3.5 and the node schema, and its text in the plain projection (3.6) and in the strict and lossy projections (5.3, 5.5).

Releasing

A release is a tag on main whose version is the top heading of CHANGELOG.md. Before tagging:

pixi run check passes on Linux and macOS, which the workflow in .gitea/workflows/ runs on every push. The check also confirms that every count the prose states (formats, schemas, vectors, broken packages) matches the repository, and that pixi.toml, the registries and tools/build.py state the same version.
python3 tools/stress.py passes with its default seed, and with one or two others.
The JSON-LD context is served at https://scient.ing/ns/scienting/1.jsonld and every schema at its $id under https://scient.ing/ns/schemas/, both as static files; context/README.md records whether they resolved last time someone checked. Layers claim to be W3C Web Annotations, and a JSON-LD processor that cannot fetch the context cannot read them.
The decisions that freeze on release are made: enumeration values and format identifiers never change meaning afterwards (spec/22 §22.5), so a name worth changing is changed now.
The CHANGELOG entry describes the release in the terms of the chapters it touches.

Writing

The checker enforces two rules on every Markdown file: one sentence per line, and US spelling. pixi run lint runs markdownlint with the repository's settings. Define each term where it first appears, because the guides and READMEs are written for newcomers as well as implementers.

All fixture text is synthetic, written for the fixtures. The keys in fixtures/keys/ are public test keys and must never sign anything else.

Licenses

Each path keeps the license that LICENSE.md gives it: prose CC-BY-4.0, the normative data CC0-1.0, and tools/ Apache-2.0.

4.1 KiB Raw Blame History