alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The manuspec specification
This specification states exactly what each manuspec file holds and how every value in it is computed. The schemas, SQL, JSON-LD context, registries, vectors and fixtures are normative too, and each chapter says which of them it relies on. A chapter and a vector that disagree signal a bug, so please report it in the repository, https://git.scient.ing/infra/manuspec.
Two readers will get the most from it: implementers who build a reader, writer, converter or checker, and experts who want the precise rules. Newcomers should start with the guides, which explain what the formats are for. This text assumes you know why and tells you how.
Reading paths
Find the row for the job you are doing and read its chapters after those in the first row. Chapter numbers are those in the table below.
| If you are writing | Read |
|---|---|
| Anything at all | 0, 1, 3, 4 and 5: introduction, conventions, nodes, identity and the content values. |
| A reader of packages | Then 2, 6, 7 and 12: the model, anchors, geometry and the package. |
| A converter | Then 9, 10 and 11: figures and tables, references, and corrections with the gate. |
| A store or a server | Then 8, 13 and 14: receipts, the database form, and rights. |
| A writer or reader of annotation, teaching or data files | Then 15 to 21. |
| A validator | Chapter 22, with registry/rules.json. |
Chapters
| Chapter | Subject |
|---|---|
| 0. Introduction | Scope, conformance, versions and status, and how the repository is arranged |
| 1. Shared conventions | Canonical JSON, numbers, hashes, normalization, offsets, times, determinism, IDs, keys, files and prose |
| 2. Works, versions, witnesses and documents | The four objects; document.json and witnesses.json |
| 3. Nodes | Nodes, their kinds and children, content, attributes, inline runs and the plain projection |
| 4. Identity | Minting IDs, permanence, retirement, aliases, ids.jsonl, store-qualified names and record IDs |
| 5. Content values | The five content values, the projections, alignment, suspects, keyed hashes and the metadata hash |
| 6. Addresses and anchors | Addresses, anchors and the five resolutions, with drift |
| 7. Geometry | Page fingerprints, stamp rules, pages.jsonl and locations.jsonl |
| 8. Fidelity, receipts and spot-checks | Fidelity levels, receipts, signatures, Merkle roots, the roster, revocations, spot-checks and error bounds |
| 9. Figures, tables and equations | Figures, panels, tables, cells and equations |
| 10. References and citations | Reference nodes, references.jsonl and its resolutions, and sentence-level citations |
| 11. Corrections | Corrections and the gate |
| 12. The package | The files of a package, paths, manifest, provenance, text policies and assets |
| 13. The database form and the bundle | The database form and the SQLite bundle |
| 14. Rights | Rights records, policies, stages and backends |
| 15. Datasets | Frictionless Data Packages with units, uncertainty and origin; anchors.jsonl; content hash |
| 16. Digitization records | How values were read off a figure panel: target, axes, series, method, verification and confirmation |
| 17. Views | Vega-Lite specifications that pin their data, and optional computations |
| 18. Layers and annotations | Layers and annotations |
| 19. Exercises, submissions and grades | Exercises, submissions and grades |
| 20. Projects | Members and roles, documents, reading paths, held layers, datasets, views and drafts, and settings |
| 21. Drafts | Drafts: snapshots and sentence-ID sidecars |
| 22. Conformance | Conformance, the suite and the rule catalog |