0 Fork 0

Files

T

feat: zipped packages, target set, checklists, audit fixes and a stress test

Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):

- Zipped packages: the oracle writes and reads `.sct.zip` archives under
  the rules of spec/12 §12.2; validate.py and load.py accept them; a
  byte-identical zip of the tiny package joins the fixtures, and two
  broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
  oracle and pinned by vectors/targets.jsonl under each digit rule; the
  rule now covers cells as well as sentences and the leaves of anchored
  containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
  covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
  WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
  accessible export can find a confirmed figure description; the layer
  fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
  used them.

Audit fixes, each with a vector or a broken package where one was
missing:

- Folder packages: symbolic links are refused on validate and load,
  manifest paths must not collide under case folding, listed files are
  matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
  a lone surrogate or invalid UTF-8 is refused instead of translated;
  integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
  signed; a qa receipt's nodes must be the sample at the original's
  hashes, its scope must name qa.of, its errors must be sampled nodes;
  statuses are computed once, iteratively, so chains of spot-checks
  hundreds deep settle; Base64 and manifest shape errors become refusals
  instead of tracebacks; counts and the fidelity of roster-less packages
  are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
  validated against the node schema and the references between nodes
  are checked, so a set to a bad value, an insert with unknown
  properties or a delete of a referenced header cell is refused; a
  relabel to null drops the label; a reclassify drops foreign
  attributes; a split strips whitespace across runs; ids.jsonl order
  follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
  node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
  ({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
  the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
  "10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
  computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
  revocation may be carried by several packages.

Checking:

- tools/stress.py: seeded randomized property tests over canonical JSON,
  normalization, notation, random correction walks, anchor resolution,
  Merkle roots, spot-check samples, error bounds, fingerprints, mutated
  archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
  the repository, the zipped fixture and crafted archives, the checklist
  coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-10-14 20:07:22 -04:00

11 KiB

Raw Blame History

Glossary

A scientific paper changes after it is published: errata appear, readings get corrected, annotations pile up. manuspec is a set of file formats for scientific papers that keeps each sentence findable through those changes (repository: https://git.scient.ing/infra/manuspec). This glossary defines its terms for readers from newcomer to expert, implementers included. Each entry gives the plain meaning, then the precise one, then the chapter that defines it.

A hash is a fixed-length fingerprint of data; changing one character changes it. SHA-256 is the hash function the formats use. The specification is at version 0.1.0, and each format is candidate or provisional in registry/formats.json. Format names begin with scienting-, as in scienting-doc/1; that prefix is the formats' namespace, a separate matter from the repository's name. "Outside term" marks a standard from another project.

Address: A string that names one node, optionally pinned to its state by the first six digits of its node_hash and narrowed to a span of code points (numbered characters of text): key#id@hash6[start:end]. (6.1)
Alias: A legacy ID kept so that old references still lead to the node that now holds their text. (4.3)
Align key: A hash of a node's lossy projection, used only to propose that two readings hold the same sentence; it never certifies anything. (5.1)
Anchor: The stored form of a pinned address, with its quotation and context, so that a reference can be found again after the text changes. It resolves as current, changed below, moved, reattached or stale. (6.2, 6.5)
Asset store: Where a team keeps files by SHA-256: witnesses, images, crops and withheld text. (12.6)
Bundle: One SQLite file with every table and a full-text index, which a reader or a store queries. Format scienting-bundle/1, provisional. (13.5)
Changed below: The resolution of an anchor on a container that is unchanged in itself while something inside it changed. (6.5)
Checklist: A record of checking an export, such as accessible HTML, against a named standard, criterion by criterion; it is covered by the receipt of the person who checked. Format scienting-checklist/1, provisional. (8.11)
Correction: An operation on an established document, with its evidence and author; never edited, only appended. (11)
Description: An annotation that describes a figure, panel or table for readers who cannot see it, as a short text equivalent (alt) or a full description (long); an accessible export uses confirmed descriptions only. (18.4)
Document: The text established for one version of one work, from one or more witnesses, as a tree of nodes. (2.1, 3)
Drift: The amount that has changed below a container since an anchor on it was made, counted from the corrections: the share of the container's leaves touched by later corrections. It is reported as none, low or high, with a default threshold of a quarter. (6.6)
Exercise: An annotation whose body holds a prompt, an answer key and a grading rule. The key is anchors, a verdict, a chain, a value or planted errors, by exercise kind. (19.1)
Fidelity level: How far a leaf's text was checked against its source, from F0 converted to F3 proofread. (8.1)
JATS: Outside term. Journal Article Tag Suite, ANSI/NISO Z39.96-2024, an XML format in which publishers and archives exchange journal content (the NISO page). In manuspec, jats is a witness kind: the fixture okafor2025open was established from a JATS file and a PDF that agree. The formats ingest JATS and do not replace it. (2.3)
Keyed hash: A content value computed as an HMAC (a hash mixed with a secret key) under a team key or a key derived from page fingerprints, so a short text cannot be recovered by hashing guesses. (5.8)
Layer: A named set of annotations with one purpose, audience, visibility and license. (18.2)
Leaf: A node that holds text itself: a sentence, cell, reference, author line, keyword line, equation, equation line, code or algorithm. A leaf anchor points at one whole. (6.4)
Lossy projection: A node's text with scripts flattened, citation marks, whitespace and dashes removed, and case folded; used for proposing alignments. (5.5)
Match hash: A hash of a node's strict projection. Two readings agree only when their match hashes are equal and their complex-notation keys are equal. A complex-notation key is notation:source for a piece of math too complex for the formats to normalize, with the source's whitespace removed (5.3). (5.1, 5.3)
Node: An addressable part of a document: a section, paragraph, sentence, figure, panel, caption, table, row, cell, equation, footnote, reference and the rest. Each node has a permanent ID and five content values; table columns and figure-text items have IDs and are not nodes. (3, 4.2)
Own hash: A hash of a node's own content without its children, used to judge anchors on containers that have content of their own. The node_hash is a hash of the own hash and the children's node hashes. (5.1, 5.2)
Package: A document as a folder, or a zip of one, of canonical JSON (a plain-text data format) and JSON Lines (one JSON value per line) files, with a manifest whose hash pins them all. (12)
Page fingerprint: A hash of a page's size, its text with download stamps removed, and its images, so that differently stamped downloads share their pages. (7.1)
Plain projection: A node's text as one string, on which offsets, quotations and text hashes are defined. (3.6)
Policy: A project owner's signed decision that settles rights a record leaves unknown. It never overrides a use the record sets to true or false. (14.2)
Project: A team's curated set of documents, with its members, layers, data, reading paths and drafts. Format scienting-project/1, provisional. (20)
Receipt: One signature over a Merkle root of what a person reviewed in a sitting. The signature sets the fidelity level of what it lists. A Merkle root is a single hash that commits to a list of items; receipts build it as RFC 6962 (an internet standards document) describes, a tree that RFC 9162 obsoletes and defines identically. (8.3, 8.5)
Revision: The index of the last correction applied to a document, or 0 when there is none; anchors and receipts record it. (11)
Revocation: A signed record that withdraws a receipt, or every receipt by one signer in a period. (8.7)
Rights record: A witness's license, the basis on which the team holds it, and the uses that basis allows. (14.1)
Roster: The list of keys that may sign, with the person who holds each, that person's role in the team, and what the key may sign. (8.7)
Selector: Outside term. In the W3C Web Annotation Data Model, the part of an annotation's target that says where in the source resource the annotation applies. The model defines TextQuoteSelector (exact text, with optional prefix and suffix) and TextPositionSelector (offsets, which the model calls very brittle), among others (the W3C model). Layers use a custom AnchorSelector, which carries an anchor, followed by a TextQuoteSelector for tools that do not know it; they use neither TextPositionSelector nor FragmentSelector. (18.3)
Spot-check: A second reviewer's check of a sample of a receipt's nodes, drawn from a seed nobody can steer: the checker's deterministic signature over the receipt's record hash. (8.8)
Stamp rule: A pattern that recognizes a publisher's download stamp in a page's text, so that fingerprints ignore it. (7.2)
Strict projection: A node's text with notation folded to one canonical form and structure kept; used for certifying agreement. (5.3)
Suspect: A doubtful reading raised for a person or a review session to answer. (5.7)
Targeted F3: Proofreading the nodes a team's claims, quotations and datasets rest on, rather than every node. The target set also includes the leaves of every anchored container, every equation and, under the project's digit rule (all, anchored-sections or none), sentences and cells with a digit in a text or math run (an inline stretch of plain text, or of TeX or Typst math, within the node; 3.5). (8.10)
Text leaf: A leaf whose text a container's projections are made of: any leaf except an equation line, or an equation inside a sentence, whose text is already part of its equation or its sentence. (3.6)
Text policy: Whether a package carries its text (full) or withholds it (checks-only). (12.5)
Verdict: A judgment of whether a cited passage supports a citing sentence: supports, partly, does not, or cannot tell. (19.2)
Version: One expression of a work: a preprint, the version of record, an erratum, the Supporting Information. (2.1)
Web Annotation: Outside term. The W3C Recommendation of 23 February 2017 for annotations with a body, a target and a motivation (the W3C model). It gives sentences no permanent IDs and reports no outcomes such as current or stale; for annotation in general, it and its clients are the better choice. Layers are Web Annotations. (18)
Witness: One file of one version of a work, identified by its SHA-256. (2.1, 2.3)
Work: A piece of scholarship across all its versions. (2.1)

11 KiB Raw Blame History

Glossary

11 KiB

Raw Blame History