0 Fork 0

Files

T

feat: zipped packages, target set, checklists, audit fixes and a stress test

Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):

- Zipped packages: the oracle writes and reads `.sct.zip` archives under
  the rules of spec/12 §12.2; validate.py and load.py accept them; a
  byte-identical zip of the tiny package joins the fixtures, and two
  broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
  oracle and pinned by vectors/targets.jsonl under each digit rule; the
  rule now covers cells as well as sentences and the leaves of anchored
  containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
  covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
  WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
  accessible export can find a confirmed figure description; the layer
  fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
  used them.

Audit fixes, each with a vector or a broken package where one was
missing:

- Folder packages: symbolic links are refused on validate and load,
  manifest paths must not collide under case folding, listed files are
  matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
  a lone surrogate or invalid UTF-8 is refused instead of translated;
  integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
  signed; a qa receipt's nodes must be the sample at the original's
  hashes, its scope must name qa.of, its errors must be sampled nodes;
  statuses are computed once, iteratively, so chains of spot-checks
  hundreds deep settle; Base64 and manifest shape errors become refusals
  instead of tracebacks; counts and the fidelity of roster-less packages
  are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
  validated against the node schema and the references between nodes
  are checked, so a set to a bad value, an insert with unknown
  properties or a delete of a referenced header cell is refused; a
  relabel to null drops the label; a reclassify drops foreign
  attributes; a split strips whitespace across runs; ids.jsonl order
  follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
  node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
  ({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
  the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
  "10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
  computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
  revocation may be carried by several packages.

Checking:

- tools/stress.py: seeded randomized property tests over canonical JSON,
  normalization, notation, random correction walks, anchor resolution,
  Merkle roots, spot-check samples, error bounds, fingerprints, mutated
  archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
  the repository, the zipped fixture and crafted archives, the checklist
  coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-10-14 20:07:22 -04:00

5.1 KiB

Raw Blame History

18. Layers and annotations

Everything people and models add to a document is an annotation, and annotations are grouped in layers. scienting-layer/1 is the W3C Web Annotation Data Model with the anchor as a selector, so any Web Annotation tool can read it. The JSON-LD context is context/scienting-1.jsonld, published at https://scient.ing/ns/scienting/1.jsonld.

18.1 One rule decides what is a claim and what is a layer

If a statement is true independently of this paper, it belongs in a claim store as a claim, where it is checked and leveled. If it is about reading this paper, it belongs in a layer, where it is attributed and signed but not checked.

18.2 The layer

A layer is a folder with layer.json and annotations.jsonl. layer.json is an AnnotationCollection with the two contexts, format, an id of the form urn:scienting:layer:<store>/<record id>, a label, a purpose, an audience (novice, student, expert or mixed), a visibility (private, team or public), a license (CC BY 4.0 by default when public), its creator, created, its project, the docs it annotates and its total. A reader turns layers on and off, and the audience dial picks the level of glosses.

18.3 The annotation

Each line of annotations.jsonl is a W3C Annotation:

Property	Holds
`@context`	`["http://www.w3.org/ns/anno.jsonld", "https://scient.ing/ns/scienting/1.jsonld"]`.
`id`	`urn:scienting:annotation:<store>/<record id>` (4.6).
`type`, `layer`	`Annotation`, and the layer's ID.
`motivation`	A W3C motivation: `commenting`, `describing`, `linking`, `questioning`, `assessing`, `replying` and the others.
`creator`, `created`, `modified`	An agent `{id, type, name}`, with `type` `Person` or `Software`, and times.
`body`	One body (18.4).
`target`	A `SpecificResource` whose `source` is `urn:scienting:doc:<store>/<key>` and whose `selector` lists an `AnchorSelector` holding a `scienting-anchor/1`, optionally followed by a W3C `TextQuoteSelector` for tools that do not know ours; or, for a reply, a `SpecificResource` whose source is another annotation.
`status`	`proposed`, `confirmed`, `rejected` or `withdrawn`. A model's annotation is `proposed` until a person confirms it.
`receipt`	For a confirmed annotation, the receipt that covers it.

The TextQuoteSelector, when present, repeats the anchor quote's exact, prefix and suffix, with an empty string for a prefix or suffix the quote lacks. It is present only when the anchor's quote is plain text; an anchor whose quote is hashed or keyed (6.3) has none, so the fallback never shows text the anchor withholds.

An annotation's content hash is SHA-256 of the canonical JSON of the annotation without status, receipt and modified. A confirmed annotation is covered by its confirmer's receipt, which lists ["annotation", <record id>, <content hash>]. Replies are annotations that target annotations, so discussions are threads anchored to sentences.

18.4 Bodies

`kind`	W3C type	Adds	Says
`gloss`	`TextualBody`	`audience`	An explanation at `novice`, `student` or `expert` level.
`claim-link`	`SpecificResource`	`source` (`lode:<tag>`), `pin`, `role`	That the span `states`, `uses` or `disputes` a claim.
`symbol`	`SpecificResource`	`source`, `pin`, `symbol`	That this occurrence of a symbol is a symbol claim.
`comment`	`TextualBody`		A remark, opinion or interpretation.
`caveat`	`TextualBody`		A warning a reader should see.
`question`	`TextualBody`	`for`	A question for a `reader`, a `group` or the `authors`.
`exercise`	`TextualBody`	`exercise`	A task with a key and a rule (19).
`citation-context`	`TextualBody`	`reference`, `verdict`, `reason`	Why a sentence cites a reference, and whether it supports it.
`prerequisite`	`SpecificResource`	`source`	What to read or know first.
`crosslink`	`SpecificResource`	`source`, `selector`	That the span corresponds to a span in another document.
`translation`	`TextualBody`	`language`	The span in another language.
`description`	`TextualBody`	`role`	A description of a figure, panel or table for readers who cannot see it: `alt`, a short text equivalent, or `long`, a full description.
`generated`	`TextualBody`	`generator`	Text a model wrote about the span; never mistaken for source.

A textual body has a value, and optionally a format (text/plain or text/markdown) and a language. Nothing a model writes is ever stored as source text. A model may propose a description, and like any annotation it stays proposed until a person confirms it; an accessible export uses confirmed descriptions only, and a figure with none has no description. Other text a model writes about a span, such as a summary, is a generated annotation, which names its generator. fixtures/records/layer/ holds a layer with one annotation of each common kind and the receipt that confirms some of them.

5.1 KiB Raw Blame History