Files
paperspec/sql/store.sql
T
alexmandClaude ea0a7efa78 feat: zipped packages, target set, checklists, audit fixes and a stress test
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):

- Zipped packages: the oracle writes and reads `.sct.zip` archives under
  the rules of spec/12 §12.2; validate.py and load.py accept them; a
  byte-identical zip of the tiny package joins the fixtures, and two
  broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
  oracle and pinned by vectors/targets.jsonl under each digit rule; the
  rule now covers cells as well as sentences and the leaves of anchored
  containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
  covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
  WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
  accessible export can find a confirmed figure description; the layer
  fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
  used them.

Audit fixes, each with a vector or a broken package where one was
missing:

- Folder packages: symbolic links are refused on validate and load,
  manifest paths must not collide under case folding, listed files are
  matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
  a lone surrogate or invalid UTF-8 is refused instead of translated;
  integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
  signed; a qa receipt's nodes must be the sample at the original's
  hashes, its scope must name qa.of, its errors must be sampled nodes;
  statuses are computed once, iteratively, so chains of spot-checks
  hundreds deep settle; Base64 and manifest shape errors become refusals
  instead of tracebacks; counts and the fidelity of roster-less packages
  are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
  validated against the node schema and the references between nodes
  are checked, so a set to a bad value, an insert with unknown
  properties or a delete of a referenced header cell is refused; a
  relabel to null drops the label; a reclassify drops foreign
  attributes; a split strips whitespace across runs; ids.jsonl order
  follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
  node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
  ({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
  the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
  "10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
  computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
  revocation may be carried by several packages.

Checking:

- tools/stress.py: seeded randomized property tests over canonical JSON,
  normalization, notation, random correction walks, anchor resolution,
  Merkle roots, spot-check samples, error bounds, fingerprints, mutated
  archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
  the repository, the zipped fixture and crafted archives, the checklist
  coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-14 20:07:22 -04:00

155 lines
4.3 KiB
SQL

-- The team store: everything a team adds to its documents.
-- Normative: spec/13-database.md. Load sql/doc.sql first.
-- Portable to SQLite 3.37+, Dolt and PostgreSQL 12+, under the same rules as sql/doc.sql.
-- License: CC0-1.0.
CREATE TABLE roster_keys (
store VARCHAR(128) NOT NULL,
id VARCHAR(128) NOT NULL,
person TEXT NOT NULL,
role VARCHAR(16) NOT NULL CHECK (role IN ('owner', 'editor', 'reader', 'guest')),
scheme VARCHAR(16) NOT NULL CHECK (scheme IN ('ed25519', 'ssh', 'gpg', 'sigstore')),
public_key TEXT NOT NULL,
purposes TEXT NOT NULL,
registered_at VARCHAR(20) NOT NULL,
revoked_at VARCHAR(20),
PRIMARY KEY (store, id)
);
CREATE TABLE projects (
project_key VARCHAR(128) NOT NULL,
store VARCHAR(128),
name TEXT NOT NULL,
description TEXT,
claims TEXT,
settings TEXT,
record TEXT NOT NULL,
PRIMARY KEY (project_key)
);
CREATE TABLE members (
project_key VARCHAR(128) NOT NULL,
person VARCHAR(256) NOT NULL,
role VARCHAR(8) NOT NULL CHECK (role IN ('owner', 'editor', 'reader', 'guest')),
signing_keys TEXT NOT NULL,
PRIMARY KEY (project_key, person)
);
CREATE TABLE project_documents (
project_key VARCHAR(128) NOT NULL,
doc_key VARCHAR(257) NOT NULL,
reason TEXT NOT NULL,
tags TEXT,
reading_status TEXT,
PRIMARY KEY (project_key, doc_key)
);
CREATE TABLE reading_paths (
project_key VARCHAR(128) NOT NULL,
id VARCHAR(128) NOT NULL,
title TEXT NOT NULL,
steps TEXT NOT NULL,
PRIMARY KEY (project_key, id)
);
CREATE TABLE layers (
id VARCHAR(256) NOT NULL,
project_key VARCHAR(128),
label TEXT NOT NULL,
purpose TEXT NOT NULL,
audience VARCHAR(8) CHECK (audience IN ('novice', 'student', 'expert', 'mixed')),
visibility VARCHAR(8) NOT NULL CHECK (visibility IN ('private', 'team', 'public')),
license TEXT NOT NULL,
record TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE annotations (
id VARCHAR(256) NOT NULL,
layer VARCHAR(256) NOT NULL,
body_kind VARCHAR(24) NOT NULL CHECK (body_kind IN ('gloss', 'claim-link', 'symbol', 'comment', 'caveat', 'question', 'exercise', 'citation-context', 'prerequisite', 'crosslink', 'translation', 'description', 'generated')),
status VARCHAR(10) NOT NULL CHECK (status IN ('proposed', 'confirmed', 'rejected', 'withdrawn')),
receipt VARCHAR(32),
creator VARCHAR(256) NOT NULL,
created VARCHAR(20) NOT NULL,
record TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE INDEX annotations_layer ON annotations (layer);
-- Derived from annotation targets, so a reader finds the annotations on a node.
CREATE TABLE annotation_targets (
annotation VARCHAR(256) NOT NULL,
ord INTEGER NOT NULL,
doc_key VARCHAR(257),
node VARCHAR(128),
target_annotation VARCHAR(256),
PRIMARY KEY (annotation, ord)
);
CREATE INDEX annotation_targets_node ON annotation_targets (doc_key, node);
CREATE TABLE datasets (
id VARCHAR(128) NOT NULL,
doc_key VARCHAR(257),
content_hash VARCHAR(64) NOT NULL,
descriptor TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE dataset_values (
dataset VARCHAR(128) NOT NULL,
resource VARCHAR(128) NOT NULL,
row_key VARCHAR(128) NOT NULL,
field VARCHAR(128) NOT NULL,
printed VARCHAR(64),
origin_kind VARCHAR(10) NOT NULL CHECK (origin_kind IN ('reported', 'digitized', 'computed', 'measured')),
anchor TEXT,
PRIMARY KEY (dataset, resource, row_key, field)
);
CREATE TABLE digitizations (
id VARCHAR(32) NOT NULL,
doc_key VARCHAR(257) NOT NULL,
panel VARCHAR(16) NOT NULL,
page_fp VARCHAR(64),
method VARCHAR(8) NOT NULL CHECK (method IN ('manual', 'traced', 'proposed')),
confirmation VARCHAR(32),
dataset VARCHAR(128),
record TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE views (
id VARCHAR(128) NOT NULL,
project_key VARCHAR(128),
spec TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE policies (
id VARCHAR(32) NOT NULL,
project_key VARCHAR(128),
signer VARCHAR(128) NOT NULL,
made_at VARCHAR(20) NOT NULL,
valid_until VARCHAR(20),
record TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE submissions (
id VARCHAR(32) NOT NULL,
exercise VARCHAR(256) NOT NULL,
student VARCHAR(64) NOT NULL,
submitted_at VARCHAR(20) NOT NULL,
record TEXT NOT NULL,
PRIMARY KEY (id)
);
CREATE TABLE grades (
submission VARCHAR(32) NOT NULL,
score VARCHAR(32) NOT NULL,
max_score VARCHAR(32) NOT NULL,
record TEXT NOT NULL,
PRIMARY KEY (submission)
);