paperspec/schemas/README.md

Schemas

This folder holds the JSON (JavaScript Object Notation) Schemas (draft 2020-12) for every file and record that the manuspec formats (manuspec is the specification this repository holds) define: 39 schema files. A JSON Schema is a file, itself written in JSON, that states what a valid JSON file looks like. Newcomers can read it as a form: it lists the fields a record must have and what kind of value each field takes. Implementers will find the loading recipe and the traps under "Validating" and "Writing schemas". Experts will find the chapter and status of each format under "Which schema for which file" and "What a schema does not check".

Each schema's $id, its own address as a name, is https://scient.ing/ns/schemas/<file>. Schemas refer to each other by those names, and a validator (a program that checks a file against a schema) that loads the whole folder resolves every reference. The $id base is normative and stays as it is. The scienting- prefix on format identifiers such as scienting-anchor/1 is the formats' namespace, a separate matter from the repository's name.

Which schema for which file

The schemas are grouped by topic below. The number after each name is the chapter of spec/ that defines the file. common.schema.json is defined in spec/01-conventions.md, the chapter on rules that apply to every format.

Topic	Schemas (chapter)
Shared definitions	common.schema.json (1): hashes, IDs, keys, addresses, times, decimals, boxes, licenses, signatures, evidence, quotes.
A package	manifest.schema.json (12), document.schema.json (2, 12), node.schema.json (3), runs.schema.json (3), witnesses.schema.json (2), provenance.schema.json (12), page.schema.json (7), location.schema.json (7), reference.schema.json (10), id-entry.schema.json (4), correction.schema.json (11), receipt.schema.json (8), revocation.schema.json (8), roster.schema.json (8), suspect.schema.json (5), mathml.schema.json (12)
Pointing into documents	anchor.schema.json (6)
Rights	rights.schema.json (14), policy.schema.json (14), stages.schema.json (14), backend.schema.json (14), profile.schema.json (14), stamps.schema.json (7)
Review	qa-report.schema.json (8), spot-check.schema.json (8), checklist.schema.json (8)
Annotation and teaching	layer.schema.json (18), annotation.schema.json (18), exercise.schema.json (19), submission.schema.json (19), grade.schema.json (19)
Data	dataset.schema.json (15), value-anchor.schema.json (15), digitization.schema.json (16), view.schema.json (17)
Teams and drafts	project.schema.json (20), snapshot.schema.json (21), draft-ids.schema.json (21)

A record that travels on its own carries a format such as scienting-anchor/1. registry/formats.json maps each format to its schema, its chapter and its status.

The formats registry lists 29 formats: 18 are candidate (complete, with vectors and fixtures, ready to implement) and 11 are provisional (complete; expected to change as implementations gain experience), as defined in spec/00-introduction.md §0.4. The provisional formats are scienting-project/1, scienting-bundle/1, scienting-dataset/1, scienting-digitization/1, scienting-view/1, scienting-snapshot/1, scienting-draft-ids/1, scienting-exercise/1, scienting-submission/1, scienting-grade/1 and scienting-checklist/1. The Data schemas for datasets, digitizations and views, the Teams and drafts schemas, the three teaching schemas and checklist.schema.json belong to provisional formats. The rest belong to candidate formats or to no format of their own.

The bundle has no JSON Schema: its registry entry points to sql/bundle-sqlite.sql. Of the 39 schemas, 11 have no format of their own because each describes a line or part of a larger file: common, node, runs, witnesses, provenance, location, reference, id-entry, mathml, annotation and value-anchor. Of the schemas that do have one, 25 require a format property; spot-check.schema.json describes a signing payload rather than a file. The others (page, draft-ids and view) describe a line of a file or a Vega-Lite specification (a JSON description of a chart), and the record itself does not need to carry the identifier.

Validating

The example below uses the jsonschema package (4.18 or later).

import json, pathlib
from jsonschema import Draft202012Validator
from referencing import Registry, Resource

schemas = {p.name: json.loads(p.read_text()) for p in pathlib.Path("schemas").glob("*.json")}
registry = Registry().with_resources((s["$id"], Resource.from_contents(s)) for s in schemas.values())
validator = Draft202012Validator(schemas["anchor.schema.json"], registry=registry)
validator.validate(json.load(open("fixtures/records/anchor-span.json")))

It raises no error when run from the repository root, since anchor-span.json is a valid anchor. A schema refers to another by a relative name, and the Registry object resolves it. Here anchor.schema.json points to common.schema.json#/$defs/key, which a validator resolves against the $id above into https://scient.ing/ns/schemas/common.schema.json. The Registry maps that name to the file's contents, so nothing is fetched over the network. Draft202012Validator(schemas["anchor.schema.json"]) fails with Unresolvable: common.schema.json#/$defs/key when the Registry is missing. Never load a schema alone.

Any draft 2020-12 validator works the same way: give it every schema under its $id, then validate against the one you need. python3 tools/check.py validates every fixture with jsonschema this way, and checks that every schema keeps to the keyword subset.

What a schema does not check

A schema checks shape: required properties, types, patterns, enumerations. The anchor in fixtures/records/anchor-span.json has a node_hash (a fingerprint of the node's content) of 64 lowercase hex digits, and a schema can confirm exactly that. It cannot confirm that the hash is the right one: replace it with 64 zeros and the record still validates. It also cannot check that a node's children are allowed or that a receipt's signature verifies, nor that a correction's ID derives from its content. The rules of spec/22-conformance.md cover these checks, each with an ID in registry/rules.json. For the four examples the IDs are doc.hash.node-hash, doc.tree.child, doc.correction.id and doc.receipt.signature.

A validator that reports package.schema has found only a shape error. The other rules need the chapters' computations, and the test vectors in vectors/ (for example hashes.jsonl, corrections.jsonl and signatures.jsonl) give the expected outputs. The schemas are normative, and conformance has further steps: spec/22 §22.2 lists them in order.

Writing schemas

Validators in many languages read the same schemas, which limits how the schemas are written. The schemas use only the keywords listed in spec/22-conformance.md. Their patterns avoid \d, \w, \s, \b and lookbehind, so validators in every language read them alike. The repository's own validator refuses a schema that uses any other keyword or those pattern tokens. Optional properties are omitted rather than set to null, except where a schema says a property is nullable.