89 lines
7.6 KiB
Markdown
89 lines
7.6 KiB
Markdown
# Schemas
|
|
|
|
This folder holds the JSON (JavaScript Object Notation) Schemas (draft 2020-12) for every file and record that the paperspec formats (paperspec is the specification this repository holds) define: 42 schema files.
|
|
A JSON Schema is a file, itself written in JSON, that states what a valid JSON file looks like.
|
|
Newcomers can read it as a form: it lists the fields a record must have and what kind of value each field takes.
|
|
Implementers will find the loading recipe and the traps under "Validating" and "Writing schemas".
|
|
Experts will find the chapter and status of each format under "Which schema for which file" and "What a schema does not check".
|
|
|
|
Each schema's `$id`, its own address as a name, is `https://paperspec.scient.ing/schemas/<file>`.
|
|
Schemas refer to each other by those names, and a validator (a program that checks a file against a schema) that loads the whole folder resolves every reference.
|
|
The `$id` base is normative and stays as it is.
|
|
The `scienting-` prefix on format identifiers such as `scienting-anchor/1` is the formats' namespace, a separate matter from the repository's name.
|
|
|
|
## Which schema for which file
|
|
|
|
The schemas are grouped by topic below.
|
|
The number after each name is the chapter of `spec/` that defines the file.
|
|
`common.schema.json` is defined in `spec/01-conventions.md`, the chapter on rules that apply to every format.
|
|
|
|
| Topic | Schemas (chapter) |
|
|
| --- | --- |
|
|
| Shared definitions | `common.schema.json` (1): hashes, IDs, keys, addresses, times, decimals, boxes, licenses, signatures, evidence, quotes. |
|
|
| A package | `manifest.schema.json` (12), `document.schema.json` (2, 12), `node.schema.json` (3), `runs.schema.json` (3), `witnesses.schema.json` (2), `provenance.schema.json` (12), `page.schema.json` (7), `location.schema.json` (7), `reference.schema.json` (10), `id-entry.schema.json` (4), `correction.schema.json` (11), `receipt.schema.json` (8), `revocation.schema.json` (8), `roster.schema.json` (8), `suspect.schema.json` (5), `mathml.schema.json` (12) |
|
|
| Pointing into documents | `anchor.schema.json` (6) |
|
|
| Rights | `rights.schema.json` (14), `policy.schema.json` (14), `stages.schema.json` (14), `backend.schema.json` (14), `profile.schema.json` (14), `stamps.schema.json` (7) |
|
|
| Review | `qa-report.schema.json` (8), `spot-check.schema.json` (8), `checklist.schema.json` (8), `proof.schema.json` (8, 11) |
|
|
| The transparency log | `log-entry.schema.json` (23), `checkpoint.schema.json` (23) |
|
|
| Annotation and teaching | `layer.schema.json` (18), `annotation.schema.json` (18), `exercise.schema.json` (19), `submission.schema.json` (19), `grade.schema.json` (19) |
|
|
| Data | `dataset.schema.json` (15), `value-anchor.schema.json` (15), `digitization.schema.json` (16), `view.schema.json` (17) |
|
|
| Teams and drafts | `project.schema.json` (20), `snapshot.schema.json` (21), `draft-ids.schema.json` (21) |
|
|
|
|
A record that travels on its own carries a `format` such as `scienting-anchor/1`. [registry/formats.json](../registry/formats.json) maps each format to its schema, its chapter and its status.
|
|
|
|
The formats registry lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience), as defined in `spec/00-introduction.md` §0.4.
|
|
The provisional formats are `scienting-project/1`, `scienting-bundle/1`, `scienting-dataset/1`, `scienting-digitization/1`, `scienting-view/1`, `scienting-snapshot/1`, `scienting-draft-ids/1`, `scienting-exercise/1`, `scienting-submission/1`, `scienting-grade/1`, `scienting-checklist/1`, `scienting-log-entry/1` and `scienting-checkpoint/1`.
|
|
The Data schemas for datasets, digitizations and views, the Teams and drafts schemas, the three teaching schemas, `checklist.schema.json` and the two log schemas belong to provisional formats.
|
|
The rest belong to `candidate` formats or to no format of their own.
|
|
|
|
The bundle has no JSON Schema: its registry entry points to `sql/bundle-sqlite.sql`.
|
|
Of the 42 schemas, 11 have no format of their own because each describes a line or part of a larger file: `common`, `node`, `runs`, `witnesses`, `provenance`, `location`, `reference`, `id-entry`, `mathml`, `annotation` and `value-anchor`.
|
|
Of the schemas that do have one, 27 require a `format` property; `spot-check.schema.json` describes the input a VRF proof is made over rather than a file.
|
|
The others (`page`, `draft-ids`, `view` and `log-entry`) describe a line of a file, a Vega-Lite specification (a JSON description of a chart) or an item of a tree, and the record itself does not need to carry the identifier.
|
|
|
|
## Validating
|
|
|
|
The example below uses the `jsonschema` package (4.18 or later).
|
|
|
|
```python
|
|
import json, pathlib
|
|
from jsonschema import Draft202012Validator
|
|
from referencing import Registry, Resource
|
|
|
|
schemas = {p.name: json.loads(p.read_text()) for p in pathlib.Path("schemas").glob("*.json")}
|
|
registry = Registry().with_resources((s["$id"], Resource.from_contents(s)) for s in schemas.values())
|
|
validator = Draft202012Validator(schemas["anchor.schema.json"], registry=registry)
|
|
validator.validate(json.load(open("fixtures/records/anchor-span.json")))
|
|
```
|
|
|
|
It raises no error when run from the repository root, since `anchor-span.json` is a valid anchor.
|
|
A schema refers to another by a relative name, and the `Registry` object resolves it.
|
|
Here `anchor.schema.json` points to `common.schema.json#/$defs/key`, which a validator resolves against the `$id` above into `https://paperspec.scient.ing/schemas/common.schema.json`.
|
|
The `Registry` maps that name to the file's contents, so nothing is fetched over the network.
|
|
`Draft202012Validator(schemas["anchor.schema.json"])` fails with `Unresolvable: common.schema.json#/$defs/key` when the `Registry` is missing.
|
|
Never load a schema alone.
|
|
|
|
Any draft 2020-12 validator works the same way: give it every schema under its `$id`, then validate against the one you need.
|
|
`paperspec check` validates every fixture with `jsonschema` this way, and checks that every schema keeps to the keyword subset.
|
|
|
|
## What a schema does not check
|
|
|
|
A schema checks shape: required properties, types, patterns, enumerations.
|
|
The anchor in `fixtures/records/anchor-span.json` has a `node_hash` (a fingerprint of the node's content) of 64 lowercase hex digits, and a schema can confirm exactly that.
|
|
It cannot confirm that the hash is the right one: replace it with 64 zeros and the record still validates.
|
|
It also cannot check that a node's children are allowed or that a receipt's signature verifies, nor that a correction's ID derives from its content.
|
|
The rules of [spec/22-conformance.md](../spec/22-conformance.md) cover these checks, each with an ID in [registry/rules.json](../registry/rules.json).
|
|
For the four examples the IDs are `doc.hash.node-hash`, `doc.tree.child`, `doc.correction.id` and `doc.receipt.signature`.
|
|
|
|
A validator that reports `package.schema` has found only a shape error.
|
|
The other rules need the chapters' computations, and the test vectors in `vectors/` (for example `hashes.jsonl`, `corrections.jsonl` and `signatures.jsonl`) give the expected outputs.
|
|
The schemas are normative, and conformance has further steps: spec/22 §22.2 lists them in order.
|
|
|
|
## Writing schemas
|
|
|
|
Validators in many languages read the same schemas, which limits how the schemas are written.
|
|
The schemas use only the keywords listed in [spec/22-conformance.md](../spec/22-conformance.md#223-schemas).
|
|
Their patterns avoid `\d`, `\w`, `\s`, `\b` and lookbehind, so validators in every language read them alike.
|
|
The repository's own validator refuses a schema that uses any other keyword or those pattern tokens.
|
|
Optional properties are omitted rather than set to null, except where a schema says a property is nullable.
|