# 0. Introduction

## 0.1 Scope

These specifications define the data that tools for reading, correcting, annotating and checking scientific papers exchange: documents and their packages, anchors, receipts, rights records, layers, exercises, datasets, digitizations, views, projects and bundles.
They define data, not programs.
A conforming file can be read, checked and loaded without any particular program, and every rule is stated so that a second implementation, in any language, computes the same bytes.

The [guides](../docs/README.md) explain what each format is for, and the [rationale](../docs/rationale.md) why its rules are what they are.

## 0.2 Conformance language

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY and OPTIONAL are to be interpreted as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.

Three classes of implementation are distinguished.

| Class | Does | Must |
| --- | --- | --- |
| Producer | Writes files in a format. | Write only files that validate, with values computed as specified. |
| Consumer | Reads files in a format. | Accept every valid file, and refuse every file the suite lists as invalid, naming the rule. |
| Validator | Decides whether a file is valid. | Check every rule in [22](22-conformance.md) that the file's contents allow it to check. |

An implementation states, for each format, which versions it reads and writes.

## 0.3 Format identifiers and versions

Every format has an identifier of the form `scienting-<name>/<major>`, such as `scienting-doc/1`.
A file or record that can travel on its own carries its identifier in a `format` property.
Records inside a package file whose format the manifest fixes, such as the lines of `nodes.jsonl`, do not repeat it.

A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version, with a migration.
Adding an optional property, a node kind, a run kind or an enumeration value is a minor change.
A consumer of version *n* MUST refuse a major version it does not know, and SHOULD warn on, but accept, enumeration values added in a later minor version.

The specification as a whole is versioned with semantic versioning; this is version 0.1.0.
[registry/formats.json](../registry/formats.json) lists every format with its media type, schema and status.

## 0.4 Status

Each format has one of three statuses, listed in [registry/formats.json](../registry/formats.json).

| Status | Meaning |
| --- | --- |
| `candidate` | Complete, with vectors and fixtures, and ready to implement. |
| `provisional` | Complete, but expected to change as implementations gain experience with it. |
| `reserved` | Named and shaped, but not yet specified in full. |

Some fields matter only to some users: a personal library has no use for spot-checks, and a team that reads only open-access papers rarely needs a policy.
Those fields are defined all the same, so that files move between a personal library and a team's store without conversion.
A consumer that does not use a field MUST still preserve it when it rewrites a file.

## 0.5 How the repository is arranged

| Path | Holds | Normative |
| --- | --- | --- |
| `spec/` | These chapters. | Yes |
| `schemas/` | JSON Schemas, draft 2020-12. | Yes |
| `sql/` | Table definitions for SQLite, Dolt and PostgreSQL, and the SQLite bundle. | Yes |
| `context/` | The JSON-LD context for layers. | Yes |
| `registry/` | Formats, rule IDs, stamp rules and the default stage requirements. | Yes |
| `vectors/` | Test vectors, one case per line. | Yes |
| `fixtures/` | Valid packages and records, invalid packages, and the keys that signed them. | Yes |
| `tools/` | The programs that compute and check the vectors and fixtures, and load packages into SQLite. | No |
| `docs/` | Guides for people getting to know the formats. | No |

The programs in `tools/` compute every expected value, so that none is typed by hand.
They are not a library: implementations conform to the chapters and the vectors, and `scienting-core` is the reference implementation of the shared rules.