paperspec/spec/00-introduction.md

0. Introduction

This chapter is for three readers. A newcomer, who knows what a scientific paper and a citation are, finds what the specification covers and how to read it. An implementer, who will write a reader, writer, converter or checker, finds the rules for conformance, identifiers, versions and status. An expert, who knows the neighboring standards, finds the status of each format and how the specification relates to those standards.

0.1 Scope

Tools that read, correct, annotate and check papers can work together only if they agree on the files they exchange. The specification defines that data: documents and their packages, anchors, receipts, rights records, layers, exercises, datasets, digitizations, views, projects and bundles. A package is a document stored as a set of files; an anchor is a reference to a place in a document that stays checkable when the document changes.

It defines data, not programs. A conforming file can be read, checked and loaded without any particular program, and every rule is stated so that a second implementation, in any language, computes the same bytes.

The specification does not replace the standards around it. JATS (ANSI/NISO Z39.96-2024) remains the format for publishers and archives to exchange journal articles; here it is only one kind of input to a document. Layers are W3C Web Annotations with a custom selector for the anchor. docs/comparison.md compares the specification with the standards near it and says where each is the better choice.

The guides explain what each format is for, and the rationale why its rules are what they are.

0.2 Conformance language

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY and OPTIONAL are to be interpreted as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.

The suite is the set of test vectors (inputs paired with the outputs every implementation must compute) and fixtures (example files, valid and invalid) in the repository. 22 describes it.

The specification distinguishes three classes of implementation.

Class	Does	Must
Producer	Writes files in a format.	Write only files that validate, with values computed as specified.
Consumer	Reads files in a format.	Accept every valid file, and refuse every file the suite lists as invalid, naming the rule.
Validator	Decides whether a file is valid.	Check every rule in 22 that the file's contents allow it to check.

An implementation states, for each format, which versions it reads and writes.

0.3 Format identifiers and versions

Every format has an identifier of the form scienting-<name>/<major>, such as scienting-doc/1. The identifier tells a reader which rules apply to a file. The prefix scienting- is the namespace of the formats. The namespace is not the name of this repository, paperspec, and formats are named after what they hold.

A file or record that can travel on its own carries its identifier in a format property. The anchor in fixtures/records/anchor-leaf.json begins {"doc":"hale2026harmonic-damb","format":"scienting-anchor/1", .... Records inside a package file whose format the manifest fixes, such as the lines of nodes.jsonl, do not repeat it.

A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version, with a migration. Adding an optional property, a node kind, a run kind, or an enumeration value is a minor change. A consumer of version n MUST refuse a major version it does not know, and SHOULD warn on, but accept, enumeration values added in a later minor version.

The specification as a whole is versioned with semantic versioning; this is version 0.1.0. registry/formats.json lists every format with its media type, schema and status.

0.4 Status

Each format has one of three statuses, listed in registry/formats.json. In version 0.1.0 the registry lists 32 formats: 19 are candidate and 13 are provisional; none is reserved.

Status	Meaning
candidate	Complete, with vectors and fixtures, and ready to implement.
provisional	Complete, but expected to change as implementations gain experience with it.
reserved	Named and shaped, but not yet specified in full.

Some fields matter only to some users: a personal library has no use for spot-checks, and a team that reads only open-access papers rarely needs a policy. Those fields are defined anyway, so that files move between a personal library and a team's store without conversion. A consumer that does not use a field MUST still preserve it when it rewrites a file.

0.5 How the repository is arranged

A normative folder defines what conforming means. The other folders help people use the specification.

Path	Holds	Normative
spec/	These chapters.	Yes
schemas/	JSON Schemas, draft 2020-12.	Yes
sql/	Table definitions for SQLite, Dolt and PostgreSQL, and the SQLite bundle.	Yes
context/	The JSON-LD context for layers.	Yes
registry/	Formats, rule IDs, stamp rules and the default stage requirements.	Yes
vectors/	Test vectors, one case per line.	Yes
fixtures/	Valid packages and records, invalid packages, and the keys that signed them.	Yes
<<<<<<< HEAD
src/	The Python package paperspec: the reference implementation, and the paperspec command that computes and checks the vectors and fixtures and loads packages into SQLite.	No
tools/	How to run those commands.	No
docs/	Guides for people getting to know the formats.	No

The reference implementation in src/paperspec/ computes every expected value, so that none is typed by hand. Other repositories of the project import it, but implementations conform to the chapters and the vectors, not to it. ||||||| parent of 4fa326d (First implementation (#1))

| tools/ | The programs that compute and check the vectors and fixtures, and load packages into SQLite. | No | | docs/ | Guides for people getting to know the formats. | No |

The programs in tools/ compute every expected value, so that none is typed by hand. They are not a library, and implementations conform to the chapters and the vectors.

4fa326d (First implementation (#1))