First implementation (#1)

Reviewed-on: #1

spec/00-introduction.md

@@ -0,0 +1,89 @@
+# 0. Introduction
+
+This chapter is for three readers.
+A newcomer, who knows what a scientific paper and a citation are, finds what the specification covers and how to read it.
+An implementer, who will write a reader, writer, converter or checker, finds the rules for conformance, identifiers, versions and status.
+An expert, who knows the neighboring standards, finds the status of each format and how the specification relates to those standards.
+
+## 0.1 Scope
+
+Tools that read, correct, annotate and check papers can work together only if they agree on the files they exchange.
+The specification defines that data: documents and their packages, anchors, receipts, rights records, layers, exercises, datasets, digitizations, views, projects and bundles.
+A package is a document stored as a set of files; an anchor is a reference to a place in a document that stays checkable when the document changes.
+
+It defines data, not programs.
+A conforming file can be read, checked and loaded without any particular program, and every rule is stated so that a second implementation, in any language, computes the same bytes.
+
+The specification does not replace the standards around it.
+JATS ([ANSI/NISO Z39.96-2024](https://www.niso.org/publications/z3996-2024-jats)) remains the format for publishers and archives to exchange journal articles; here it is only one kind of input to a document.
+Layers are [W3C Web Annotations](https://www.w3.org/TR/annotation-model/) with a custom selector for the anchor. [docs/comparison.md](../docs/comparison.md) compares the specification with the standards near it and says where each is the better choice.
+
+The [guides](../docs/README.md) explain what each format is for, and the [rationale](../docs/rationale.md) why its rules are what they are.
+
+## 0.2 Conformance language
+
+The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY and OPTIONAL are to be interpreted as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.
+
+The suite is the set of test vectors (inputs paired with the outputs every implementation must compute) and fixtures (example files, valid and invalid) in the repository. [22](22-conformance.md) describes it.
+
+The specification distinguishes three classes of implementation.
+
+| Class | Does | Must |
+| --- | --- | --- |
+| Producer | Writes files in a format. | Write only files that validate, with values computed as specified. |
+| Consumer | Reads files in a format. | Accept every valid file, and refuse every file the suite lists as invalid, naming the rule. |
+| Validator | Decides whether a file is valid. | Check every rule in [22](22-conformance.md) that the file's contents allow it to check. |
+
+An implementation states, for each format, which versions it reads and writes.
+
+## 0.3 Format identifiers and versions
+
+Every format has an identifier of the form `scienting-<name>/<major>`, such as `scienting-doc/1`.
+The identifier tells a reader which rules apply to a file.
+The prefix `scienting-` is the namespace of the formats.
+The namespace is not the name of this repository, paperspec, and formats are named after what they hold.
+
+A file or record that can travel on its own carries its identifier in a `format` property.
+The anchor in `fixtures/records/anchor-leaf.json` begins `{"doc":"hale2026harmonic-damb","format":"scienting-anchor/1", ...`.
+Records inside a package file whose format the manifest fixes, such as the lines of `nodes.jsonl`, do not repeat it.
+
+A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version, with a migration.
+Adding an optional property, a node kind, a run kind, or an enumeration value is a minor change.
+A consumer of version *n* MUST refuse a major version it does not know, and SHOULD warn on, but accept, enumeration values added in a later minor version.
+
+The specification as a whole is versioned with semantic versioning; this is version 0.1.0. [registry/formats.json](../registry/formats.json) lists every format with its media type, schema and status.
+
+## 0.4 Status
+
+Each format has one of three statuses, listed in [registry/formats.json](../registry/formats.json).
+In version 0.1.0 the registry lists 32 formats: 19 are `candidate` and 13 are `provisional`; none is `reserved`.
+
+| Status | Meaning |
+| --- | --- |
+| `candidate` | Complete, with vectors and fixtures, and ready to implement. |
+| `provisional` | Complete, but expected to change as implementations gain experience with it. |
+| `reserved` | Named and shaped, but not yet specified in full. |
+
+Some fields matter only to some users: a personal library has no use for spot-checks, and a team that reads only open-access papers rarely needs a policy.
+Those fields are defined anyway, so that files move between a personal library and a team's store without conversion.
+A consumer that does not use a field MUST still preserve it when it rewrites a file.
+
+## 0.5 How the repository is arranged
+
+A normative folder defines what conforming means.
+The other folders help people use the specification.
+
+| Path | Holds | Normative |
+| --- | --- | --- |
+| `spec/` | These chapters. | Yes |
+| `schemas/` | JSON Schemas, draft 2020-12. | Yes |
+| `sql/` | Table definitions for SQLite, Dolt and PostgreSQL, and the SQLite bundle. | Yes |
+| `context/` | The JSON-LD context for layers. | Yes |
+| `registry/` | Formats, rule IDs, stamp rules and the default stage requirements. | Yes |
+| `vectors/` | Test vectors, one case per line. | Yes |
+| `fixtures/` | Valid packages and records, invalid packages, and the keys that signed them. | Yes |
+| `tools/` | The programs that compute and check the vectors and fixtures, and load packages into SQLite. | No |
+| `docs/` | Guides for people getting to know the formats. | No |
+
+The programs in `tools/` compute every expected value, so that none is typed by hand.
+They are not a library, and implementations conform to the chapters and the vectors.