feat: documents and session tokens

README.md

@@ -1,52 +1,76 @@
 # varve-spec
 
-The specifications of the Scientific Computing Studio's data formats: documents and their packages, anchors, receipts, rights, layers, exercises, datasets, digitizations, views, projects and bundles.
-This is the repository the design calls `spec`, the baseline every other part implements.
+**The data formats behind Lode, paperpack and Varve: how a team's scientific papers are stored once, corrected once, addressed down to the sentence, and kept.**
 
-`spec` defines data, and data outlives code.
-Every file any part writes is defined here, and a conforming file can be read without the part that wrote it.
-Formats are named `scienting-<name>/<major>`, never after a product, so a product can be renamed without touching data.
+A research team usually cares about a few dozen papers it understands completely, not the world's literature.
+Those papers deserve better than a folder of PDFs: every sentence, table cell and equation should have a permanent address; every correction should be recorded with its evidence; every note, claim and extracted number should point at exactly what it came from; and everyone should be able to see how far each piece was checked, and by whom.
 
-## What is here
+This repository defines the files that make that possible.
+It holds no application.
+paperpack writes these files, Lode reads them to check claims, and Varve stores, renders and annotates them; any other program can do the same, because every format is documented here and readable without the studio's code.
 
-| Path | Holds |
-| --- | --- |
-| [spec/](spec/README.md) | The normative chapters. Start with [spec/00-introduction.md](spec/00-introduction.md). |
-| [schemas/](schemas/) | JSON Schemas (draft 2020-12) for every file and record. |
-| [sql/](sql/) | Tables for SQLite, Dolt and PostgreSQL, and the SQLite bundle with full-text search. |
-| [context/](context/) | The JSON-LD context that makes layers W3C Web Annotations. |
-| [registry/](registry/) | Formats and media types, validation rule IDs, download-stamp rules and the default stage requirements. |
-| [vectors/](vectors/README.md) | Test vectors for every computed value. |
-| [fixtures/](fixtures/README.md) | Valid packages and records, invalid packages with the rule each breaks, and the test keys that signed them. |
-| [tools/](tools/README.md) | The non-normative oracle that computed the vectors, the builder and the checker. |
-| [DECISIONS.md](DECISIONS.md) | Where this text refines or departs from the design, and the decisions still open. |
-
-## Check it
-
-Python 3.10 or later, with nothing installed:
-
-```sh
-python3 tools/check.py
+```text
+cornell1995second#k3m9q2a@4f1a0c[12:57]
+└──── document ──┘└ node ┘└ pin ┘└span┘
 ```
 
-It validates every schema, record and package, refuses every invalid package for the stated rule, recomputes every vector, rebuilds the fixtures and vectors and compares them byte for byte, loads every package into SQLite and unloads it byte for byte, and lints the prose.
-With `jsonschema` installed it also validates every record a second time with that library.
-`make check` does the same and runs `markdownlint` when it is available.
+That string names code points 12 to 57 of one sentence, as it read when its hash began `4f1a0c`.
+It survives reconversion with a better tool, a typo fixed elsewhere in the paper, and a teammate's differently stamped download of the same PDF.
 
-## Implement it
+## Start here
 
-An implementation conforms to the chapters and the vectors.
-`scienting-core` (Python, standard library only) is the reference implementation of the shared rules, and `core-ts` the browser's; each runs this suite in its CI.
-The oracle in `tools/` is not a library and is not meant to be imported: it exists so that no vector is typed by hand.
+| If you want to | Read |
+| --- | --- |
+| Understand what the project is and how the parts fit | [docs/overview.md](docs/overview.md) |
+| Learn the ideas one at a time, each with an example | [docs/concepts.md](docs/concepts.md) |
+| Follow one paper from its first reading to a signed record | [docs/walkthrough.md](docs/walkthrough.md) |
+| Run something in the next five minutes | [docs/examples.md](docs/examples.md) |
+| Implement a reader or writer | [spec/](spec/README.md), then [vectors/](vectors/README.md) |
+| Know why a rule is the way it is | [docs/rationale.md](docs/rationale.md) |
+| Look up a term | [docs/glossary.md](docs/glossary.md) |
 
-## Status
+## What is in the repository
 
-Version 0.1.0.
-The formats of milestone M1 are `candidate`; datasets, digitizations, views, projects, bundles and the draft formats are `provisional` until their milestones; see [registry/formats.json](registry/formats.json).
+| Folder | Holds |
+| --- | --- |
+| [spec/](spec/README.md) | The specification, in 23 chapters. Normative. |
+| [schemas/](schemas/README.md) | JSON Schemas for every file and record. |
+| [sql/](sql/README.md) | Tables for SQLite, Dolt and PostgreSQL, and the SQLite bundle with full-text search. |
+| [context/](context/README.md) | The JSON-LD context that makes annotation layers W3C Web Annotations. |
+| [registry/](registry/README.md) | The list of formats, the rule IDs validators report, download-stamp rules, and default rights requirements. |
+| [vectors/](vectors/README.md) | Test vectors: inputs and the exact outputs every implementation must compute. |
+| [fixtures/](fixtures/README.md) | Example packages and records, deliberately broken packages, and the test keys that signed them. |
+| [tools/](tools/README.md) | The programs that build and check everything above. |
+| [docs/](docs/README.md) | Guides for people getting to know the project. |
+
+## Try it
+
+With [pixi](https://pixi.sh), one command sets up a pinned environment and checks everything:
+
+```sh
+pixi run check
+```
+
+The checker validates every schema, record and package, confirms that every broken package is refused for the stated reason, recomputes every test vector, rebuilds everything and compares it byte for byte, loads every package into SQLite and back, and lints the prose and the links.
+It ends with `All checks pass.`
+`pixi task list` shows the other tasks: `quick`, `build`, `lint` and `load`.
+
+The tools need nothing but Python 3.10 or later, so without pixi the same check is `python3 tools/check.py`; pixi adds the `jsonschema` package as a second schema validator and markdownlint for the Markdown.
+
+To poke at a real document, load the example paper into SQLite and look around; [docs/examples.md](docs/examples.md) has queries to start from.
+
+```sh
+pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic
+```
+
+## Formats
+
+Every format is named `scienting-<name>/<major>`, such as `scienting-doc/1` or `scienting-anchor/1`, and never after a product, so products can be renamed without touching data.
+[registry/formats.json](registry/formats.json) lists them all with their media types, schemas and chapters.
 
 ## Licenses
 
-The chapters and other prose are licensed CC BY 4.0.
-Schemas, SQL, the JSON-LD context, the registries, the vectors and the fixtures are dedicated to the public domain under CC0 1.0, so anyone may implement the formats.
-The tools are licensed Apache-2.0.
+The specification and other prose are CC BY 4.0, so anyone may implement the formats.
+Schemas, SQL, the JSON-LD context, the registries, the vectors and the fixtures are CC0 1.0.
+The tools are Apache-2.0.
 See [LICENSE.md](LICENSE.md).