71 lines
4.4 KiB
Markdown
71 lines
4.4 KiB
Markdown
# 12. The package
|
|
|
|
A document travels as a package: a folder named by its key, or the same folder zipped.
|
|
|
|
## 12.1 Files
|
|
|
|
| Path | Required | Holds |
|
|
| --- | --- | --- |
|
|
| `MANIFEST.json` | Yes | The manifest ([12.3](#123-the-manifest)). |
|
|
| `document.json` | Yes | The document ([2.2](02-model.md#22-documentjson)). |
|
|
| `nodes.jsonl` | Yes | Every node, in document order ([3](03-nodes.md)). |
|
|
| `witnesses.json` | Yes | Every witness, with its rights ([2.3](02-model.md#23-witnessesjson)). |
|
|
| `provenance.json` | Yes | Tools and versions, backends, cached readings, stages and where each ran. |
|
|
| `pages.jsonl` | No | Every page of every witness ([7.3](07-geometry.md#73-pagesjsonl)). |
|
|
| `locations.jsonl` | No | Geometry per node and page ([7.4](07-geometry.md#74-locationsjsonl)). |
|
|
| `references.jsonl` | No | Parsed references and their resolution ([10](10-references.md)). |
|
|
| `corrections.jsonl` | No | Every correction, in order ([11](11-corrections.md)). |
|
|
| `receipts.jsonl` | No | Receipts and revocations, sorted by `at` and then `id` ([8](08-fidelity.md)). |
|
|
| `ids.jsonl` | No | Retired IDs and legacy aliases ([4.4](04-identity.md#44-idsjsonl)). |
|
|
| `suspects.jsonl` | No | Unresolved suspects, sorted by `id` ([5.7](05-hashes.md#57-suspects)). |
|
|
| `roster.json` | No | The roster its receipts need ([8.7](08-fidelity.md#87-revocations-and-the-roster)). |
|
|
| `assets/` | No | Images and crops, as `<sha256>.<extension>`, present or listed only. |
|
|
| `datasets/<id>/` | No | Datasets ([15](15-datasets.md)). |
|
|
| `derived/` | No | Rebuildable renderings, never authoritative ([12.7](#127-derived-files)). |
|
|
|
|
An optional JSON Lines file with no records is omitted.
|
|
Every file is in the canonical form of [1.9](01-conventions.md#19-files).
|
|
|
|
## 12.2 Paths
|
|
|
|
Paths use `/`, are relative, have no empty, `.` or `..` segment, no backslash and no NUL, and are already normalized.
|
|
In a zip, entries are UTF-8 named, sit under one top-level folder named by the key, are stored or deflated, carry no symbolic links, and are unique; a consumer refuses an archive that breaks any of these, or whose expanded size exceeds its own limit.
|
|
A producer that wants byte-identical archives writes entries sorted by path, dated 1980-01-01 00:00, with no extra fields.
|
|
|
|
## 12.3 The manifest
|
|
|
|
`MANIFEST.json`, `scienting-package/1`, is `{format, doc, key, files}`: the package format, `scienting-doc/1`, the document key, and every other file as `{path, sha256, size}`, sorted by the UTF-8 bytes of the path.
|
|
An asset whose bytes stay in the asset store is listed with `present: false`; nothing else may be.
|
|
The SHA-256 of `MANIFEST.json` pins the whole package.
|
|
The manifest is meant to have the same shape as Lode's `lode-cache/1`, so Lode can hold a package as a source cache.
|
|
|
|
## 12.4 Provenance
|
|
|
|
`provenance.json` holds `tools` (`{name, version}`), and optionally `backends`, `readings` (each cached reading's ID, witness, backend and SHA-256), `stages` (each stage's name, tool, input hashes, output hash, the `processor` where it ran, and the uses it ran under) and a `session`.
|
|
Every stage that uses a model or an outside service records where it ran ([14](14-rights.md)).
|
|
|
|
## 12.5 Text policies
|
|
|
|
| Policy | Means |
|
|
| --- | --- |
|
|
| `full` | Every leaf keeps its content. |
|
|
| `checks-only` | Every leaf has `withheld: true` and no `content`; panels drop their figure text; references keep their resolution but no CSL; there are no corrections, suspects or datasets; and the hashing scheme is keyed. |
|
|
|
|
A checks-only package is how a team shares its annotations on a paywalled paper without sharing the paper.
|
|
Its text lives in the team's private asset store, and only key holders can check its values.
|
|
`fixtures/packages/hale2026harmonic.checks-only` is such an export of `hale2026harmonic` under the team key in `fixtures/keys/`.
|
|
|
|
## 12.6 Assets
|
|
|
|
Assets are named by SHA-256, so every read is rehashed and a regenerated crop is a different asset.
|
|
Bytes that live only in the asset store are listed in the manifest with `present: false`, with their size.
|
|
|
|
## 12.7 Derived files
|
|
|
|
`derived/` holds what a pinned tool makes from the package, such as `derived/mathml.jsonl`, whose lines are `{node, run, tool, mathml}` with `run` null for an equation node.
|
|
Nothing derived is content, and a consumer may delete and rebuild it.
|
|
|
|
## 12.8 Media type and extension
|
|
|
|
A zipped package has the extension `.sct.zip` and the media type `application/vnd.scienting.doc+zip`.
|