1.9 KiB
15. Datasets
A dataset, scienting-dataset/1, is a Frictionless Data Package (datapackage.json with CSV or Parquet resources) with three additions.
| Addition | Meaning |
|---|---|
| Units | Every numeric field has a unit, a string Pint can parse, as Lode's dimension check expects. |
| Uncertainty | A field may give uncertainty: {field?, kind}, the field that holds it and its kind, sd, se, resolution or unknown. |
| Origin | Each resource has an origin, or origin_field and origins when it varies by row. |
| Origin | Holds |
|---|---|
reported |
Values anchored to the table cells or sentences that print them, in anchors.jsonl. |
digitized |
Values read off a figure: the digitization's ID, and anchors.jsonl naming each value's series and point. |
computed |
A Lode claim's implementation, by claim and pin. |
measured |
The team's own data, with its provenance. |
The descriptor also carries format and an id, ds- and a key.
Each line of anchors.jsonl names a resource, a row (by primary key) and a field, and either an anchor or a digitization, series and point.
A value or row is addressed as ds-…/<row>/<field>.
Values keep their digits as printed: CSV cells are never reformatted, and in JSON values are strings, so 1.526 and 1.5260 stay different.
A digitized value never passes as reported, and its uncertainty is at least the resolution of its digitization.
A dataset's content hash is SHA-256 of the canonical JSON of {"format": "scienting-dataset/1", "files": [[path, sha256, size], …]} over its files, sorted by path.
A dataset can be a Lode source of kind dataset, so a claim can rest on a value read from a figure, with that fact visible in its evidence.
fixtures/packages/hale2026harmonic/datasets/ds-hale-bond-params reports Table 1, and the suite checks that every value resolves to the cell that prints it.