feat: rewrite docs

sql/README.md

@@ -1,12 +1,24 @@
 # SQL
 
-Table definitions for the database form of packages, a team's store, and the bundle.
+This folder holds the table definitions for the database form of packages, a team's store, and the bundle.
+
+## What the database form is for
+
+A package is a folder of canonical JSON files, one document per folder ([chapter 12](../spec/12-package.md)).
+Canonical JSON is JSON with exactly one valid byte sequence, so a file's hash changes only when its content does.
+
+Handing over a folder is easy.
+Querying across hundreds of documents in folders is hard.
+The database form holds the same data as rows so that a team can ask SQL questions of it, and a package loaded into the tables unloads back to the same bytes.
+About 500 documents come to about a million and a half rows, and SQLite and Dolt hold that comfortably ([chapter 13](../spec/13-database.md#132-tables)).
+
+## The three files
 
 | File | Holds | Runs on |
 | --- | --- | --- |
 | `doc.sql` | Documents: works, documents, witnesses, nodes, pages, locations, references, citations, corrections, IDs, receipts, revocations, suspects and package files. | SQLite, Dolt, PostgreSQL |
 | `store.sql` | What a team adds: roster keys, projects, members, reading paths, layers, annotations, datasets, digitizations, views, policies, submissions and grades. Load `doc.sql` first. | SQLite, Dolt, PostgreSQL |
-| `bundle-sqlite.sql` | The bundle's extra tables: metadata, a full-text index (FTS5) and room for embeddings. Load both files above first. | SQLite |
+| `bundle-sqlite.sql` | The bundle's extra tables: metadata, a full-text index (FTS5, SQLite's search extension) and room for embeddings. Load both files above first. | SQLite |
 
 ## Creating a database
 
@@ -16,19 +28,16 @@ psql -d lab -f sql/doc.sql && psql -d lab -f sql/store.sql
 dolt sql < sql/doc.sql && dolt sql < sql/store.sql
 ```
 
-To load packages into SQLite with nothing installed, use `python3 tools/load.py --sqlite lab.db PACKAGE…`; [docs/examples.md](../docs/examples.md#load-packages-into-sqlite) has queries to try.
+[CHECK: the sqlite3, psql and dolt commands were not run here; the PostgreSQL database `lab` and the Dolt repository must already exist.]
 
-## Conventions
-
-- No foreign keys: validators check references, so the same file runs everywhere.
-- Enumerations are `VARCHAR` with a `CHECK`; JSON values are canonical JSON in `TEXT`; booleans are `INTEGER` 0 or 1.
-- No table, column or index name is reserved in any of the three engines, so the formats' `key`, `index`, `at`, `by`, `before` and `after` become `doc_key`, `idx`, `made_at`, `signer`, `before_hash` and `after_hash`.
-- Each row keeps every property of its record, so a package loaded into the tables unloads back to the same bytes.
-- `citations`, `receipt_nodes` and `annotation_targets` are derived on load, for queries.
+To load packages into SQLite with nothing installed but Python 3, run `python3 tools/load.py --sqlite lab.db PACKAGE…`.
+The file [docs/examples.md](../docs/examples.md#load-packages-into-sqlite) has queries to try.
+The loader copies files that have no table (assets, dataset files, derived files) by SHA-256 into `lab.db.assets/`.
 
 ## A first query
 
-Every sentence that cites a given reference, which is the citation's context:
+The query below finds every sentence that cites a given reference, which is the citation's context.
+It returns sentence IDs, and `nodes.content` holds each sentence's text.
 
 ```sql
 SELECT c.sentence
@@ -37,4 +46,20 @@ JOIN refs r ON r.doc_key = c.doc_key AND r.node = c.ref_node
 WHERE c.doc_key = 'hale2026harmonic' AND r.label = '2';
 ```
 
-[Chapter 13](../spec/13-database.md) defines the mapping from files to tables.
+On `fixtures/packages/hale2026harmonic` it returns one row, `f929eq9`: the sentence ending "served as the first test case, with a bending band near 1595 cm⁻¹."
+
+## Conventions
+
+- Foreign keys: none.
+  Validators check references, so the same file runs everywhere.
+- Types: enumerations are `VARCHAR` with a `CHECK`, JSON values are canonical JSON in `TEXT`, and booleans are `INTEGER` 0 or 1.
+- Names: no table, column or index name is reserved in SQLite, PostgreSQL or MySQL (Dolt is MySQL-compatible).
+  The formats' `key`, `index`, `at`, `by`, `before` and `after` become `doc_key`, `idx`, `made_at`, `signer`, `before_hash` and `after_hash`.
+- Round trip: each row keeps every property of its record, so unloading restores the same bytes and the same manifest hash.
+  Unloading copies the files with no table from the asset store, which must travel with the database. `tools/check.py` tests this for every fixture package.
+- Derived tables: `citations`, `receipt_nodes` and `annotation_targets` are built on load for queries and are not part of the package.
+
+## Status
+
+The version is 0.1.0.
+In `registry/formats.json`, `scienting-doc/1` is `candidate`, and the bundle and the project, dataset, digitization and view formats are `provisional`. [Chapter 13](../spec/13-database.md) defines the mapping from files to tables.

sql/bundle-sqlite.sql

@@ -1,4 +1,4 @@
--- scienting-bundle/1: the SQLite file a reader, a server or an MCP server reads.
+-- scienting-bundle/1: the SQLite file a reader or a server reads.
 -- Normative: spec/13-database.md. SQLite only: it uses FTS5.
 -- A bundle holds sql/doc.sql and sql/store.sql, then these tables.
 -- License: CC0-1.0.