Files
paperspec/docs/package/guides/load-into-sqlite.md
T
alexmandClaude 30d310f64d Publish the package and its documentation, and move to the papers organization
Add the Zensical documentation of the Python package (docs/package, built by the docs
workflow), merge the standard publish workflow with the setuptools-scm tag check, add the
SPDX header rule to ruff, restructure the README, and point every repository and registry
address at the Gitea organization papers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
2026-10-19 04:23:49 -04:00

47 lines
1.9 KiB
Markdown

# Load packages into SQLite
`paperspec load` validates packages and loads them into a SQLite database with the tables of `sql/doc.sql` and `sql/store.sql`.
A node, a reference, a correction and a receipt each get rows you can join and query.
## Load a package
```sh
paperspec load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
```
```text
hale2026harmonic-damb: 75 nodes
```
- Every package is validated first, and nothing is loaded if any fails.
Each failure is printed as `package: where: rule: message`, as `paperspec validate` prints it.
- A new file gets the tables, and an existing one takes more packages, so you can add papers to the same database later.
- Files with no table of their own (assets, datasets, derived files) are copied by SHA-256 into a folder beside the database, `lab.db.assets/`, which must travel with it.
- The full and checks-only forms of one document share its key, so load them into separate databases.
## Query it
```sh
python -m sqlite3 lab.db "SELECT kind, COUNT(*) FROM nodes GROUP BY kind ORDER BY 2 DESC, kind LIMIT 3"
```
```text
('cell', 23)
('sentence', 18)
('row', 6)
```
The guide at <https://paperspec.scient.ing/docs/examples.html> has queries to start from: the sentences still below fidelity F2, the citation graph at sentence level, and who signed what.
## Build a bundle with full-text search
```sh
paperspec load --sqlite search.db --bundle --built-from REV paper/ other/
```
A bundle is a `scienting-bundle/1` file: it also gets the tables of `sql/bundle-sqlite.sql`, the full-text search index and `bundle_meta`.
It is built once, into a file that does not exist yet.
`--bundle` needs `--built-from`, which names the store revision the packages come from, and the command prints the number of search rows it added.
`--store` names the store, and defaults to the packages' common store.
`--built-from` and `--store` are refused without `--bundle`.