dev
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  alexmandClaude
|
516148146f
|
feat: publish the package from CI, and clean up its references
Publishing - .gitea/workflows/publish.yml publishes paperspec to the instance's PyPI registry (/api/packages/infra/pypi) when a v* tag is pushed: it checks the repository, builds the sdist and wheel, refuses to upload unless the built version equals the tag, runs twine check --strict, and uploads with the secrets PACKAGE_USER and PACKAGE_TOKEN (--skip-existing, so a rerun on the same tag is safe). Run by hand, it refuses anything but a tag. - pixi dev environment: twine >= 6.1 (the first that reads Metadata 2.4), and `dist` (sdist and wheel) replaces the `wheel` task. - deploy/README.md: how to set up the secrets, release a version, and install from the registry with pixi, uv or pip, tying paperspec to the registry where the installer allows it. - CONTRIBUTING.md "Releasing": a tag versions the package; a format change takes the new format version as the top CHANGELOG heading, and a code-only release takes the next patch as a subsection, as 0.1.1 does. - pyproject: fallback_version 0+unknown for source trees without git; the license expression is Apache-2.0 AND CC0-1.0, for the code and the data the wheel carries. Cleanup (from an audit of every reference to the package) - spec/00 lists src/ (the package) and tools/ (how to run it), and no longer says the reference implementation is not a library. - Stale explanations fixed in pixi.toml, .gitignore, the Caddyfile, the vectors, context, sql and tools READMEs, docs/examples.md, docs/faq.md, spec/01, docs/rationale.md and docs/comparison.md: the commands, what they need (the tools extra), the fixture folder hale2026harmonic-damb, and three lines that held two sentences. - tools/README.md counts all six commands, says what needs which extra (package.validate and corrections.gate need [schema]), and that the vectors ship but the fixtures they name do not. - The CLI's install and clone hints point at the registry instructions and an https clone URL; schema.py raises an ImportError naming paperspec[schema], as crypto.py does for [crypto]. - check's Unicode check now guards the package's pinned constant. - site skips build/, dist/ and *.egg-info. - CHANGELOG 0.1.1 describes the final state. pixi run check passes (as `python -m paperspec check`); the rebuild is byte-identical. pixi.lock needs `pixi lock` for twine before pushing: the publish workflow installs with --locked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
alexmandClaude
|
88379d152c
|
feat!: one paperspec command with subcommands, help and examples
The six console scripts (paperspec-check, -build, -validate, -load, -stress, -site) become subcommands of a single `paperspec` command: paperspec validate, load, check, build, stress and site. `python -m paperspec COMMAND` runs the same thing. - paperspec/cli/main.py builds the whole parser, apart from the commands, so `paperspec --help` and `paperspec COMMAND --help` work without a checkout or the tools extra; a command's module is imported only when it runs. - `paperspec` with no command prints the overview: what each command does, examples, and which work anywhere (validate, load) and which need a checkout (check, build, stress, site). Each command's help has a description with advice, and worked examples; a misspelled command gets a suggestion (argparse suggest_on_error); `--version` prints the version and the pinned Unicode version. - Each command module exposes run(args) and returns the exit status. Arguments that parse but do not fit together raise UsageError, which prints the command's usage and exits 2: load now also refuses --built-from or --store without --bundle. - The failure messages say what to do: install paperspec[tools], clone the repository, rerun stress with the printed seed, or run paperspec build when the rebuild check differs. check and stress say how many failures they did not list. - check runs the rebuild as `python -m paperspec build`. - pyproject installs one script, `paperspec`; the pixi tasks and the docs use the subcommands. pixi run check passes (run here as `python -m paperspec check` with src/ on the path); the rebuild is byte-identical. pixi.lock still needs `pixi lock`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
alexmandClaude
|
607062056f
|
feat!: pin Python 3.14 and Unicode 16.0, and make the tools commands
Python and Unicode
- requires-python is >=3.14,<3.15: 3.14 is the only Python with the
Unicode 16.0 data the specification pins (3.13 has 15.1, 3.15 has 17.0).
- `import paperspec` raises ImportError unless unicodedata reports
paperspec.UNICODE_VERSION (16.0.0), so a source checkout or an install
that bypasses requires-python cannot compute values on other data.
Commands
- tools/{check,build,validate,load,stress,site}.py move with history to
src/paperspec/cli/ and install as paperspec-check, -build, -validate,
-load, -stress and -site; each also runs as `python -m paperspec.cli.X`.
- paperspec.cli.require() exits with "install paperspec[tools]" when
cryptography or jsonschema is missing; paperspec.cli.checkout() finds
the repository the check, build, stress and site commands work on (the
editable install's own checkout, else the one around the current
folder) and exits with a message outside one. tools/__init__.py is gone.
- validate and load read the schemas and SQL through the new
paperspec.resources.path(), which finds them in a wheel and in an
editable install alike, so they work anywhere the package is installed.
- check runs the rebuild as `python -m paperspec.cli.build`.
- build still writes the provenance tool label "paperspec tools/build.py":
it is part of every hashed and signed fixture.
- pixi tasks and the check workflow run the commands; tools/README.md
documents them, and the other docs name the commands instead of paths.
License
- LICENSE.md gives `src/` Apache-2.0 alongside `tools/`. Since the
previous commit moved the code to src/, no row had covered it.
pixi run check passes (run as `python -m paperspec.cli.check` with src/ on
the path, since this environment cannot install the entry points); the
byte-for-byte rebuild matches, so no fixture or vector changed.
pixi.lock still needs `pixi lock`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
alexmandClaude
|
3831b52a2b
|
feat: ML-DSA-65, did:key, named hash algorithm, model provenance, log tiles and witnesses
- hashing.algorithm names the hash function in every document; a consumer refuses one it does not implement (doc.hash.algorithm) - ML-DSA-65 (FIPS 204) is an implemented signature scheme: pure-Python reference in tools/oracle/mldsa.py checked against NIST ACVP vectors, a receipt signed by the new roster key dana, and signature vectors for key generation and a deterministic signature - roster keys may carry a did:key, which the validator checks against the public key (doc.roster.did), and a witness purpose for parties that cosign log checkpoints - model provenance: a correction's author and an annotation's generator name the model with provider, version and the hashes of its weights, prompt and parameters, and where it ran - the transparency log gains the tlog-tiles static layout (fixtures/records/log/), a project settings.log with origin, witnesses and threshold, and an acceptance rule; the fixture checkpoint is cosigned by a witness - store.sql: roster_keys accepts cose and ml-dsa-65 and gains a did column - spec chapters 01, 08, 11, 18, 20 and 23, the rationale, glossary, fixture, vector and tool guides updated; stress harness covers ML-DSA Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
alexmandClaude
|
a09d158e16
|
feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
alexmandClaude
|
76d76353d9
|
feat: rename to paperspec and add the website
- Rename manuspec to paperspec everywhere outside pixi.toml: prose, the tools, provenance tool names and the fixture key derivations, so every fixture and vector is rebuilt. - Move the namespace addresses under the site, mirroring the repository: schema $id https://paperspec.scient.ing/schemas/<file>, the JSON-LD context https://paperspec.scient.ing/context/scienting-1.jsonld, and term IRIs https://paperspec.scient.ing/vocab#<term>. Format identifiers (scienting-*) and URNs (urn:scienting:) are unchanged. - tools/site.py (standard library only) builds _site/: every file of the repository at its own path, each Markdown file rendered as a page with sidebar navigation, heading anchors matching the forge's, folder listings, a vocabulary page for the context's IRIs, a heading search and a 404 page; light and dark themes. - check.py builds the site and checks that every Markdown file has a page, every internal link and fragment resolves, no Markdown is left unrendered, and every $id, the context address and the vocabulary IRIs are files of the site. - deploy/: a Caddyfile with the media types and CORS the data needs, and a guide; .gitea/workflows/site.yml builds on main and publishes with rsync when the SITE_* secrets are set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
alexmandClaude
|
ea0a7efa78
|
feat: zipped packages, target set, checklists, audit fixes and a stress test
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
alexmandClaude
|
8c8b51cc3f
|
feat: validator, bundle builder and release checks for 0.1.0
- Add tools/validate.py: validates packages, and with --correction runs proposed corrections through the gate against a package without writing anything (spec/11 §11.4). The gate now refuses a proposal that fails the correction schema, rule correction.schema. - tools/load.py validates every package before loading, and with --bundle --built-from builds a scienting-bundle/1 with the FTS5 search index and bundle_meta (spec/13 §13.5). Search rows are defined per unit (sentence, cell, equation outside a sentence, figure-text) with the parent kind as context. - Roster keys carry a role, and a key with the role owner may revoke any receipt (spec/08 §8.7, spec/20); roster schema, store table and the fixture roster updated. The checker signs a trial revocation to prove the rule both ways. - A qa: receipt raises the nodes it sampled except those in its errors (spec/08 §8.2); the example paper's fidelity counts become 27 F3, 13 F2, 4 F1 and 1 F0. - The destination chooses the keyed scheme, not the text policy: hmac-team between the team's machines, hkdf-pages outside the team (spec/05 §5.8, spec/12 §12.5). - A TextQuoteSelector must repeat the anchor's plain quote and is absent for hashed or keyed anchors (spec/18 §18.3). - Add fixtures/records/suspect.json, a scienting-suspect/1 record. - Oracle: hashes are kept incrementally and checked after every correction against a full recomputation; load and write refuse unsafe manifest paths; Ed25519, HKDF and schema validation come from the cryptography and jsonschema packages; canonical JSON uses the built-in encoder with an exact RFC 8785 fallback. - The tools need Python 3.14 with cryptography and jsonschema; the py310 CI environment is dropped and the workflow runs the default only. - Resolve every [CHECK] note in the docs, READMEs and context; add CONTRIBUTING.md; every tool answers --help and refuses unknown options. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
alexm
|
002e2c3552
|
feat: rewrite docs | ||
|
alexm
|
8b2df58266
|
feat: documents and session tokens | ||
|
alexm
|
62f54ed869
|
Initial commit |