feat: alignment pairs a misread sentence with its other reading; a suspect names its witness
spec/05 §5.6 gains a substitution step: an unpaired sentence beside one at least 0.8 alike (2L/(|a|+|b|) over the lossy projections' code points) is paired with it, so a misread is one number, equation or word suspect rather than a dropped and an added one. §5.7 and suspect.schema.json gain an optional `witness`, checked by doc.suspect.witness. alignment.jsonl gains a `substitutions` case; the first case is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
9 files changed
+104
-7
No files matched your search
@@ -35,3 +35,14 @@ Packaging, commands and lazy imports only; the formats stay at 0.1.0, and no com
|
||||
- `src/` is Apache-2.0, as `tools/` was, and the package metadata states `Apache-2.0 AND CC0-1.0` for the code and the data it carries.
|
||||
- The pixi workspace installs the package editable, and a `dev` environment builds the sdist and the wheel with `pixi run -e dev dist`.
|
||||
- Pushing a version tag publishes the package to the Gitea package registry through `.gitea/workflows/publish.yml` ([deploy/](deploy/README.md#publish-the-python-package)); a release tag versions the package, and [CONTRIBUTING.md](CONTRIBUTING.md#releasing) says how that relates to the format version.
|
||||
|
||||
### Unreleased
|
||||
|
||||
Two changes asked for by the first producer that compares a converted reading with a second witness (paperstract with a JATS file).
|
||||
Both are minor changes in the sense of [spec/00](spec/00-introduction.md): no valid file becomes invalid, and no hash, ID or key changes; the next release numbers them.
|
||||
|
||||
- Alignment has a substitution step ([spec/05 §5.6](spec/05-hashes.md#56-alignment), step 3): in each stretch between pairs, an unpaired sentence of the first reading is paired with the first unpaired sentence of the second whose similarity (2*L* / (|*a*| + |*b*|) over code points of the lossy projections, *L* their longest common subsequence) is at least 0.8, both of prose length 20 or more.
|
||||
A sentence one reading misread is now one `number`, `equation` or `word` suspect rather than a `dropped` and an `added` one, so a producer re-aligning the same two readings can get different suspects than before; the pairs and suspects of readings with no such sentence are unchanged, as the first alignment vector is.
|
||||
`paperspec.align` gains `lcs_length` (bit-parallel, Hyyrö 2004) and `similarity`, and `alignment.jsonl` a second case, `substitutions`: a misread number, a misread word, a sentence too short to be paired this way and two sentences too unlike to be one.
|
||||
- A suspect may name the witness its second reading came from (`witness`, a witness ID; [spec/05 §5.7](spec/05-hashes.md#57-suspects), `suspect.schema.json`), and a package whose suspect names a witness it does not have breaks `doc.suspect.witness`.
|
||||
No broken package exercises the rule yet: adding one changes the count of broken packages that the 0.1.0 entry above states, so it waits for the release.
|
||||
Reference in new issue
Block a user