feat: alignment pairs a misread sentence with its other reading; a suspect names its witness

spec/05 §5.6 gains a substitution step: an unpaired sentence beside one at
least 0.8 alike (2L/(|a|+|b|) over the lossy projections' code points) is
paired with it, so a misread is one number, equation or word suspect rather
than a dropped and an added one. §5.7 and suspect.schema.json gain an
optional `witness`, checked by doc.suspect.witness. alignment.jsonl gains a
`substitutions` case; the first case is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-16 22:41:15 -04:00
1 parent 9e24ca2306
commit 3836a6c885
9 files changed
+104 -7

No files matched your search

+1 -1
View File
@@ -138,7 +138,7 @@ A candidate format changes only with a recorded reason and a new or changed vect
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `src/paperspec/`.
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
The vectors hold 311 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` two cases.
The vectors hold 312 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` two pairs of readings, `offsets.jsonl` two cases.
The vectors do not show the following.
+1
View File
@@ -1 +1,2 @@
{"a":[{"complex":[],"container":"h","context":"heading","id":"a1","lossy":"methods","runs":[{"kind":"text","text":"Methods"}],"strict":"Methods"},{"complex":[],"container":"p1","context":"paragraph","id":"a2","lossy":"everybondwasscannedinstepsof0.01åarounditsminimum.","runs":[{"kind":"text","text":"Every bond was scanned in steps of 0.01 Å around its minimum."}],"strict":"Every bond was scanned in steps of 0.01 Å around its minimum."},{"complex":["tex:\\frac{a}{b}"],"container":"p1","context":"paragraph","id":"a3","lossy":"whereisthebondlength.","runs":[{"kind":"text","text":"where "},{"kind":"math","notation":"tex","source":"\\frac{a}{b}"},{"kind":"text","text":" is the bond length."}],"strict":"where  is the bond length."},{"complex":[],"container":"p1","context":"paragraph","id":"a4","lossy":"thefittedrateis103persecondatroomtemperature.","runs":[{"kind":"text","text":"The fitted rate is 10³ per second at room temperature."}],"strict":"The fitted rate is 10^{3} per second at room temperature."},{"complex":["tex:\\frac{c}{d}"],"container":"p2","context":"paragraph","id":"a5","lossy":"whereisthebondlength.","runs":[{"kind":"text","text":"where "},{"kind":"math","notation":"tex","source":"\\frac{c}{d}"},{"kind":"text","text":" is the bond length."}],"strict":"where  is the bond length."},{"complex":[],"container":"p2","context":"paragraph","id":"a6","lossy":"thissentencewasdroppedbythesecondreadingentirely.","runs":[{"kind":"text","text":"This sentence was dropped by the second reading entirely."}],"strict":"This sentence was dropped by the second reading entirely."},{"complex":[],"container":"p2","context":"paragraph","id":"a7","lossy":"thelastsentenceofthesectionisuniqueandlongenough.","runs":[{"kind":"text","text":"The last sentence of the section is unique and long enough."}],"strict":"The last sentence of the section is unique and long enough."}],"b":[{"complex":[],"container":"x","context":"paragraph","id":"b1","lossy":"methods","runs":[{"kind":"text","text":"Methods"}],"strict":"Methods"},{"complex":[],"container":"q1","context":"paragraph","id":"b2","lossy":"everybondwasscannedinstepsof0.01åarounditsminimum.","runs":[{"kind":"text","text":"Every bond was scanned in steps of 0.01 Å around its minimum."}],"strict":"Every bond was scanned in steps of 0.01 Å around its minimum."},{"complex":["tex:\\frac{a}{b}"],"container":"q1","context":"paragraph","id":"b3","lossy":"whereisthebondlength.","runs":[{"kind":"text","text":"where "},{"kind":"math","notation":"tex","source":"\\frac{a}{b}"},{"kind":"text","text":" is the bond length."}],"strict":"where  is the bond length."},{"complex":[],"container":"q2","context":"paragraph","id":"b4","lossy":"thefittedrateis103persecondatroomtemperature.","runs":[{"kind":"text","text":"The fitted rate is 103 per second at room temperature."}],"strict":"The fitted rate is 103 per second at room temperature."},{"complex":["tex:\\frac{c}{e}"],"container":"q2","context":"paragraph","id":"b5","lossy":"whereisthebondlength.","runs":[{"kind":"text","text":"where "},{"kind":"math","notation":"tex","source":"\\frac{c}{e}"},{"kind":"text","text":" is the bond length."}],"strict":"where  is the bond length."},{"complex":[],"container":"q2","context":"paragraph","id":"b6","lossy":"asentenceonlythesecondreadinghas,alsolong.","runs":[{"kind":"text","text":"A sentence only the second reading has, also long."}],"strict":"A sentence only the second reading has, also long."},{"complex":[],"container":"q2","context":"paragraph","id":"b7","lossy":"thelastsentenceofthesectionisuniqueandlongenough.","runs":[{"kind":"text","text":"The last sentence of the section is unique and long enough."}],"strict":"The last sentence of the section is unique and long enough."}],"expect":{"pairs":[{"a":"a1","agree":true,"b":"b1"},{"a":"a2","agree":true,"b":"b2"},{"a":"a3","agree":true,"b":"b3"},{"a":"a4","agree":false,"b":"b4"},{"a":"a5","agree":false,"b":"b5"},{"a":"a7","agree":true,"b":"b7"}],"suspects":[{"a":"a1","b":"b1","kind":"structure"},{"a":"a4","b":"b4","kind":"number"},{"a":"a5","b":"b5","kind":"equation"},{"a":"a4","b":"b4","kind":"join"},{"a":"a5","b":"b5","kind":"join"},{"a":"a6","b":null,"kind":"dropped"},{"a":null,"b":"b6","kind":"added"}]},"id":"two-readings"}
{"a":[{"complex":[],"container":"p1","context":"paragraph","id":"c1","lossy":"theforceconstantswerefittedtoabinitioscansofeverybond.","runs":[{"kind":"text","text":"The force constants were fitted to ab initio scans of every bond."}],"strict":"The force constants were fitted to ab initio scans of every bond."},{"complex":[],"container":"p1","context":"paragraph","id":"c2","lossy":"thebondconstantwas310.0kcal/molforeveryaliphaticcarbon.","runs":[{"kind":"text","text":"The bond constant was 310.0 kcal/mol for every aliphatic carbon."}],"strict":"The bond constant was 310.0 kcal/mol for every aliphatic carbon."},{"complex":[],"container":"p1","context":"paragraph","id":"c3","lossy":"thisprototypalsetwaslaterextendedtonucleicacids.","runs":[{"kind":"text","text":"This prototypal set was later extended to nucleic acids."}],"strict":"This prototypal set was later extended to nucleic acids."},{"complex":[],"container":"p1","context":"paragraph","id":"c4","lossy":"seebelow.","runs":[{"kind":"text","text":"See below."}],"strict":"See below."},{"complex":[],"container":"p1","context":"paragraph","id":"c5","lossy":"torsionswererefitagainstasecond,largersetofconformers.","runs":[{"kind":"text","text":"Torsions were refit against a second, larger set of conformers."}],"strict":"Torsions were refit against a second, larger set of conformers."},{"complex":[],"container":"p2","context":"paragraph","id":"c6","lossy":"thechargescamefromatwostagerestrainedfittothepotential.","runs":[{"kind":"text","text":"The charges came from a two-stage restrained fit to the potential."}],"strict":"The charges came from a two-stage restrained fit to the potential."}],"b":[{"complex":[],"container":"q1","context":"paragraph","id":"d1","lossy":"theforceconstantswerefittedtoabinitioscansofeverybond.","runs":[{"kind":"text","text":"The force constants were fitted to ab initio scans of every bond."}],"strict":"The force constants were fitted to ab initio scans of every bond."},{"complex":[],"container":"q1","context":"paragraph","id":"d2","lossy":"thebondconstantwas3l0.0kcal/molforeveryaliphaticcarbon.","runs":[{"kind":"text","text":"The bond constant was 3l0.0 kcal/mol for every aliphatic carbon."}],"strict":"The bond constant was 3l0.0 kcal/mol for every aliphatic carbon."},{"complex":[],"container":"q1","context":"paragraph","id":"d3","lossy":"thisprototypicalsetwaslaterextendedtonucleicacids.","runs":[{"kind":"text","text":"This prototypical set was later extended to nucleic acids."}],"strict":"This prototypical set was later extended to nucleic acids."},{"complex":[],"container":"q1","context":"paragraph","id":"d4","lossy":"seebclow.","runs":[{"kind":"text","text":"See bclow."}],"strict":"See bclow."},{"complex":[],"container":"q1","context":"paragraph","id":"d5","lossy":"onlythesecondreadingprintsthislineaboutwatermodels.","runs":[{"kind":"text","text":"Only the second reading prints this line about water models."}],"strict":"Only the second reading prints this line about water models."},{"complex":[],"container":"q2","context":"paragraph","id":"d6","lossy":"thechargescamefromatwostagerestrainedfittothepotential.","runs":[{"kind":"text","text":"The charges came from a two-stage restrained fit to the potential."}],"strict":"The charges came from a two-stage restrained fit to the potential."}],"expect":{"pairs":[{"a":"c1","agree":true,"b":"d1"},{"a":"c2","agree":false,"b":"d2"},{"a":"c3","agree":false,"b":"d3"},{"a":"c6","agree":true,"b":"d6"}],"suspects":[{"a":"c2","b":"d2","kind":"number"},{"a":"c3","b":"d3","kind":"word"},{"a":"c4","b":null,"kind":"dropped"},{"a":"c5","b":null,"kind":"dropped"},{"a":null,"b":"d4","kind":"added"},{"a":null,"b":"d5","kind":"added"}]},"id":"substitutions"}