alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
70 lines
2.9 KiB
Python
70 lines
2.9 KiB
Python
"""The target set of targeted F3: the nodes a team's claims, data and quotations rest on.
|
|
|
|
Non-normative; see spec/08-fidelity.md §8.10.
|
|
"""
|
|
|
|
from .model import LEAVES, MATH_KINDS
|
|
from .text import is_digit
|
|
|
|
DIGIT_RULES = ("all", "anchored-sections", "none")
|
|
# The kinds that count as the section a node sits in: a section, or the front or back matter outside any.
|
|
SECTION_KINDS = frozenset(["section", "front", "back"])
|
|
DIGIT_KINDS = frozenset(["sentence", "cell"])
|
|
DIGIT_RUNS = frozenset(["text", "math"])
|
|
|
|
|
|
def has_digit_run(node):
|
|
"""Whether a `text` or `math` run of the node holds a digit (General Category Nd)."""
|
|
for r in node.get("content") or []:
|
|
if not isinstance(r, dict) or r.get("kind") not in DIGIT_RUNS:
|
|
continue
|
|
s = r.get("text") if r["kind"] == "text" else r.get("source")
|
|
if any(is_digit(ch) for ch in s or ""):
|
|
return True
|
|
return False
|
|
|
|
|
|
def enclosing_section(doc, nid):
|
|
"""The nearest ancestor-or-self that is a section, or the front or back matter, else the root."""
|
|
if doc.nodes[nid]["kind"] in SECTION_KINDS:
|
|
return nid
|
|
for a in reversed(doc.ancestors(nid)):
|
|
if doc.nodes[a]["kind"] in SECTION_KINDS:
|
|
return a
|
|
return doc.root
|
|
|
|
|
|
def target_set(doc, anchored, digit_rule="all"):
|
|
"""The target set as `[[node, reason], …]` in document order, with `reason` the first that applies of
|
|
`anchored`, `equation` and `digits`.
|
|
|
|
`anchored` lists the live nodes that extracts, dataset values, confirmed claim links, draft quotations or
|
|
pinned citations anchor; a container among them contributes its leaves. Every equation and equation line
|
|
is a target. Under `all`, so is every sentence or cell with a digit in a text or math run; under
|
|
`anchored-sections`, only those in a section that holds an anchored node; under `none`, none.
|
|
"""
|
|
if digit_rule not in DIGIT_RULES:
|
|
raise ValueError("digit rule must be one of %s" % ", ".join(DIGIT_RULES))
|
|
anchored = set(anchored)
|
|
unknown = sorted(a for a in anchored if a not in doc.nodes)
|
|
if unknown:
|
|
raise ValueError("anchored nodes are not live: %s" % ", ".join(unknown))
|
|
reasons = {}
|
|
for nid in sorted(anchored):
|
|
if doc.nodes[nid]["kind"] in LEAVES:
|
|
reasons[nid] = "anchored"
|
|
else:
|
|
for leaf in doc.leaves(nid):
|
|
reasons.setdefault(leaf, "anchored")
|
|
for nid in doc.preorder():
|
|
if doc.nodes[nid]["kind"] in MATH_KINDS:
|
|
reasons.setdefault(nid, "equation")
|
|
if digit_rule != "none":
|
|
sections = {enclosing_section(doc, a) for a in anchored}
|
|
for nid in doc.preorder():
|
|
n = doc.nodes[nid]
|
|
if n["kind"] in DIGIT_KINDS and has_digit_run(n):
|
|
if digit_rule == "all" or enclosing_section(doc, nid) in sections:
|
|
reasons.setdefault(nid, "digits")
|
|
return [[nid, reasons[nid]] for nid in doc.preorder() if nid in reasons]
|