alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
3.3 KiB
19. Exercises, submissions and grades
Because every sentence and cell has a permanent address, the answer to a verification exercise is an address, and an address can be graded.
The three formats of this chapter, exercise, submission and grade, are provisional (0.4): no course has used them yet, and the first that does will shape them.
19.1 Exercises
An exercise is an annotation of kind exercise whose exercise property holds a scienting-exercise/1: an exercise kind, a prompt, a key and a rule.
| Kind | The student | The key |
|---|---|---|
anchor |
Finds the sentence or cell that supports a statement. | elements: each a set of acceptable anchors, with points and feedback. |
check |
Decides whether a cited passage supports the citing sentence, and anchors the passage. | A verdict with points, and elements. |
trace |
Follows a value back to its original measurement. | A chain: an element per link, in order. |
reproduce |
Digitizes a plot and recomputes a value. | A value, a tolerance, a unit, the confirmed digitization it comes from, and points. |
verify |
Checks a model's summary, written with addresses, in which errors were planted. |
errors: each a set of anchors and the correct verdict, with points. |
The rule's match is same-node (the student's anchor resolves to the node a key anchor resolves to), overlap (and, where the key has a span, the student's span covers at least min_overlap of it; default 0.5), or chain (each link matches in order, with credit per link).
Feedback is {hit, miss} per element.
19.2 Verdicts
| Verdict | Means | As claim evidence, if a person confirms it |
|---|---|---|
supports |
The passage says what the citing sentence claims. | supports |
partly |
It says some of it, or with a qualification the citing sentence drops. | qualifies |
does-not |
It does not say it, or says the opposite. | disputes, or no record when merely irrelevant; the reviewer chooses. |
cannot-tell |
The passage, or the student's access to it, is not enough. | No record. |
19.3 Submissions
A submission, scienting-submission/1, holds the exercise's ID, the document and revision, a pseudonymous student ID, and answers: anchors, a verdict, a chain of anchor lists, a value, or flags (each an anchor and a verdict), as the kind needs.
The student's name never enters a submission; the mapping from pseudonym to student stays in the course's records.
19.4 Grading
Before grading, every anchor in the key is resolved, and if any is stale nothing is graded with it.
Each element of the key earns its points when any of the student's anchors matches any of its anchors under the rule; a check earns its verdict's points when the verdicts are equal; a reproduce earns its points when the value is within the tolerance; a verify earns an error's points when a flag matches it with the correct verdict.
Points are decimal strings, summed exactly.
The result is a scienting-grade/1: the submission, the exercise, the score, the max, and an item per element with its points, its maximum and its feedback.
grading.jsonl grades a hit, a miss and an overlap.
Student submissions are educational records, so they live in the course's private store, never on a public log.