Initial commit

spec/19-exercises.md

@@ -0,0 +1,42 @@
+# 19. Exercises, submissions and grades
+
+Because every sentence and cell has a permanent address, the answer to a verification exercise is an address, and an address can be graded.
+
+## 19.1 Exercises
+
+An exercise is an annotation of kind `exercise` whose `exercise` property holds a `scienting-exercise/1`: an `exercise` kind, a `prompt`, a `key` and a `rule`.
+
+| Kind | The student | The key |
+| --- | --- | --- |
+| `anchor` | Finds the sentence or cell that supports a statement. | `elements`: each a set of acceptable anchors, with points and feedback. |
+| `check` | Decides whether a cited passage supports the citing sentence, and anchors the passage. | A `verdict` with points, and `elements`. |
+| `trace` | Follows a value back to its original measurement. | A `chain`: an element per link, in order. |
+| `reproduce` | Digitizes a plot and recomputes a value. | A `value`, a `tolerance`, a `unit`, the confirmed `digitization` it comes from, and points. |
+| `verify` | Checks a model's `summary`, written with addresses, in which errors were planted. | `errors`: each a set of anchors and the correct verdict, with points. |
+
+The rule's `match` is `same-node` (the student's anchor resolves to the node a key anchor resolves to), `overlap` (and, where the key has a span, the student's span covers at least `min_overlap` of it; default `0.5`), or `chain` (each link matches in order, with credit per link).
+Feedback is `{hit, miss}` per element.
+
+## 19.2 Verdicts
+
+| Verdict | Means | As Lode evidence, if a person confirms it |
+| --- | --- | --- |
+| `supports` | The passage says what the citing sentence claims. | `supports` |
+| `partly` | It says some of it, or with a qualification the citing sentence drops. | `qualifies` |
+| `does-not` | It does not say it, or says the opposite. | `disputes`, or no record when merely irrelevant; the reviewer chooses. |
+| `cannot-tell` | The passage, or the student's access to it, is not enough. | No record. |
+
+## 19.3 Submissions
+
+A submission, `scienting-submission/1`, holds the exercise's ID, the document and revision, a pseudonymous `student` ID, and `answers`: `anchors`, a `verdict`, a `chain` of anchor lists, a `value`, or `flags` (each an anchor and a verdict), as the kind needs.
+The student's name never enters a submission; the mapping from pseudonym to student stays in the course's records.
+
+## 19.4 Grading
+
+Before grading, every anchor in the key is resolved, and if any is stale nothing is graded with it.
+Each element of the key earns its points when any of the student's anchors matches any of its anchors under the rule; a `check` earns its verdict's points when the verdicts are equal; a `reproduce` earns its points when the value is within the tolerance; a `verify` earns an error's points when a flag matches it with the correct verdict.
+Points are decimal strings, summed exactly.
+The result is a `scienting-grade/1`: the submission, the exercise, the `score`, the `max`, and an item per element with its points, its maximum and its feedback.
+`grading.jsonl` grades a hit, a miss and an overlap.
+
+Student submissions are educational records, so they live in the course's private store, never on a public log.