Add the term, equation, value, and misconception registries (src/model/course/glossary.rs) and the new optional fields on objectives, targets, lectures, readings, bank options, and policy. Load registry entries from objective files, glossary.yaml, and course.yaml, with one definition site per id as for every other registry. Port every edit in docs/design/2.1/build_schemas.py into the hand-written schema builders, add glossary.schema.json, and test the generated schemas against the targets; the only differences are two 2.0 spellings the code already had. The 2.0 example fragments load and validate unchanged. The 2.1 example bank gains `status: draft`, which every coursebank item requires. tests/docs.rs skips docs/design/, which holds design records rather than guide pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
125 lines
15 KiB
Markdown
125 lines
15 KiB
Markdown
# The lecture-spec pipeline: design
|
|
|
|
A sixth pipeline, upstream of the other five, that prepares, assesses, improves, audits, and proofreads the coursebank fragments that define a lecture and go on the course website: the lecture's description and materials, its objectives and targets (with explanations, ceilings, and item-writing notes), its readings, and the terms, equations, values, and misconceptions its targets need. It replaces the single objectives prompt with stages that each do one job in their own context, check each other blind, and stop for the instructor at the two decisions that matter.
|
|
|
|
This is the design for review. It is built after coursebank 2.1 validates the new format, so its mechanical gate is `coursebank validate` and `coursebank lint` rather than a second copy of those rules.
|
|
|
|
## What it produces
|
|
|
|
For one lecture (or a module, in audit mode):
|
|
|
|
- the fragments, as they will sit in the course repository: `lectures/l-<id>.yaml`, `objectives/lo-<id>.yaml` for every objective it creates or edits (objectives, targets, and the glossary entries their targets first need), and additions to `references.yaml` marked "verify before use";
|
|
- the rendered lecture page, as students will see it;
|
|
- `CHECK-ME.md`: what the instructor must verify by hand (proposed references, locators that could not be found in the supplied reading text, ceiling disputes decided by judgment, every assumption), each with its evidence;
|
|
- a review packet: the blind readers' findings and what was done about each.
|
|
|
|
Everything comes back as files, in one zip, per the shared driver rules. Nothing is committed to the course repository unless the instructor asks.
|
|
|
|
## Where it sits
|
|
|
|
```
|
|
material, readings, prior lectures
|
|
│
|
|
▼
|
|
lecture-spec pipeline ──► coursebank fragments (the source of truth) ──► lecture page on the website
|
|
│ │
|
|
│ ├──► writing pipeline: the brief expands each target's explanation; terms, values from the registry
|
|
│ ├──► lecture pipeline: targets, terms, equations, misconceptions (the first source for mistake boxes)
|
|
│ └──► MCQ pipeline: ceilings, `authoring` notes, misconception ids for distractors
|
|
▼
|
|
CHECK-ME.md, review packet
|
|
```
|
|
|
|
The three downstream pipelines stop reading `terms.yaml`, `values.yaml`, and their own misconception notes, and call `coursebank lint` for the checks those files fed.
|
|
|
|
## Modes
|
|
|
|
| mode | input | what runs | output |
|
|
|---|---|---|---|
|
|
| **build** | a lecture's material (slides, notes, outline, draft objectives, readings) | every stage | new fragments |
|
|
| **revise** | an existing lecture's fragments, plus what changed and why | every stage, starting from the current fragments; ids kept | edits, shown as diffs |
|
|
| **audit** | an existing lecture's fragments | intake, the mechanical gate, the blind readers, synthesis | findings only; nothing changed |
|
|
| **proofread** | an existing lecture's fragments | the mechanical gate and the proofreader, on student-facing fields only | exact text edits |
|
|
| **module** | every lecture in a module | the module auditor, after an audit of each lecture | module findings |
|
|
| **link** | the migrated `glossary.yaml` | a one-time pass per lecture: list each migrated entry on the targets that need it, move it to its objective file | edits |
|
|
|
|
## Stages
|
|
|
|
The driver is the shared one (`orchestrator/SUBAGENTS.md`): one chat, a run-state file, sealed prompts for blind readers, files back as bundles. Models follow the shared role table.
|
|
|
|
| # | stage | who | model, effort | blind | writes |
|
|
|---|---|---|---|---|---|
|
|
| 0 | **Intake** | clerk | Sonnet, medium | | `sources/`: the registry slice (`coursebank course build`), the course policy, the earlier lectures' fragments, the material, each reading extracted to text with its headings kept, a files request for what is missing |
|
|
| 1 | **Inventory** | clerk, one per material file when there are several | Sonnet, high | | `inventory.yaml`: every claim, case, term as used, equation, number, flagged mistake, and reading section in the material, each with a pointer to where it is. Later stages cite inventory entries, which is what makes "never fabricate" checkable |
|
|
| 2 | **Objectives and targets** | author | Opus, high | | the objectives (top down, four to eight), the targets (three to eight each), ceilings with `authoring.ceiling_item`, `supplied`, `constraints`, `number_work`, `relation_form`, `examples`; prerequisites; the `teaches` order; the lecture `description`; a short read of the lecture and the scaffolding table |
|
|
| | **Gate A** | instructor | | | approves the claims and ceilings. Everything after this hangs on them, so this is where a change is cheap |
|
|
| 3 | **Glossary and explanations** | author, one per objective, in parallel | Opus, high | | each objective's `description`, each target's `explanation`, and the terms, equations, values, and misconceptions its targets list: reused by id when the registry has them, drafted as `status: proposed` when it does not |
|
|
| 3j | **Join** | clerk | Sonnet, medium | | one set of fragments; two objectives proposing the same term, or one term defined twice in different words, go to the judge |
|
|
| 4 | **Readings** | probe-cites, one per reference, in parallel | Sonnet, high | | each reading's `targets`, `summary`, `focus`, `skip`, `minutes`. A locator must be a heading or page found in the extracted reading; one that is not becomes a CHECK-ME line, never a guess |
|
|
| 5 | **Mechanical gate** | driver | | | `coursebank validate`, `coursebank lint`, `speclint` (below). Errors go back to the stage that owns them, in the same subagent |
|
|
| 6 | **Blind readers** | six, in one message | see below | yes | `review/responses/` |
|
|
| 7 | **Synthesis** | judge | Opus, high | | `review/findings.md`, each finding routed to the stage that fixes it |
|
|
| 8 | **Revise** | the authors of stages 2 to 4, continued | Opus, high | | the fixes; then stage 5 again, and the blind readers again on whatever changed |
|
|
| 9 | **Proofread** | proofreader | Sonnet, high | | exact replacements in student-facing fields only: voice, register, the course's banned words and punctuation. A replacement that would change meaning is routed back instead of applied |
|
|
| 10 | **Hand-off** | clerk | Sonnet, medium | | the fragments placed in the repository's working tree (not committed), the rendered lecture page, `CHECK-ME.md`, the bundle |
|
|
| | **Gate B** | instructor | | | approves; `status: proposed` comes off what was approved |
|
|
|
|
### The blind readers (stage 6)
|
|
|
|
| reader | sees | asks | why |
|
|
|---|---|---|---|
|
|
| **item writer** (Sonnet, high; one per objective) | each target's text, explanation, and listed glossary entries, and the course's exam conditions; **not** the ceiling | write the most demanding fair item you can for this target, say its level, what the stem supplies, the arithmetic and the time it takes; then try one level higher and say why it fails | the objectives prompt asked the same model to set a ceiling and to justify it. A writer who does not know the ceiling either reaches it or does not: below means the ceiling is inflated or the text unclear, above means it is set too low. It also catches arithmetic the exam conditions rule out, and stems that need a constraint |
|
|
| **student reading the target list** (Sonnet, high) | the rendered lecture page and the course guide's section on reading targets | for each target, write the question you think it promises and what you would practise; mark any word you could not look up | the guide tells students "a target is a question template with the options removed". If a student cannot name the question, the target fails the promise. Also catches a response format named in the text, and a term used before the page defines it |
|
|
| **glossary novice** (Sonnet, high) | the terms, equations, values, and misconceptions only, in the order the page shows them | which definitions you could not follow, which used a word not yet defined, which symbol you could not place, which implication you could not see, which correction did not convince you | definitions and equation entries are what students study from; they are read cold here, the way a student meets them on the page |
|
|
| **transfer probe** (Sonnet, medium) | each target's text only | write one case, not from any lecture, on which this target could be tested; say if you cannot | the prompt's rule "could a student meet this target on a case they have never seen" becomes a test. The synthesis compares the cases with the lecture's own examples in `inventory.yaml` |
|
|
| **accuracy reviewer** (Opus, high; not blind) | everything, with the extracted readings and the material | is every definition, equation, implication, assumption, failure case, value, correction, summary, and reference true and supported by the sources; does each textbook name exist in the reading | a wrong definition or value is now rendered on the website and copied into slides, briefs, and items. One reviewer checks it before it spreads |
|
|
| **leak check** (Sonnet, high; one per target with a level 3 or 4 ceiling) | the lecture page as a student sees it, then that target's `authoring.ceiling_item` | could you answer this item from the page alone, by finding the sentence | an explanation that states the answer to the hardest item on its own target turns a level 3 item into reading. The fix is in the explanation, not the item |
|
|
|
|
## `speclint`: the pipeline's own checks
|
|
|
|
What coursebank does not check, because it is a pipeline norm rather than a format rule. Each reads its numbers from the pipeline's course config, so a course can change them.
|
|
|
|
- **Counts.** Objectives per lecture (four to eight), targets per objective (three to eight), targets per lecture (a warning band the course sets; the objectives prompt's 30 to 45 and the guide's "twelve to forty" disagree, so this is one number per course, flagged both ways), two dense lectures in a row.
|
|
- **Ceilings.** The leading verb of each target against the course's verb table; level spread per lecture and whether the module can supply the blueprint, bonus included; reportability (an objective that could not carry two items on one exam).
|
|
- **Exam conditions.** When `policy.exam_conditions.calculator` is false: targets with `number_work` longer than the item budget, and the conceptual-to-numeric ratio in a lecture that carries an equation (at least two to one). A response format named in the text ("select", "in one sentence").
|
|
- **Examples.** A proper noun from the inventory's examples appearing in target text, for the transfer probe and the synthesis to judge.
|
|
- **Text.** Explanation length (two to four sentences), definitions of one sentence, the course's prose register (no em dashes, the banned vocabulary list), second person and verb first for objective and target text.
|
|
- **Reuse.** A drafted target whose text is close to an existing one elsewhere in the registry: a revisit to register on the lecture, not a new id.
|
|
- **Readings.** Every assessed target has an assigned reading or is flagged; every locator is found in the extracted reading; no `(ref, locator)` twice.
|
|
- **Prerequisites.** Reach back four or more lectures with no revisit, for the lecture to re-activate deliberately.
|
|
|
|
## The shared modules (the prompt, rewritten)
|
|
|
|
The objectives prompt's content survives, split by the stage that needs it and corrected where it disagreed with the course or with itself.
|
|
|
|
| module | from the prompt | changed |
|
|
|---|---|---|
|
|
| `registry-format.md` | "The schema you must match" | the 2.1 format: glossary registries, explanations, `authoring`, brace references, where entries live |
|
|
| `levels-and-ceilings.md` | the level scheme, the ladder, the verb table, the rules | course level names from `policy.levels`; the verb table is the guide's (define, state, name, identify / explain, compare, classify, interpret / predict or order given a change / several conditions / best explanation or a flaw), so targets, items, and the guide use one list |
|
|
| `exam-conditions.md` | "closed-book, calculator-free, and all multiple choice" | read from `policy.exam_conditions` and `policy.blueprints`; the calculator rules apply only when the course says so |
|
|
| `relations.md` | "six ways to test a relation" | the six forms are now the `implications` of an equation entry, and the source of level 2 and 3 targets on it; what each symbol means, what it assumes, where it fails |
|
|
| `concepts-not-examples.md` | "concepts, not the examples that teach them" | examples drawn from several fields, not one; where each example goes is recorded in `authoring.examples` |
|
|
| `format-independence.md` | the four rules and the constructed-response tag | unchanged in substance |
|
|
| `glossary-writing.md` | new | how to write a definition, a term's kind and textbook names, an equation entry, a value's `say`, a misconception's belief and correction |
|
|
| `readings.md` | the readings section and "never fabricate" | locators verified against the extracted text; `minutes` |
|
|
| `house-style.md`, `prose-register.md` | house style, prose quality | the course voice from `course-context.md`; shared with the writing pipeline's register rules |
|
|
| `course-contract.md` | new | the promises in *Learning to Learn* that a target list must keep: targets as question templates, ceilings as published, common mistakes as the source of distractors, numbers stated one way |
|
|
|
|
## What changes from the objectives prompt
|
|
|
|
1. **Course rules come from the course.** The prompt hard-codes one course's exam conditions, blueprint, density norms, and banned words. They now come from `policy` and the pipeline's course config, so the same pipeline serves every course.
|
|
2. **One judgment per context.** The prompt asks one model to inventory, decide objectives, set ceilings, write readings, and grade itself, in one pass. The stages separate them, and the decision that matters most, the claims and their ceilings, gets its own context and the instructor's gate before anything else is built on it.
|
|
3. **Ceilings are tested, not asserted.** The blind item writer replaces the prompt's self-written ceiling notes.
|
|
4. **"Never fabricate" is checked.** Inventory pointers, locators matched against extracted readings, and an accuracy reviewer with the sources in front of it.
|
|
5. **The judgments are kept.** The ceiling evidence, the stem constraints, the supplied data, and where each example went live in `authoring`, where the item writer and the slide drafter will find them, instead of in a chat transcript.
|
|
6. **The lecture is fully specified.** Terms, equations, values, and misconceptions, each defined once, with the short canonical text students see on the page.
|
|
7. **Student-facing text gets a proofread** by a reader who did not write it, limited to exact replacements so ids and meanings cannot drift.
|
|
8. **The module is a mode,** not a suggestion at the end of the prompt.
|
|
|
|
## Decisions to make when it is built
|
|
|
|
- Whether the hand-off writes into the course repository's working tree (shown as a diff) or only returns the fragments in the bundle.
|
|
- Which reading formats you can supply for locator checks (text exports, HTML, PDF), since a locator is verified against the extracted text.
|
|
- The pipeline's course config: the density band, the verb table, the banned words and punctuation, the voice notes, per course.
|
|
- Whether Gate A also shows the item writer's results (one more stop, but ceilings argued with evidence in hand) or only the drafted claims.
|