Files
coursebank/docs/design/2.1/PIPELINE-DESIGN.md
T
alexmandClaude 51a2fd235f feat(schema): 2.1 types, fragments, and JSON Schemas
Add the term, equation, value, and misconception registries
(src/model/course/glossary.rs) and the new optional fields on objectives,
targets, lectures, readings, bank options, and policy. Load registry
entries from objective files, glossary.yaml, and course.yaml, with one
definition site per id as for every other registry. Port every edit in
docs/design/2.1/build_schemas.py into the hand-written schema builders,
add glossary.schema.json, and test the generated schemas against the
targets; the only differences are two 2.0 spellings the code already had.

The 2.0 example fragments load and validate unchanged. The 2.1 example
bank gains `status: draft`, which every coursebank item requires.
tests/docs.rs skips docs/design/, which holds design records rather than
guide pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:55:18 -05:00

15 KiB

The lecture-spec pipeline: design

A sixth pipeline, upstream of the other five, that prepares, assesses, improves, audits, and proofreads the coursebank fragments that define a lecture and go on the course website: the lecture's description and materials, its objectives and targets (with explanations, ceilings, and item-writing notes), its readings, and the terms, equations, values, and misconceptions its targets need. It replaces the single objectives prompt with stages that each do one job in their own context, check each other blind, and stop for the instructor at the two decisions that matter.

This is the design for review. It is built after coursebank 2.1 validates the new format, so its mechanical gate is coursebank validate and coursebank lint rather than a second copy of those rules.

What it produces

For one lecture (or a module, in audit mode):

  • the fragments, as they will sit in the course repository: lectures/l-<id>.yaml, objectives/lo-<id>.yaml for every objective it creates or edits (objectives, targets, and the glossary entries their targets first need), and additions to references.yaml marked "verify before use";
  • the rendered lecture page, as students will see it;
  • CHECK-ME.md: what the instructor must verify by hand (proposed references, locators that could not be found in the supplied reading text, ceiling disputes decided by judgment, every assumption), each with its evidence;
  • a review packet: the blind readers' findings and what was done about each.

Everything comes back as files, in one zip, per the shared driver rules. Nothing is committed to the course repository unless the instructor asks.

Where it sits

material, readings, prior lectures
        │
        ▼
  lecture-spec pipeline ──► coursebank fragments (the source of truth) ──► lecture page on the website
        │                              │
        │                              ├──► writing pipeline: the brief expands each target's explanation; terms, values from the registry
        │                              ├──► lecture pipeline: targets, terms, equations, misconceptions (the first source for mistake boxes)
        │                              └──► MCQ pipeline: ceilings, `authoring` notes, misconception ids for distractors
        ▼
  CHECK-ME.md, review packet

The three downstream pipelines stop reading terms.yaml, values.yaml, and their own misconception notes, and call coursebank lint for the checks those files fed.

Modes

mode input what runs output
build a lecture's material (slides, notes, outline, draft objectives, readings) every stage new fragments
revise an existing lecture's fragments, plus what changed and why every stage, starting from the current fragments; ids kept edits, shown as diffs
audit an existing lecture's fragments intake, the mechanical gate, the blind readers, synthesis findings only; nothing changed
proofread an existing lecture's fragments the mechanical gate and the proofreader, on student-facing fields only exact text edits
module every lecture in a module the module auditor, after an audit of each lecture module findings
link the migrated glossary.yaml a one-time pass per lecture: list each migrated entry on the targets that need it, move it to its objective file edits

Stages

The driver is the shared one (orchestrator/SUBAGENTS.md): one chat, a run-state file, sealed prompts for blind readers, files back as bundles. Models follow the shared role table.

# stage who model, effort blind writes
0 Intake clerk Sonnet, medium sources/: the registry slice (coursebank course build), the course policy, the earlier lectures' fragments, the material, each reading extracted to text with its headings kept, a files request for what is missing
1 Inventory clerk, one per material file when there are several Sonnet, high inventory.yaml: every claim, case, term as used, equation, number, flagged mistake, and reading section in the material, each with a pointer to where it is. Later stages cite inventory entries, which is what makes "never fabricate" checkable
2 Objectives and targets author Opus, high the objectives (top down, four to eight), the targets (three to eight each), ceilings with authoring.ceiling_item, supplied, constraints, number_work, relation_form, examples; prerequisites; the teaches order; the lecture description; a short read of the lecture and the scaffolding table
Gate A instructor approves the claims and ceilings. Everything after this hangs on them, so this is where a change is cheap
3 Glossary and explanations author, one per objective, in parallel Opus, high each objective's description, each target's explanation, and the terms, equations, values, and misconceptions its targets list: reused by id when the registry has them, drafted as status: proposed when it does not
3j Join clerk Sonnet, medium one set of fragments; two objectives proposing the same term, or one term defined twice in different words, go to the judge
4 Readings probe-cites, one per reference, in parallel Sonnet, high each reading's targets, summary, focus, skip, minutes. A locator must be a heading or page found in the extracted reading; one that is not becomes a CHECK-ME line, never a guess
5 Mechanical gate driver coursebank validate, coursebank lint, speclint (below). Errors go back to the stage that owns them, in the same subagent
6 Blind readers six, in one message see below yes review/responses/
7 Synthesis judge Opus, high review/findings.md, each finding routed to the stage that fixes it
8 Revise the authors of stages 2 to 4, continued Opus, high the fixes; then stage 5 again, and the blind readers again on whatever changed
9 Proofread proofreader Sonnet, high exact replacements in student-facing fields only: voice, register, the course's banned words and punctuation. A replacement that would change meaning is routed back instead of applied
10 Hand-off clerk Sonnet, medium the fragments placed in the repository's working tree (not committed), the rendered lecture page, CHECK-ME.md, the bundle
Gate B instructor approves; status: proposed comes off what was approved

The blind readers (stage 6)

reader sees asks why
item writer (Sonnet, high; one per objective) each target's text, explanation, and listed glossary entries, and the course's exam conditions; not the ceiling write the most demanding fair item you can for this target, say its level, what the stem supplies, the arithmetic and the time it takes; then try one level higher and say why it fails the objectives prompt asked the same model to set a ceiling and to justify it. A writer who does not know the ceiling either reaches it or does not: below means the ceiling is inflated or the text unclear, above means it is set too low. It also catches arithmetic the exam conditions rule out, and stems that need a constraint
student reading the target list (Sonnet, high) the rendered lecture page and the course guide's section on reading targets for each target, write the question you think it promises and what you would practise; mark any word you could not look up the guide tells students "a target is a question template with the options removed". If a student cannot name the question, the target fails the promise. Also catches a response format named in the text, and a term used before the page defines it
glossary novice (Sonnet, high) the terms, equations, values, and misconceptions only, in the order the page shows them which definitions you could not follow, which used a word not yet defined, which symbol you could not place, which implication you could not see, which correction did not convince you definitions and equation entries are what students study from; they are read cold here, the way a student meets them on the page
transfer probe (Sonnet, medium) each target's text only write one case, not from any lecture, on which this target could be tested; say if you cannot the prompt's rule "could a student meet this target on a case they have never seen" becomes a test. The synthesis compares the cases with the lecture's own examples in inventory.yaml
accuracy reviewer (Opus, high; not blind) everything, with the extracted readings and the material is every definition, equation, implication, assumption, failure case, value, correction, summary, and reference true and supported by the sources; does each textbook name exist in the reading a wrong definition or value is now rendered on the website and copied into slides, briefs, and items. One reviewer checks it before it spreads
leak check (Sonnet, high; one per target with a level 3 or 4 ceiling) the lecture page as a student sees it, then that target's authoring.ceiling_item could you answer this item from the page alone, by finding the sentence an explanation that states the answer to the hardest item on its own target turns a level 3 item into reading. The fix is in the explanation, not the item

speclint: the pipeline's own checks

What coursebank does not check, because it is a pipeline norm rather than a format rule. Each reads its numbers from the pipeline's course config, so a course can change them.

  • Counts. Objectives per lecture (four to eight), targets per objective (three to eight), targets per lecture (a warning band the course sets; the objectives prompt's 30 to 45 and the guide's "twelve to forty" disagree, so this is one number per course, flagged both ways), two dense lectures in a row.
  • Ceilings. The leading verb of each target against the course's verb table; level spread per lecture and whether the module can supply the blueprint, bonus included; reportability (an objective that could not carry two items on one exam).
  • Exam conditions. When policy.exam_conditions.calculator is false: targets with number_work longer than the item budget, and the conceptual-to-numeric ratio in a lecture that carries an equation (at least two to one). A response format named in the text ("select", "in one sentence").
  • Examples. A proper noun from the inventory's examples appearing in target text, for the transfer probe and the synthesis to judge.
  • Text. Explanation length (two to four sentences), definitions of one sentence, the course's prose register (no em dashes, the banned vocabulary list), second person and verb first for objective and target text.
  • Reuse. A drafted target whose text is close to an existing one elsewhere in the registry: a revisit to register on the lecture, not a new id.
  • Readings. Every assessed target has an assigned reading or is flagged; every locator is found in the extracted reading; no (ref, locator) twice.
  • Prerequisites. Reach back four or more lectures with no revisit, for the lecture to re-activate deliberately.

The shared modules (the prompt, rewritten)

The objectives prompt's content survives, split by the stage that needs it and corrected where it disagreed with the course or with itself.

module from the prompt changed
registry-format.md "The schema you must match" the 2.1 format: glossary registries, explanations, authoring, brace references, where entries live
levels-and-ceilings.md the level scheme, the ladder, the verb table, the rules course level names from policy.levels; the verb table is the guide's (define, state, name, identify / explain, compare, classify, interpret / predict or order given a change / several conditions / best explanation or a flaw), so targets, items, and the guide use one list
exam-conditions.md "closed-book, calculator-free, and all multiple choice" read from policy.exam_conditions and policy.blueprints; the calculator rules apply only when the course says so
relations.md "six ways to test a relation" the six forms are now the implications of an equation entry, and the source of level 2 and 3 targets on it; what each symbol means, what it assumes, where it fails
concepts-not-examples.md "concepts, not the examples that teach them" examples drawn from several fields, not one; where each example goes is recorded in authoring.examples
format-independence.md the four rules and the constructed-response tag unchanged in substance
glossary-writing.md new how to write a definition, a term's kind and textbook names, an equation entry, a value's say, a misconception's belief and correction
readings.md the readings section and "never fabricate" locators verified against the extracted text; minutes
house-style.md, prose-register.md house style, prose quality the course voice from course-context.md; shared with the writing pipeline's register rules
course-contract.md new the promises in Learning to Learn that a target list must keep: targets as question templates, ceilings as published, common mistakes as the source of distractors, numbers stated one way

What changes from the objectives prompt

  1. Course rules come from the course. The prompt hard-codes one course's exam conditions, blueprint, density norms, and banned words. They now come from policy and the pipeline's course config, so the same pipeline serves every course.
  2. One judgment per context. The prompt asks one model to inventory, decide objectives, set ceilings, write readings, and grade itself, in one pass. The stages separate them, and the decision that matters most, the claims and their ceilings, gets its own context and the instructor's gate before anything else is built on it.
  3. Ceilings are tested, not asserted. The blind item writer replaces the prompt's self-written ceiling notes.
  4. "Never fabricate" is checked. Inventory pointers, locators matched against extracted readings, and an accuracy reviewer with the sources in front of it.
  5. The judgments are kept. The ceiling evidence, the stem constraints, the supplied data, and where each example went live in authoring, where the item writer and the slide drafter will find them, instead of in a chat transcript.
  6. The lecture is fully specified. Terms, equations, values, and misconceptions, each defined once, with the short canonical text students see on the page.
  7. Student-facing text gets a proofread by a reader who did not write it, limited to exact replacements so ids and meanings cannot drift.
  8. The module is a mode, not a suggestion at the end of the prompt.

Decisions to make when it is built

  • Whether the hand-off writes into the course repository's working tree (shown as a diff) or only returns the fragments in the bundle.
  • Which reading formats you can supply for locator checks (text exports, HTML, PDF), since a locator is verified against the extracted text.
  • The pipeline's course config: the density band, the verb table, the banned words and punctuation, the voice notes, per course.
  • Whether Gate A also shows the item writer's results (one more stop, but ceilings argued with evidence in hand) or only the drafted claims.