docs: schema 2.1 guide, codes, changelog, and report

A guide page for the four registries: where entries live, how targets
point at them, the other new fields, the brace-reference table, what
students never see, the lecture and glossary pages, every new error and
lint rule, and the migration from terms.yaml and values.yaml. The setup
guide points at it.

The README covers glossary.yaml, the nine schemas, the 2.1 checks, the
glossary lint family and the note severity, the new lecture commands and
report sections, migrate glossary, and schema version 2.1. Module docs
and the changelog follow.

docs/design/2.1/CODES.md maps the specification's E, W, and I codes to
coursebank's messages and lint codes, each with its test, and REPORT.md
records what changed, the two remaining schema differences, every choice
made where the specification left room, and suggested changes to it. A
new test covers the validation messages the specification does not
number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-11-16 12:31:01 -05:00
1 parent 6c07122fc5
commit b5cae6afac
11 files changed
+535 -29

No files matched your search

+38 -16
View File
@@ -73,15 +73,16 @@ Writing comma-separated values (CSV) instead is available with `ingest --format
The pooled statistics for each item (difficulty, discrimination, per-option selection rates, flags, and any item response theory (IRT) parameters) live in a fifth file, `analysis/calibration.yaml`, written only by `calibrate --apply`.
A command that loads the course attaches those statistics to each item by id, so the statistics are available wherever the item is used, though the bank file never contains them.
The files `course.yaml`, `references.yaml`, `lectures/*.yaml`, and `objectives/*.yaml` are one course file split by subject, and a course that keeps everything in `course.yaml` loads unchanged.
The files `course.yaml`, `references.yaml`, `glossary.yaml`, `lectures/*.yaml`, and `objectives/*.yaml` are one course file split by subject, and a course that keeps everything in `course.yaml` loads unchanged.
The full directory layout is:
| Path | Role | Written by | Ignored by `init`'s `.gitignore` |
|------|------|-----------|---------|
| `course.yaml` | identity, policy, units | you | no |
| `references.yaml` | works cited | you | no |
| `lectures/*.yaml` | one lecture each: readings and what it teaches | you | no |
| `objectives/*.yaml` | one learning objective each, with its learning targets | you | no |
| `glossary.yaml` | terms, equations, values, and misconceptions no single objective owns | you, or `migrate glossary` | no |
| `lectures/*.yaml` | one lecture each: readings, materials, and what it teaches | you | no |
| `objectives/*.yaml` | one learning objective each, with its learning targets and the glossary entries they first need | you | no |
| `banks/*.yaml` | item banks | you | no |
| `assessments/*.yaml` | assessment records | `assemble`, then you | no |
| `seals/*.yaml` | frozen copy of what was printed | `seal` | no |
@@ -93,7 +94,7 @@ The full directory layout is:
| `schema/` | JSON (JavaScript Object Notation) Schemas for editor validation | `init`, `schema` | no |
| `templates/` | Typst templates and `typst.yaml` | `template dump`, `template config` | no |
`coursebank course files` lists the files the course is assembled from, `coursebank course build` prints the merged course, and `coursebank course where ID` says which file defines a given unit, lecture, objective, target, reference, or stimulus id.
`coursebank course files` lists the files the course is assembled from, `coursebank course build` prints the merged course, and `coursebank course where ID` says which file defines a given unit, lecture, objective, target, reference, stimulus, term, equation, value, or misconception id.
Every command merges the fragments in memory, and each id may be defined in only one place.
`course build` is for reading and for other programs: nothing in coursebank reads what it writes.
@@ -291,10 +292,17 @@ An objective is what a report classifies, and a target is what items are tagged
An id may exist in only one of the two registries, and an item that tags an objective that has targets is a validation error: tag the target.
Lectures list the objectives they `teach`, and readings attach to lectures with a `ref`, a `locator`, a `role` (`assigned` or `supplemental`), and the targets they support.
The `policy` block carries defaults such as `points_per_item` (1.0), `options_per_item` (4), `mastery_threshold` (0.75), `min_items_for_mastery` (2), `allow_partial_credit` (true), `bonus_levels`, and a `grade_scale`.
Schema 2.1 adds the course's names for the five levels (`levels`), its `exam_conditions`, named `blueprints`, and `target_text_forbid`, patterns that objective and target text must not match; `exam_conditions` and `blueprints` are stored and validated, and no command reads them yet.
Schema 2.1 also adds four registries beside the objectives and targets: terms (`term-`), equations (`eq-`), values (`val-`), and misconceptions (`mis-`).
Each entry is defined once, in the objective file whose targets first need it or in `glossary.yaml`, and a target lists the entries it turns on and carries a student-facing `explanation`.
Student-facing prose names an entry in braces, such as `{term-gap-penalty}`, and every page renders the reference.
A bank option names the misconception it encodes with `misconception_id`.
The guide's `glossary` page has the fields, the brace table, and the example.
### Editor support
`init` and `schema` write eight JSON Schemas into `schema/`: `course`, `references`, `lecture`, `objective`, `calibration`, `measurements`, `bank`, and `assessment`.
`init` and `schema` write nine JSON Schemas into `schema/`: `course`, `references`, `glossary`, `lecture`, `objective`, `calibration`, `measurements`, `bank`, and `assessment`.
Put a modeline at the top of a YAML file and an editor with a YAML language server validates the file as you type:
```yaml
@@ -309,10 +317,11 @@ Use the `../schema/` form shown above.
`coursebank validate` checks that every file is correct and consistent, and it exits 2 if it finds any problem.
It covers the course (empty identity fields, grade scale, duplicate ids, reference fields, prerequisite cycles, level ceilings), every bank against the course (the rules above, unknown targets, lectures, stimuli, and references), every assessment record against the catalog, every seal's stem digests, and calibration against the banks.
For the 2.1 registries it checks id prefixes, ids unique across every registry, the entries each target lists, equation symbols, values, citation keys, brace references in student-facing prose, that every `conflicts` and `target_text_forbid` pattern compiles, and each option's `misconception_id`.
A bank item that carries a `calibration:` block fails validation.
`coursebank lint` checks craft.
It has 30 rules in five families, and each rule has a code.
It has 44 rules in six families, and each rule has a code.
| Family | Codes | What they flag |
|--------|-------|----------------|
@@ -321,11 +330,13 @@ It has 30 rules in five families, and each rule has a code.
| Completeness (7) | `complete-distractor-design`, `complete-distractor-explanation`, `complete-student-feedback`, `complete-alt-text`, `complete-format-level`, `complete-bonus-policy`, `complete-design-conflict` | missing design intent, feedback, or alt text |
| Pool (3) | `pool-thin`, `pool-unused`, `pool-unjustified-retirement` | too few usable distractors for a form |
| Evidence (5) | `evidence-nonfunctioning`, `evidence-stale`, `evidence-difficulty`, `evidence-discrimination`, `evidence-duplicate-stem` | observed results that contradict the prediction or the text |
| Glossary (14) | `glossary-unused`, `glossary-no-textbook-name`, `glossary-no-exam-wording`, `glossary-circular-definition`, `glossary-coined-in-item`, `glossary-value-unsourced`, `glossary-value-conflict`, `glossary-forbidden-target-wording`, `glossary-unlisted-term`, `glossary-recall-value-printed`, `glossary-equation-no-implications`, `glossary-unresolved-reference`, `glossary-defined-elsewhere`, `glossary-proposed` | entries nothing needs, coined terms in items, printed recall values, conflicting wordings of a number, and entries waiting for approval |
Each finding has a severity of low, medium, or high, and a finding's severity can differ from its rule's nominal one (for example, `cue-uneven-length` is medium when the spread is extreme).
Each finding has a severity of note, low, medium, or high, and a finding's severity can differ from its rule's nominal one (for example, `cue-uneven-length` is medium when the spread is extreme).
Two rules report notes: `glossary-defined-elsewhere` and `glossary-proposed`.
Retired items are skipped, and findings are grouped by item.
The options are `--list-rules` (list every code and severity, then exit), `--only CODES`, `--ignore CODES` (both comma-separated), `--min-severity low|medium|high` (default `low`), and `--no-fail`.
`lint` exits 2 if it finds anything, unless `--no-fail` is given, which is why a pre-commit hook of `coursebank validate && coursebank lint` works.
The options are `--list-rules` (list every code and severity, then exit), `--only CODES`, `--ignore CODES` (both comma-separated), `--min-severity note|low|medium|high` (default `note`), and `--no-fail`.
`lint` exits 2 if it finds anything above a note, unless `--no-fail` is given, which is why a pre-commit hook of `coursebank validate && coursebank lint` works.
To block a commit only on serious findings, add `--min-severity high`.
`coursebank catalog` summarizes the pool: item, bank, and assemblable counts, and counts by status and by level.
@@ -553,6 +564,10 @@ The guide describes the lock as a deterrent, not a grading system of record (`do
`lecture readings ID` and `lecture objectives ID` write the readings block and the learning-objectives block for one lecture, as Quarto Pandoc definition lists (`--style quarto`, the default) or as plain Markdown bullets (`--style plain`).
They print to standard output without `--out`.
`lecture page ID` writes the whole page for one lecture: its description, each objective with its targets (each with its explanation and its ceiling named by `policy.levels`), new and earlier terms, equations, numbers to know, common mistakes, readings with minutes, and materials.
`lecture glossary` writes the course glossary page: every term alphabetically, then every equation and value, each with the lecture that introduces it.
Both take `--style` and `--out`, and `lecture page` takes `--glossary-href` for where links to earlier terms point (default `glossary.html`).
None of these prints `authoring`, `notes_private`, `evidence`, `conflicts`, or `status`.
`references export` writes the bibliography as `hayagriva` (the default, read by Typst), `csl-json` (read by Zotero and Pandoc), or `bibtex`.
The `label`, `role`, `base_url`, and `note` fields do not survive into those formats.
@@ -769,6 +784,8 @@ The student report answers "what should I do next?"
It includes the student's performance by level and by objective, strengths, the objectives to focus on, per-question results with the feedback and hint written for the option the student chose, dropped questions, lectures to review with readings, and a study list.
It leaves out the correct answers, other students' data, and any numeric rank.
Two optional switches change that: `--solutions` adds the worked solution for every missed question (the command warns about the effect on reusing the exam), and `--misconceptions` names the misconception each chosen distractor was written to catch, which is off by default.
Each missed target shows what meeting it looks like, from the target's `explanation`.
A chosen option with a `misconception_id` shows that misconception's correction, and misses that share one across questions are grouped under it.
`--ability` adds the IRT ability estimate and `--no-comparison` omits the comparison with the class.
Section switches turn individual sections off: `--no-questions`, `--no-feedback`, `--no-hints`, `--no-levels`, `--no-objectives`, `--no-strengths`, `--no-focus`, `--no-dropped-questions`, `--no-review-lectures`, and `--no-study`.
Files go to `reports/ID/`, named for the student key, unless `--out DIR` says otherwise.
@@ -792,7 +809,8 @@ A reference link is resolved in this order: an explicit `url`, then `base_url` p
## Migrating an older course
Every `migrate` subcommand takes `--dry-run`, which shows what would change and writes nothing.
The current schema version is `2.0`, and major versions 1 and 2 are readable, so a 1.0 course loads unchanged and migration is optional.
The current schema version is `2.1`, and major versions 1 and 2 are readable, so a 1.0 or 2.0 course loads unchanged and migration is optional.
2.1 is additive: a course that validated under 2.0 validates under 2.1 with no new errors.
| Command | What it does |
|---------|-------------|
@@ -804,20 +822,21 @@ The current schema version is `2.0`, and major versions 1 and 2 are readable, so
| `migrate stems` | drop the ignored `version:` and `history:` fields |
| `migrate options` | rewrite option letters as names derived from option text; run with `--dry-run` first; seals keep their letters on purpose; exits 2 if some items need hand-naming |
| `migrate ids` | rewrite pre-2.0 `bank::item` ids as plain item ids in records, seals, and the response store |
| `migrate glossary --terms FILE --values FILE` | bring a 1.0 `terms.yaml` and `values.yaml` into `glossary.yaml`: ids gain `term-` and `val-`, `course` becomes `term`, `meaning` becomes `definition`, and `first` and `lectures` are dropped because 2.1 derives them; reports renamed ids, every dropped `first` or `lectures` the targets now contradict, and entries no target lists yet; leaves the old files in place and skips entries the course already declares, so a second run is a no-op |
Run `--dry-run` first, and commit before applying.
The hints that some `migrate` commands print afterwards name commands that do not exist (see the limits section).
## Command reference
coursebank has 21 top-level names and 47 runnable commands: 9 single commands and 12 groups holding 38 subcommands.
coursebank has 21 top-level names and 50 runnable commands: 9 single commands and 12 groups holding 41 subcommands.
Every command accepts `--course PATH` (or `-C PATH`) and `--quiet` (or `-q`).
Identifiers are positional.
| Stage | Command | What it does |
|-------|---------|--------------|
| Project setup | `init --code C --title T --term TERM [--with-examples]` | create a course; refuses to overwrite `course.yaml` |
| | `schema` | write the eight JSON Schemas to `schema/` |
| | `schema` | write the nine JSON Schemas to `schema/` |
| | `validate` | check every file for problems that must be fixed; exits 2 on any |
| | `lint [flags]` | check item craft; see Authoring items |
| | `catalog [--coverage] [--topics]` | summarize the pool and objective coverage |
@@ -835,8 +854,8 @@ Identifiers are positional.
| | `calibrate` | pool every stored administration into `analysis/calibration.yaml` |
| | `report students ID`, `report cohort ID` | per-student and instructor reports |
| | `data` | list the stored administrations |
| Lectures | `lecture readings ID`, `lecture objectives ID`, `lecture coverage` | site partials; reading coverage |
| Migration | `migrate split`, `counters`, `order`, `references`, `variants`, `stems`, `options`, `ids` | upgrade an older course; all take `--dry-run` |
| Lectures | `lecture readings ID`, `lecture objectives ID`, `lecture page ID`, `lecture glossary`, `lecture coverage` | site partials; the whole lecture page; the course glossary page; reading coverage |
| Migration | `migrate split`, `counters`, `order`, `references`, `variants`, `stems`, `options`, `ids`, `glossary` | upgrade an older course; all take `--dry-run` |
coursebank has no `import`, `guide`, or `completions` subcommand.
The only help is `--help`, and the guide pages are rustdoc modules.
@@ -1046,12 +1065,13 @@ Each item below states what the code does.
- `export typst --form` takes one value or `all`, not a repeated flag.
- `export typst --variant student-report|cohort-report` is accepted by the parser; its behavior was not verified, so use `report` for those documents.
- `init --with-examples` writes only an empty bank, `banks/example.yaml`, and not an example assessment.
- `init` writes 8 schema files, not the 4 that `docs/guide/setup.md` shows.
- `init` writes 9 schema files, not the 4 that `docs/guide/setup.md` shows.
- `policy.exam_conditions` and `policy.blueprints` are stored and validated, and no command reads them yet; `assemble` still takes its counts from the command line.
- `schema` prints a modeline path (`../.coursebank/schema`) that differs from where it writes the files (`schema/`).
- `--apply` exists only on `calibrate`, and it writes `analysis/calibration.yaml`, not a bank. The `src/main.rs` documentation says every command that modifies a bank prints what it would change and requires `--apply`; the `migrate` commands use `--dry-run` instead.
- There is no `import` subcommand, although `src/main.rs` and a `bank` comment mention a legacy importer.
- The guide pages use `analyze items --assessment exam-2` and similar forms. On the command line the id is positional, and `calibrate` has no assessment argument. The IRT model value is `two-pl`, not `2pl`.
- `lint --rules`, as shown in two guide pages, is `lint --list-rules`. The lint rule count is 30, not the 26 in one doc comment.
- `lint --rules`, as shown in two guide pages, is `lint --list-rules`.
- `migrate counters` and `migrate ids` print follow-up hints that name `seal verify` and `analyze items --all`, neither of which exists. Use `seal --check` to verify.
### Analysis
@@ -1064,6 +1084,8 @@ Each item below states what the code does.
- `#sym.minus`, used in an authoring example, is not in the symbol table.
- Fenced code blocks in stems are not handled by the markup converter.
- The lecture and glossary pages leave `$...$` math and an equation's `tex` for Quarto to render, and a value's `say` is printed as written, so a `say` like `about 10^{59}` shows its caret and braces unless it is written in math markup.
- `conflicts` and `target_text_forbid` patterns use Rust's `regex` syntax, which has no look-around and no backreferences.
- Item figures (`assets`) go into the Typst payload only, and the bundled Typst templates do not print them. The QTI, practice, and site exports do not emit assets.
- The bundled Typst templates import `@preview` packages, which the Typst compiler resolves; whether they are cached or downloaded at compile time was not checked.