Files
coursebank/docs/design/2.1/PLAN.md
T
alexmandClaude 51a2fd235f feat(schema): 2.1 types, fragments, and JSON Schemas
Add the term, equation, value, and misconception registries
(src/model/course/glossary.rs) and the new optional fields on objectives,
targets, lectures, readings, bank options, and policy. Load registry
entries from objective files, glossary.yaml, and course.yaml, with one
definition site per id as for every other registry. Port every edit in
docs/design/2.1/build_schemas.py into the hand-written schema builders,
add glossary.schema.json, and test the generated schemas against the
targets; the only differences are two 2.0 spellings the code already had.

The 2.0 example fragments load and validate unchanged. The 2.1 example
bank gains `status: draft`, which every coursebank item requires.
tests/docs.rs skips docs/design/, which holds design records rather than
guide pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:55:18 -05:00

280 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# coursebank 2.1: implementation plan
Phase 0 output. Written after reading `SCHEMA-CHANGES.md`, the prototype, the examples, and the code on branch
`prelim-use` (clean, 412 unit + 20 integration + 4 doc tests passing). Nothing outside `docs/design/2.1/` has been
changed. Section 7 has three questions whose answers change behaviour; everything else is a proposal I will follow
unless you say otherwise.
## 1. How the schemas are produced
They are not generated from the Rust types. `src/authoring/jsonschema.rs` builds each schema by hand with
`serde_json::json!`, one function per object (`course_schema`, `objective_schema`, `target_schema`, `lecture_schema`,
`reading_mapping_schema`, `option_schema`, `policy_schema`, ...). The module doc says why: hand-written descriptions
and enum lists are better tooltips than a derivation. There is no schemars, and the Rust doc comments are *not* the
`description` strings.
- Regenerate: `coursebank schema` (writes `<course>/schema/*.schema.json` through `jsonschema::write_all`), or
`pixi run cb schema -C <course>`.
- `Kind` enumerates the eight files; `glossary.schema.json` needs a ninth, `Kind::Glossary`.
- Baseline check: I generated today's schemas and compared them with `schemas-original/` as JSON values. Six match
exactly. Two differ by one word each, already in the code before 2.1: `behaviour` (calibration, per-option
history) and `Colour` (course, `grade_scale[].group`). `schemas-original/` has the American spellings, which no
commit in this repository ever had. I will keep the code's spelling and list both as known differences.
**Effect on rule 2.** "Change the types and their doc comments, and regenerate" becomes: change the types (with doc
comments in the repository's style), and port every edit in `build_schemas.py` into `jsonschema.rs` with the
description strings copied word for word. Hand-written JSON can state `propertyNames` directly, so the id patterns go
into the schema. The validator enforces them as well, because nothing checks the YAML against the JSON Schema at load
time (serde only). An integration test compares each generated schema with `docs/design/2.1/schemas/` as a
`serde_json::Value` (order-insensitive), with an explicit allow-list of known differences. I expect the list to hold
only the two spellings above.
## 2. How fragments are discovered and merged
`src/model/course/fragment.rs`, entry point `CourseFile::load_dir` → `fragment::assemble(root)`:
- `files(layout)` lists, in order: `course.yaml` (always, role `Root`), `references.yaml` if present
(`References`), sorted `lectures/*.yaml` (`Lecture`), sorted `objectives/*.yaml` (`Objective`). Banks are not
fragments; `Catalog` loads them separately.
- Each file is parsed into `Fragment` (`deny_unknown_fields`; every section optional). `Role::allows(section)`
confines a role to its sections: an objective file may define objectives, targets, and stimuli; `course.yaml` may
define anything.
- `Merge::take` → `absorb` keeps `origins: BTreeMap<(Section, id), path>`. An id that a second file defines **in the
same section** is reported by `duplicate()` (`"<noun> `id` is defined in two places: A and B. An id has one
definition site..."`), and every merge problem comes back together in `Error::Invalid`.
- Cross-section collisions are a validation problem, not a merge one: `CourseFile::problems` reports an id that is
both an objective and a target.
- `Merge::resolve` derives `objective.lectures` from `teaches`, `objective.order` from lecture order × `teaches`
position, and a target's `lectures` from its objective's.
**What 2.1 needs:**
- `Section` gains `Terms`, `Equations`, `Values`, `Misconceptions`, which keeps origins and messages working.
`Fragment`, `Merge`, and `CourseFile` gain the four maps.
- A new `Role::Glossary` for `glossary.yaml` (loaded after `references.yaml`). It allows only the four registries.
`Role::Objective` also allows them, and `Root` already allows everything.
- `Layout::glossary_file()`.
- Lecture files and `references.yaml` may **not** define registry entries. The spec says "any fragment may define
glossary entries", but the target schemas add the registries only to `course`, `objective`, and `glossary`, and the
prompt names exactly those three. I am following the schemas.
- E2101 ("unique across all registries"): a duplicate in the same section is already caught by `absorb` and needs no
change. A collision across sections becomes one check over the six registries (lo, t, term, eq, val, mis) in
`problems()`, generalising the existing objective/target check. On 2.0 data it can only fire where 2.0 already
fires.
## 3. Where validation, lint, and rendering live, and how codes work
**Validation (errors).** `CourseFile::validate` (`src/model/course.rs`: `problems`, `target_issues`,
`reading_issues`, `prerequisite_cycles`), plus `BankFile::validate` → `validate_item` (`src/model/bank.rs`, which
receives the course). `Catalog::validate` (`src/model/catalog.rs`) runs both, and `coursebank validate`
(`src/commands/project.rs`) prints them. Messages are **plain strings with no codes**. `attribute()` prefixes each
with the file that defined the first backticked id, so new messages must name their id first, in backticks.
**Lint (warnings).** `src/authoring/lint.rs`: `enum Rule`, `Rule::ALL`, `code()`, `severity()`
(`catalog::Severity`: Low, Medium, High), `family()`, `description()`. Codes are **kebab-case and family-prefixed**:
`cue-key-longest`, `complete-distractor-design`, `evidence-stale`. `lint_catalog` runs the per-item rules and three
bank-wide ones. `coursebank lint` filters with `--only`, `--ignore`, and `--min-severity`, prints by subject, and
exits non-zero on any finding unless `--no-fail` is passed.
**Code mapping (to go in `CODES.md`).** E-codes become validation messages with stable wording, matching the
repository's uncoded errors. W and I codes become lint rules in a new `glossary` family:
| prototype | coursebank | severity (proposed) |
|---|---|---|
| E2101–E2108 | validation messages (no code); `CODES.md` gives each one's message stem and test | error |
| W2101 | `glossary-unused` | low |
| W2102 | `glossary-no-textbook-name` | medium |
| W2103 | `glossary-no-exam-wording` | low |
| W2104 | `glossary-circular-definition` | low |
| W2105 | `glossary-coined-in-item` | medium |
| W2106 | `glossary-value-unsourced` | low |
| W2107 | `glossary-value-conflict` | medium |
| W2108 | `glossary-forbidden-target-wording` | medium |
| W2109 | `glossary-unlisted-term` | low |
| W2110 | `glossary-recall-value-printed` | medium |
| W2111 | `glossary-equation-no-implications` | low |
| I2112 | `glossary-defined-elsewhere` | note (see Q3) |
| I2113 | `glossary-proposed` | note (see Q3) |
**Rendering.** The lecture pages are `src/export/lecture.rs`: `objectives_markdown` and `readings_markdown`, in
Quarto or Plain Markdown, run by `coursebank lecture objectives|readings <id>` (`src/commands/lectures.rs`). There is
no combined lecture page and no glossary page today. Student reports have two renderers, both fed by
`analysis/students.rs::missed_items` (`MissedItem.misconception` and `.feedback`, taken from the chosen option):
Markdown in `src/export/report.rs::student_markdown`, and Typst through `src/export/typst/payload.rs` and
`templates/student-report.typ`. Student-visible exports are `export/qti.rs` (Canvas), `export/site.rs` (encrypted
solutions), `export/practice.rs`, and the Typst exam and key.
## 4. Teaching order, and how braces and math are handled today
- **Teaching order** is computed in `Merge::resolve`: lectures in **id order** (a `BTreeMap`, not by date), then
each lecture's `teaches` position, which sets `objective.order`. `objectives_in_order` sorts by unit rank, then
`order`, then id. `targets(objective)` sorts by effective ceiling, then id: the 2.0 rule the spec keeps. The
prototype sorts lectures by `(date, id)` instead. The spec says "the existing one", so I will use coursebank's id
order (the example has one lecture, so the two agree on it).
- **A term's first lecture** will walk lectures in that order, the objectives in `teaches` order, and the targets in
`targets()` order (ceiling, then id). The prototype walks targets in file order. For the example both give the
preview's new-terms order.
- **Braces.** `CourseFile::expand_objective_refs` replaces `{id}` only when `id` is a declared objective or target,
so `$U_\text{final}$` passes through. `readings_markdown` renders such a reference as `T n`, the target's position
on the page, or as the target's text when it is not on the page. Only reading `summary`, `focus`, and `skip` are
expanded today. 2.1 generalises this into one `expand_refs` that resolves the six prefixes and takes a renderer
callback, keeping 2.0 behaviour for unprefixed ids. E2106 scans only the prefixed forms the prototype's `BRACE`
pattern matches, so `10^{59}` and `\text{final}` are never references.
- **Math.** `util/markup.rs` has no regex engine. `to_typst` hands `$...$` to mitex; `to_html` and `to_markdown`
leave `$` alone, and Quarto renders it. The lecture pages pass prose through to Quarto unconverted, and the new
page will too: `$d$` symbols stay inline math, and `tex` is emitted as `$$...$$`.
## 5. Tests and fixtures
Unit tests sit inline in each module (`#[cfg(test)] mod tests`) and build courses in `std::env::temp_dir()` or
directly from structs. `tests/docs.rs` checks Markdown fences and guide coverage. There is **no fixture representing
a real course**; the README's biosc1000 and biosc1540 pictures are prose. Plan: `docs/design/2.1/examples/` becomes
the fixture, read by new integration tests through `CARGO_MANIFEST_DIR`, so the examples and the tests cannot drift.
## 6. Where the specification conflicts with the code, and what I propose
1. **The schemas are hand-written, not generated** (section 1). Resolution as described there.
2. **No numeric codes** (section 3). Resolution: the mapping table above, kept in `CODES.md`.
3. **There is no regex engine** (`Cargo.toml` is deliberately small; `markup.rs` and `lint.rs` scan by hand). E2107,
W2107, and W2108 need real regular expressions → **Q1**. W2104 and W2105 (whole-word, case-insensitive) reuse
`lint.rs::contains_word` and need no regex.
4. **E2106 on 2.0 fields vs. "no new errors on a 2.0 course".** A 2.0 reading whose `focus` contains `{t-typo}`
validates today and renders the braces literally. Under E2106 as written it becomes an error → **Q2**.
5. **Info-level findings.** `Severity` has no note level, and `coursebank lint` exits non-zero on any finding. If
I2113 is an ordinary finding, every course with a proposed entry fails lint, which breaks pipelines that call
`coursebank lint` → **Q3**.
6. **The example bank does not load in coursebank 2.0.** `Item.status` is required in Rust (the JSON Schema does not
require it, which is an older mismatch), and the example item has none. With `status: approved`, validation
requires `cognitive_process`, `sources`, and `design`. Proposal: add `status: draft` to the example item, the
smallest change that keeps it a valid bank, and record it in the report. (I checked: the two 2.0 fragments
validate unchanged under today's binary.)
7. **`docs/design/2.1/` breaks `tests/docs.rs`.** Two tests fail now that the folder is in place: every `.md` under
`docs/` must be included in the guide, and the design docs have fences without a language. Proposal: have both
tests skip `docs/design/`, since design records are not guide pages.
8. **Existing lint rules and `misconception_id`.** `complete-distractor-design` fires when an option has neither
`misconception` nor `error_type`, and `complete-student-feedback` uses only the option's own text. Proposal: an
option with a `misconception_id` counts as designed and as having feedback (the registry's correction).
9. **Report wording.** Today a report prints "The option you chose is the one students pick when
{misconception}", because the option text is written as a completion. A registry `belief` is a full sentence in
the student's voice. Proposal: the option's own `misconception` keeps today's sentence; registry-only options
print "This answer reflects a common mistake: *belief*" followed by the correction. Misses sharing a
`misconception_id` are grouped under one heading, with their question numbers listed.
10. **"Reports exactly W2101 and I2113".** On the example, the existing per-item rules also fire: the bank item has
no explanations or feedback on some distractors. I will read the Phase 3 criterion as covering the 2.1 rules
(`coursebank lint --only <glossary codes>`) and show the full output in the report.
11. **The preview's reading line** reads `(T t-align-fill-matrix)` because the prototype prints the raw id.
coursebank renders `T 3` (the target's position on the page, as today). The snapshot test will expect `T 3`.
12. **`policy.blueprints` and `policy.exam_conditions`** are stored and validated but read by no command, because
no phase asks for it. `assemble --blueprint <name>` reading `policy.blueprints` would be a small follow-up.
13. **Registry id prefixes** (`propertyNames`) are enforced by the validator as an extra error, since no runtime
JSON Schema check exists. It sits next to E2101 in `CODES.md` as an addition. It cannot fire on 2.0 data, which
has no registry entries.
## 7. Questions (behaviour changes; I will wait for answers)
**Q1. Regular expressions.** Add the `regex` crate (pure Rust, no unsafe in our code, with features trimmed to
`std` plus the Unicode classes `\b`, `\s`, and case-insensitivity need)? The patterns in the examples (`5\.7\s*kJ`,
`\bcomput`, `10\^\{?100\}?`) are valid in both Python's `re` and Rust's `regex`. Rust's `regex` has no
look-around or backreferences, which `CODES.md` would document. The alternative is a small hand-written matcher in
`util/`, which keeps the dependency list but means maintaining a regex engine. **I recommend adding `regex`.**
**Q2. E2106 on existing 2.0 fields.** (a) E2106 applies to every student-facing field, so a typo'd `{t-...}` in an
old reading becomes an error. This is stricter, and arguably a bug fix, but it breaks rule 1 for such a course.
(b) E2106 applies to the four new prefixes everywhere, and to `{lo-}`/`{t-}` only in the new 2.1 fields; an
unresolved `{lo-}`/`{t-}` in a 2.0 reading field draws a lint warning instead and still renders as today.
**I recommend (b)**, since it keeps rule 1 exact.
**Q3. Note-level findings.** Add `Severity::Note` below `Low` for `glossary-defined-elsewhere` and
`glossary-proposed`, printed as `[note]`, shown by default, selectable with `--min-severity note`, and **not**
making `lint` exit non-zero? Or treat them as `Low` findings that fail lint like any other? **I recommend `Note`**,
so a course with proposed entries can still pass a pipeline's lint gate.
## 8. Phases and the files each touches
All work on a new branch `schema-2.1` (from `prelim-use`), one commit per phase, nothing pushed.
**Phase 1: types and schemas.**
- `src/model/course/glossary.rs` (new, so `course.rs` does not grow past 3,000 lines): `Term`, `TextbookName`,
`TermKind`, `Equation`, `Symbol`, `Implication`, `RelationForm`, `Value`, `Misconception`, `Evidence`,
`EvidenceKind`, `EntryStatus`, `SourceRef`, `Authoring`, `Material`, `MaterialKind`.
- `src/model/course.rs`: `SCHEMA_VERSION = "2.1"`; the four maps on `CourseFile`. New fields on `Objective`
(`description`), `Target` (`explanation`, the four id lists, `authoring`), `Lecture` (`description`,
`materials`), and `Reading` (`minutes`, through its hand-written `Serialize`/`Deserialize`). `Policy` gains
`exam_conditions`, `blueprints`, `levels`, and `target_text_forbid`. Every new field is optional, with
`skip_serializing_if`, so a 2.0 file round-trips unchanged.
- `src/model/course/fragment.rs`, `src/model/layout.rs`: sections, `Role::Glossary`, `glossary.yaml`.
- `src/model/item.rs`: `Choice.misconception_id`.
- `src/authoring/jsonschema.rs`: `Kind::Glossary`, the new builders, the changed descriptions.
- Struct literals across the crate (`CourseFile::skeleton`, test fixtures in `export/lecture.rs` and elsewhere)
gain the new fields, or `..Default::default()` where the type allows it.
- `tests/docs.rs`: skip `docs/design/` (conflict 7). `docs/design/2.1/examples/v21/banks/alignment.yaml`:
`status: draft` (conflict 6).
- `tests/schema_2_1.rs` (new): schemas equal the targets apart from the allow-list; `examples/v20` (in a temporary
course with a minimal `course.yaml` and the references it cites) and `examples/v21` load and pass the 2.0
validation; `examples/migration/glossary.expected.yaml` loads as a glossary fragment.
- Done when both example sets load and the existing suite passes (`pixi run check`).
**Phase 2: validation.**
- `src/model/course/glossary.rs`: the `glossary_issues` checks, E2102–E2107 and the id prefixes. `course.rs`:
`problems()` calls them, and the cross-registry form of E2101.
- The brace scanner, plus the student-facing prose walk (the prototype's `prose()`), in `course.rs` beside
`expand_refs`.
- `src/model/bank.rs::validate_item`: E2108.
- Tests: one per error, each built from the prototype's self-test mutation of the example, plus a check that the
example validates clean.
**Phase 3: lint.**
- `src/authoring/lint.rs`: the 13 rules and a course-level `lint_course` called from `lint_catalog`.
- If Q3 is yes: `src/model/catalog.rs` (`Severity::Note`), `src/cli.rs` (`SeverityArg`), and `lint` in
`src/commands/project.rs`, which does not fail on notes. Every `match` on `Severity` (in `seal.rs`,
`commands/handlers.rs`, and `commands/analysis.rs`) is updated.
- Changes to existing rules from conflict 8.
- Tests: one per rule, each from the self-test mutation, plus the example reporting exactly
`glossary-unused` on `val-per-decade` and `glossary-proposed` (two entries) among the glossary rules.
**Phase 4: derived data and rendering.**
- `course.rs`: `first_lecture(entry)`, and `lecture_glossary(lecture)` returning the lecture's new terms, earlier
terms, equations, values, and misconceptions, in page order.
- `export/lecture.rs`: `page_markdown` (sections in the spec's order, ceilings labelled from `policy.levels` and
otherwise from the generic level names) and `glossary_markdown`. Brace rendering follows the spec's table. A
`{term-}` links to the term's anchor on the page when the term is new there, and otherwise to the course glossary
page (`glossary.html#term-...`, overridable with `--glossary-href`). `objectives_markdown` and `readings_markdown`
keep their output, except that the new prefixes now expand in reading prose.
- `src/cli.rs` and `commands/lectures.rs`: `coursebank lecture page <id>` and `coursebank lecture glossary`.
- Reports: `analysis/students.rs` (`MissedItem` gains the target explanations, `misconception_id`, and the
correction), `export/report.rs` (the new text, and grouping by misconception),
`export/typst/payload.rs` and `templates/student-report.typ` (the same, for the PDF).
- Tests: a snapshot of the example lecture page compared with `lecture-page-preview.md` for content and order
(conflict 11 aside). The rule-5 test renders the lecture page, the glossary page, a Markdown and a Typst student
report, and the Canvas, site, and practice exports of the example bank, and asserts that no `authoring`,
`notes_private`, `evidence`, `conflicts`, or `status` text appears in any of them.
- Deliverable: the example lecture page rendered to HTML with Quarto (installed here), sent to you as the HTML
file and as a private artifact link.
**Phase 5: migration.**
- `src/migrate.rs`: `glossary(root, terms, values, write)`, which parses the 1.0 list form leniently and reports
unknown fields as dropped. It adds the id prefixes, maps `course`→`term` and `meaning`→`definition`, and drops
`first` and `lectures`, comparing them with the derived first lecture or lecture set. It reports entries no target
lists. It writes `glossary.yaml` and skips ids already defined anywhere in the course, so a second run is a no-op
that reports "already present" and writes nothing.
- When `glossary.yaml` already exists with other entries, new ones are inserted as text under the right key, in
the module's line-oriented style, so hand edits and comments survive.
- `src/cli.rs` (`MigrateCommand::Glossary { terms, values, dry_run }`) and `commands/project.rs`.
- Tests: the expected output (as values, so field order does not matter), the report, and the second-run no-op.
**Phase 6: documentation.**
- `docs/guide/glossary.md` (new, wired into `src/guide.rs`): the registries and new fields with the example, the
brace-reference table, the error and lint codes, and the migration.
- `docs/guide/setup.md` and `README.md` (the files table, "What validate checks, and what lint checks", "Migrating
an older course" with schema version 2.1, the lecture outputs, the command reference). Module docs in `lib.rs`,
`model.rs`, `fragment.rs`, and `authoring.rs` (its rule count is already stale: it says 26, and there are 30).
- `CHANGELOG.md` (Unreleased, Added). `docs/design/2.1/CODES.md` and `REPORT.md`.
## 9. Notes for the report (not blocking)
- `say: 'about 10^{59}'` renders literally as `10^{59}`, because it is not inside `$...$`. The example probably
wants `about $10^{59}$`. That is a content edit to the spec's example, which I will not make unless you ask.
- `pyv` cannot run `prototype/check21.py` on this machine: the vendored PyYAML it points at
(`pipelines/mcq-pipeline/tools/vendor`) is not there, because `pipelines/` holds only zip files. The Rust tests
reproduce every self-test case, so the port does not depend on it.