Files
coursebank/docs/design/2.1/PLAN.md
T
alexmandClaude 51a2fd235f feat(schema): 2.1 types, fragments, and JSON Schemas
Add the term, equation, value, and misconception registries
(src/model/course/glossary.rs) and the new optional fields on objectives,
targets, lectures, readings, bank options, and policy. Load registry
entries from objective files, glossary.yaml, and course.yaml, with one
definition site per id as for every other registry. Port every edit in
docs/design/2.1/build_schemas.py into the hand-written schema builders,
add glossary.schema.json, and test the generated schemas against the
targets; the only differences are two 2.0 spellings the code already had.

The 2.0 example fragments load and validate unchanged. The 2.1 example
bank gains `status: draft`, which every coursebank item requires.
tests/docs.rs skips docs/design/, which holds design records rather than
guide pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:55:18 -05:00

21 KiB
Raw Blame History

coursebank 2.1: implementation plan

Phase 0 output. Written after reading SCHEMA-CHANGES.md, the prototype, the examples, and the code on branch prelim-use (clean, 412 unit + 20 integration + 4 doc tests passing). Nothing outside docs/design/2.1/ has been changed. Section 7 has three questions whose answers change behaviour; everything else is a proposal I will follow unless you say otherwise.

1. How the schemas are produced

They are not generated from the Rust types. src/authoring/jsonschema.rs builds each schema by hand with serde_json::json!, one function per object (course_schema, objective_schema, target_schema, lecture_schema, reading_mapping_schema, option_schema, policy_schema, ...). The module doc says why: hand-written descriptions and enum lists are better tooltips than a derivation. There is no schemars, and the Rust doc comments are not the description strings.

  • Regenerate: coursebank schema (writes <course>/schema/*.schema.json through jsonschema::write_all), or pixi run cb schema -C <course>.
  • Kind enumerates the eight files; glossary.schema.json needs a ninth, Kind::Glossary.
  • Baseline check: I generated today's schemas and compared them with schemas-original/ as JSON values. Six match exactly. Two differ by one word each, already in the code before 2.1: behaviour (calibration, per-option history) and Colour (course, grade_scale[].group). schemas-original/ has the American spellings, which no commit in this repository ever had. I will keep the code's spelling and list both as known differences.

Effect on rule 2. "Change the types and their doc comments, and regenerate" becomes: change the types (with doc comments in the repository's style), and port every edit in build_schemas.py into jsonschema.rs with the description strings copied word for word. Hand-written JSON can state propertyNames directly, so the id patterns go into the schema. The validator enforces them as well, because nothing checks the YAML against the JSON Schema at load time (serde only). An integration test compares each generated schema with docs/design/2.1/schemas/ as a serde_json::Value (order-insensitive), with an explicit allow-list of known differences. I expect the list to hold only the two spellings above.

2. How fragments are discovered and merged

src/model/course/fragment.rs, entry point CourseFile::load_dir → fragment::assemble(root):

  • files(layout) lists, in order: course.yaml (always, role Root), references.yaml if present (References), sorted lectures/*.yaml (Lecture), sorted objectives/*.yaml (Objective). Banks are not fragments; Catalog loads them separately.
  • Each file is parsed into Fragment (deny_unknown_fields; every section optional). Role::allows(section) confines a role to its sections: an objective file may define objectives, targets, and stimuli; course.yaml may define anything.
  • Merge::take → absorb keeps origins: BTreeMap<(Section, id), path>. An id that a second file defines in the same section is reported by duplicate() ("<noun> id is defined in two places: A and B. An id has one definition site..."), and every merge problem comes back together in Error::Invalid.
  • Cross-section collisions are a validation problem, not a merge one: CourseFile::problems reports an id that is both an objective and a target.
  • Merge::resolve derives objective.lectures from teaches, objective.order from lecture order × teaches position, and a target's lectures from its objective's.

What 2.1 needs:

  • Section gains Terms, Equations, Values, Misconceptions, which keeps origins and messages working. Fragment, Merge, and CourseFile gain the four maps.
  • A new Role::Glossary for glossary.yaml (loaded after references.yaml). It allows only the four registries. Role::Objective also allows them, and Root already allows everything.
  • Layout::glossary_file().
  • Lecture files and references.yaml may not define registry entries. The spec says "any fragment may define glossary entries", but the target schemas add the registries only to course, objective, and glossary, and the prompt names exactly those three. I am following the schemas.
  • E2101 ("unique across all registries"): a duplicate in the same section is already caught by absorb and needs no change. A collision across sections becomes one check over the six registries (lo, t, term, eq, val, mis) in problems(), generalising the existing objective/target check. On 2.0 data it can only fire where 2.0 already fires.

3. Where validation, lint, and rendering live, and how codes work

Validation (errors). CourseFile::validate (src/model/course.rs: problems, target_issues, reading_issues, prerequisite_cycles), plus BankFile::validate → validate_item (src/model/bank.rs, which receives the course). Catalog::validate (src/model/catalog.rs) runs both, and coursebank validate (src/commands/project.rs) prints them. Messages are plain strings with no codes. attribute() prefixes each with the file that defined the first backticked id, so new messages must name their id first, in backticks.

Lint (warnings). src/authoring/lint.rs: enum Rule, Rule::ALL, code(), severity() (catalog::Severity: Low, Medium, High), family(), description(). Codes are kebab-case and family-prefixed: cue-key-longest, complete-distractor-design, evidence-stale. lint_catalog runs the per-item rules and three bank-wide ones. coursebank lint filters with --only, --ignore, and --min-severity, prints by subject, and exits non-zero on any finding unless --no-fail is passed.

Code mapping (to go in CODES.md). E-codes become validation messages with stable wording, matching the repository's uncoded errors. W and I codes become lint rules in a new glossary family:

prototype coursebank severity (proposed)
E2101–E2108 validation messages (no code); CODES.md gives each one's message stem and test error
W2101 glossary-unused low
W2102 glossary-no-textbook-name medium
W2103 glossary-no-exam-wording low
W2104 glossary-circular-definition low
W2105 glossary-coined-in-item medium
W2106 glossary-value-unsourced low
W2107 glossary-value-conflict medium
W2108 glossary-forbidden-target-wording medium
W2109 glossary-unlisted-term low
W2110 glossary-recall-value-printed medium
W2111 glossary-equation-no-implications low
I2112 glossary-defined-elsewhere note (see Q3)
I2113 glossary-proposed note (see Q3)

Rendering. The lecture pages are src/export/lecture.rs: objectives_markdown and readings_markdown, in Quarto or Plain Markdown, run by coursebank lecture objectives|readings <id> (src/commands/lectures.rs). There is no combined lecture page and no glossary page today. Student reports have two renderers, both fed by analysis/students.rs::missed_items (MissedItem.misconception and .feedback, taken from the chosen option): Markdown in src/export/report.rs::student_markdown, and Typst through src/export/typst/payload.rs and templates/student-report.typ. Student-visible exports are export/qti.rs (Canvas), export/site.rs (encrypted solutions), export/practice.rs, and the Typst exam and key.

4. Teaching order, and how braces and math are handled today

  • Teaching order is computed in Merge::resolve: lectures in id order (a BTreeMap, not by date), then each lecture's teaches position, which sets objective.order. objectives_in_order sorts by unit rank, then order, then id. targets(objective) sorts by effective ceiling, then id: the 2.0 rule the spec keeps. The prototype sorts lectures by (date, id) instead. The spec says "the existing one", so I will use coursebank's id order (the example has one lecture, so the two agree on it).
  • A term's first lecture will walk lectures in that order, the objectives in teaches order, and the targets in targets() order (ceiling, then id). The prototype walks targets in file order. For the example both give the preview's new-terms order.
  • Braces. CourseFile::expand_objective_refs replaces {id} only when id is a declared objective or target, so $U_\text{final}$ passes through. readings_markdown renders such a reference as T n, the target's position on the page, or as the target's text when it is not on the page. Only reading summary, focus, and skip are expanded today. 2.1 generalises this into one expand_refs that resolves the six prefixes and takes a renderer callback, keeping 2.0 behaviour for unprefixed ids. E2106 scans only the prefixed forms the prototype's BRACE pattern matches, so 10^{59} and \text{final} are never references.
  • Math. util/markup.rs has no regex engine. to_typst hands $...$ to mitex; to_html and to_markdown leave $ alone, and Quarto renders it. The lecture pages pass prose through to Quarto unconverted, and the new page will too: $d$ symbols stay inline math, and tex is emitted as $$...$$.

5. Tests and fixtures

Unit tests sit inline in each module (#[cfg(test)] mod tests) and build courses in std::env::temp_dir() or directly from structs. tests/docs.rs checks Markdown fences and guide coverage. There is no fixture representing a real course; the README's biosc1000 and biosc1540 pictures are prose. Plan: docs/design/2.1/examples/ becomes the fixture, read by new integration tests through CARGO_MANIFEST_DIR, so the examples and the tests cannot drift.

6. Where the specification conflicts with the code, and what I propose

  1. The schemas are hand-written, not generated (section 1). Resolution as described there.
  2. No numeric codes (section 3). Resolution: the mapping table above, kept in CODES.md.
  3. There is no regex engine (Cargo.toml is deliberately small; markup.rs and lint.rs scan by hand). E2107, W2107, and W2108 need real regular expressions → Q1. W2104 and W2105 (whole-word, case-insensitive) reuse lint.rs::contains_word and need no regex.
  4. E2106 on 2.0 fields vs. "no new errors on a 2.0 course". A 2.0 reading whose focus contains {t-typo} validates today and renders the braces literally. Under E2106 as written it becomes an error → Q2.
  5. Info-level findings. Severity has no note level, and coursebank lint exits non-zero on any finding. If I2113 is an ordinary finding, every course with a proposed entry fails lint, which breaks pipelines that call coursebank lint → Q3.
  6. The example bank does not load in coursebank 2.0. Item.status is required in Rust (the JSON Schema does not require it, which is an older mismatch), and the example item has none. With status: approved, validation requires cognitive_process, sources, and design. Proposal: add status: draft to the example item, the smallest change that keeps it a valid bank, and record it in the report. (I checked: the two 2.0 fragments validate unchanged under today's binary.)
  7. docs/design/2.1/ breaks tests/docs.rs. Two tests fail now that the folder is in place: every .md under docs/ must be included in the guide, and the design docs have fences without a language. Proposal: have both tests skip docs/design/, since design records are not guide pages.
  8. Existing lint rules and misconception_id. complete-distractor-design fires when an option has neither misconception nor error_type, and complete-student-feedback uses only the option's own text. Proposal: an option with a misconception_id counts as designed and as having feedback (the registry's correction).
  9. Report wording. Today a report prints "The option you chose is the one students pick when {misconception}", because the option text is written as a completion. A registry belief is a full sentence in the student's voice. Proposal: the option's own misconception keeps today's sentence; registry-only options print "This answer reflects a common mistake: belief" followed by the correction. Misses sharing a misconception_id are grouped under one heading, with their question numbers listed.
  10. "Reports exactly W2101 and I2113". On the example, the existing per-item rules also fire: the bank item has no explanations or feedback on some distractors. I will read the Phase 3 criterion as covering the 2.1 rules (coursebank lint --only <glossary codes>) and show the full output in the report.
  11. The preview's reading line reads (T t-align-fill-matrix) because the prototype prints the raw id. coursebank renders T 3 (the target's position on the page, as today). The snapshot test will expect T 3.
  12. policy.blueprints and policy.exam_conditions are stored and validated but read by no command, because no phase asks for it. assemble --blueprint <name> reading policy.blueprints would be a small follow-up.
  13. Registry id prefixes (propertyNames) are enforced by the validator as an extra error, since no runtime JSON Schema check exists. It sits next to E2101 in CODES.md as an addition. It cannot fire on 2.0 data, which has no registry entries.

7. Questions (behaviour changes; I will wait for answers)

Q1. Regular expressions. Add the regex crate (pure Rust, no unsafe in our code, with features trimmed to std plus the Unicode classes \b, \s, and case-insensitivity need)? The patterns in the examples (5\.7\s*kJ, \bcomput, 10\^\{?100\}?) are valid in both Python's re and Rust's regex. Rust's regex has no look-around or backreferences, which CODES.md would document. The alternative is a small hand-written matcher in util/, which keeps the dependency list but means maintaining a regex engine. I recommend adding regex.

Q2. E2106 on existing 2.0 fields. (a) E2106 applies to every student-facing field, so a typo'd {t-...} in an old reading becomes an error. This is stricter, and arguably a bug fix, but it breaks rule 1 for such a course. (b) E2106 applies to the four new prefixes everywhere, and to {lo-}/{t-} only in the new 2.1 fields; an unresolved {lo-}/{t-} in a 2.0 reading field draws a lint warning instead and still renders as today. I recommend (b), since it keeps rule 1 exact.

Q3. Note-level findings. Add Severity::Note below Low for glossary-defined-elsewhere and glossary-proposed, printed as [note], shown by default, selectable with --min-severity note, and not making lint exit non-zero? Or treat them as Low findings that fail lint like any other? I recommend Note, so a course with proposed entries can still pass a pipeline's lint gate.

8. Phases and the files each touches

All work on a new branch schema-2.1 (from prelim-use), one commit per phase, nothing pushed.

Phase 1: types and schemas.

  • src/model/course/glossary.rs (new, so course.rs does not grow past 3,000 lines): Term, TextbookName, TermKind, Equation, Symbol, Implication, RelationForm, Value, Misconception, Evidence, EvidenceKind, EntryStatus, SourceRef, Authoring, Material, MaterialKind.
  • src/model/course.rs: SCHEMA_VERSION = "2.1"; the four maps on CourseFile. New fields on Objective (description), Target (explanation, the four id lists, authoring), Lecture (description, materials), and Reading (minutes, through its hand-written Serialize/Deserialize). Policy gains exam_conditions, blueprints, levels, and target_text_forbid. Every new field is optional, with skip_serializing_if, so a 2.0 file round-trips unchanged.
  • src/model/course/fragment.rs, src/model/layout.rs: sections, Role::Glossary, glossary.yaml.
  • src/model/item.rs: Choice.misconception_id.
  • src/authoring/jsonschema.rs: Kind::Glossary, the new builders, the changed descriptions.
  • Struct literals across the crate (CourseFile::skeleton, test fixtures in export/lecture.rs and elsewhere) gain the new fields, or ..Default::default() where the type allows it.
  • tests/docs.rs: skip docs/design/ (conflict 7). docs/design/2.1/examples/v21/banks/alignment.yaml: status: draft (conflict 6).
  • tests/schema_2_1.rs (new): schemas equal the targets apart from the allow-list; examples/v20 (in a temporary course with a minimal course.yaml and the references it cites) and examples/v21 load and pass the 2.0 validation; examples/migration/glossary.expected.yaml loads as a glossary fragment.
  • Done when both example sets load and the existing suite passes (pixi run check).

Phase 2: validation.

  • src/model/course/glossary.rs: the glossary_issues checks, E2102–E2107 and the id prefixes. course.rs: problems() calls them, and the cross-registry form of E2101.
  • The brace scanner, plus the student-facing prose walk (the prototype's prose()), in course.rs beside expand_refs.
  • src/model/bank.rs::validate_item: E2108.
  • Tests: one per error, each built from the prototype's self-test mutation of the example, plus a check that the example validates clean.

Phase 3: lint.

  • src/authoring/lint.rs: the 13 rules and a course-level lint_course called from lint_catalog.
  • If Q3 is yes: src/model/catalog.rs (Severity::Note), src/cli.rs (SeverityArg), and lint in src/commands/project.rs, which does not fail on notes. Every match on Severity (in seal.rs, commands/handlers.rs, and commands/analysis.rs) is updated.
  • Changes to existing rules from conflict 8.
  • Tests: one per rule, each from the self-test mutation, plus the example reporting exactly glossary-unused on val-per-decade and glossary-proposed (two entries) among the glossary rules.

Phase 4: derived data and rendering.

  • course.rs: first_lecture(entry), and lecture_glossary(lecture) returning the lecture's new terms, earlier terms, equations, values, and misconceptions, in page order.
  • export/lecture.rs: page_markdown (sections in the spec's order, ceilings labelled from policy.levels and otherwise from the generic level names) and glossary_markdown. Brace rendering follows the spec's table. A {term-} links to the term's anchor on the page when the term is new there, and otherwise to the course glossary page (glossary.html#term-..., overridable with --glossary-href). objectives_markdown and readings_markdown keep their output, except that the new prefixes now expand in reading prose.
  • src/cli.rs and commands/lectures.rs: coursebank lecture page <id> and coursebank lecture glossary.
  • Reports: analysis/students.rs (MissedItem gains the target explanations, misconception_id, and the correction), export/report.rs (the new text, and grouping by misconception), export/typst/payload.rs and templates/student-report.typ (the same, for the PDF).
  • Tests: a snapshot of the example lecture page compared with lecture-page-preview.md for content and order (conflict 11 aside). The rule-5 test renders the lecture page, the glossary page, a Markdown and a Typst student report, and the Canvas, site, and practice exports of the example bank, and asserts that no authoring, notes_private, evidence, conflicts, or status text appears in any of them.
  • Deliverable: the example lecture page rendered to HTML with Quarto (installed here), sent to you as the HTML file and as a private artifact link.

Phase 5: migration.

  • src/migrate.rs: glossary(root, terms, values, write), which parses the 1.0 list form leniently and reports unknown fields as dropped. It adds the id prefixes, maps course→term and meaning→definition, and drops first and lectures, comparing them with the derived first lecture or lecture set. It reports entries no target lists. It writes glossary.yaml and skips ids already defined anywhere in the course, so a second run is a no-op that reports "already present" and writes nothing.
  • When glossary.yaml already exists with other entries, new ones are inserted as text under the right key, in the module's line-oriented style, so hand edits and comments survive.
  • src/cli.rs (MigrateCommand::Glossary { terms, values, dry_run }) and commands/project.rs.
  • Tests: the expected output (as values, so field order does not matter), the report, and the second-run no-op.

Phase 6: documentation.

  • docs/guide/glossary.md (new, wired into src/guide.rs): the registries and new fields with the example, the brace-reference table, the error and lint codes, and the migration.
  • docs/guide/setup.md and README.md (the files table, "What validate checks, and what lint checks", "Migrating an older course" with schema version 2.1, the lecture outputs, the command reference). Module docs in lib.rs, model.rs, fragment.rs, and authoring.rs (its rule count is already stale: it says 26, and there are 30).
  • CHANGELOG.md (Unreleased, Added). docs/design/2.1/CODES.md and REPORT.md.

9. Notes for the report (not blocking)

  • say: 'about 10^{59}' renders literally as 10^{59}, because it is not inside $...$. The example probably wants about $10^{59}$. That is a content edit to the spec's example, which I will not make unless you ask.
  • pyv cannot run prototype/check21.py on this machine: the vendored PyYAML it points at (pipelines/mcq-pipeline/tools/vendor) is not there, because pipelines/ holds only zip files. The Rust tests reproduce every self-test case, so the port does not depend on it.