feat(schema): 2.1 types, fragments, and JSON Schemas
Add the term, equation, value, and misconception registries (src/model/course/glossary.rs) and the new optional fields on objectives, targets, lectures, readings, bank options, and policy. Load registry entries from objective files, glossary.yaml, and course.yaml, with one definition site per id as for every other registry. Port every edit in docs/design/2.1/build_schemas.py into the hand-written schema builders, add glossary.schema.json, and test the generated schemas against the targets; the only differences are two 2.0 spellings the code already had. The 2.0 example fragments load and validate unchanged. The 2.1 example bank gains `status: draft`, which every coursebank item requires. tests/docs.rs skips docs/design/, which holds design records rather than guide pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
47 files changed
+10568
-55
No files matched your search
@@ -0,0 +1,279 @@
|
||||
# coursebank 2.1: implementation plan
|
||||
|
||||
Phase 0 output. Written after reading `SCHEMA-CHANGES.md`, the prototype, the examples, and the code on branch
|
||||
`prelim-use` (clean, 412 unit + 20 integration + 4 doc tests passing). Nothing outside `docs/design/2.1/` has been
|
||||
changed. Section 7 has three questions whose answers change behaviour; everything else is a proposal I will follow
|
||||
unless you say otherwise.
|
||||
|
||||
## 1. How the schemas are produced
|
||||
|
||||
They are not generated from the Rust types. `src/authoring/jsonschema.rs` builds each schema by hand with
|
||||
`serde_json::json!`, one function per object (`course_schema`, `objective_schema`, `target_schema`, `lecture_schema`,
|
||||
`reading_mapping_schema`, `option_schema`, `policy_schema`, ...). The module doc says why: hand-written descriptions
|
||||
and enum lists are better tooltips than a derivation. There is no schemars, and the Rust doc comments are *not* the
|
||||
`description` strings.
|
||||
|
||||
- Regenerate: `coursebank schema` (writes `<course>/schema/*.schema.json` through `jsonschema::write_all`), or
|
||||
`pixi run cb schema -C <course>`.
|
||||
- `Kind` enumerates the eight files; `glossary.schema.json` needs a ninth, `Kind::Glossary`.
|
||||
- Baseline check: I generated today's schemas and compared them with `schemas-original/` as JSON values. Six match
|
||||
exactly. Two differ by one word each, already in the code before 2.1: `behaviour` (calibration, per-option
|
||||
history) and `Colour` (course, `grade_scale[].group`). `schemas-original/` has the American spellings, which no
|
||||
commit in this repository ever had. I will keep the code's spelling and list both as known differences.
|
||||
|
||||
**Effect on rule 2.** "Change the types and their doc comments, and regenerate" becomes: change the types (with doc
|
||||
comments in the repository's style), and port every edit in `build_schemas.py` into `jsonschema.rs` with the
|
||||
description strings copied word for word. Hand-written JSON can state `propertyNames` directly, so the id patterns go
|
||||
into the schema. The validator enforces them as well, because nothing checks the YAML against the JSON Schema at load
|
||||
time (serde only). An integration test compares each generated schema with `docs/design/2.1/schemas/` as a
|
||||
`serde_json::Value` (order-insensitive), with an explicit allow-list of known differences. I expect the list to hold
|
||||
only the two spellings above.
|
||||
|
||||
## 2. How fragments are discovered and merged
|
||||
|
||||
`src/model/course/fragment.rs`, entry point `CourseFile::load_dir` → `fragment::assemble(root)`:
|
||||
|
||||
- `files(layout)` lists, in order: `course.yaml` (always, role `Root`), `references.yaml` if present
|
||||
(`References`), sorted `lectures/*.yaml` (`Lecture`), sorted `objectives/*.yaml` (`Objective`). Banks are not
|
||||
fragments; `Catalog` loads them separately.
|
||||
- Each file is parsed into `Fragment` (`deny_unknown_fields`; every section optional). `Role::allows(section)`
|
||||
confines a role to its sections: an objective file may define objectives, targets, and stimuli; `course.yaml` may
|
||||
define anything.
|
||||
- `Merge::take` → `absorb` keeps `origins: BTreeMap<(Section, id), path>`. An id that a second file defines **in the
|
||||
same section** is reported by `duplicate()` (`"<noun> `id` is defined in two places: A and B. An id has one
|
||||
definition site..."`), and every merge problem comes back together in `Error::Invalid`.
|
||||
- Cross-section collisions are a validation problem, not a merge one: `CourseFile::problems` reports an id that is
|
||||
both an objective and a target.
|
||||
- `Merge::resolve` derives `objective.lectures` from `teaches`, `objective.order` from lecture order × `teaches`
|
||||
position, and a target's `lectures` from its objective's.
|
||||
|
||||
**What 2.1 needs:**
|
||||
|
||||
- `Section` gains `Terms`, `Equations`, `Values`, `Misconceptions`, which keeps origins and messages working.
|
||||
`Fragment`, `Merge`, and `CourseFile` gain the four maps.
|
||||
- A new `Role::Glossary` for `glossary.yaml` (loaded after `references.yaml`). It allows only the four registries.
|
||||
`Role::Objective` also allows them, and `Root` already allows everything.
|
||||
- `Layout::glossary_file()`.
|
||||
- Lecture files and `references.yaml` may **not** define registry entries. The spec says "any fragment may define
|
||||
glossary entries", but the target schemas add the registries only to `course`, `objective`, and `glossary`, and the
|
||||
prompt names exactly those three. I am following the schemas.
|
||||
- E2101 ("unique across all registries"): a duplicate in the same section is already caught by `absorb` and needs no
|
||||
change. A collision across sections becomes one check over the six registries (lo, t, term, eq, val, mis) in
|
||||
`problems()`, generalising the existing objective/target check. On 2.0 data it can only fire where 2.0 already
|
||||
fires.
|
||||
|
||||
## 3. Where validation, lint, and rendering live, and how codes work
|
||||
|
||||
**Validation (errors).** `CourseFile::validate` (`src/model/course.rs`: `problems`, `target_issues`,
|
||||
`reading_issues`, `prerequisite_cycles`), plus `BankFile::validate` → `validate_item` (`src/model/bank.rs`, which
|
||||
receives the course). `Catalog::validate` (`src/model/catalog.rs`) runs both, and `coursebank validate`
|
||||
(`src/commands/project.rs`) prints them. Messages are **plain strings with no codes**. `attribute()` prefixes each
|
||||
with the file that defined the first backticked id, so new messages must name their id first, in backticks.
|
||||
|
||||
**Lint (warnings).** `src/authoring/lint.rs`: `enum Rule`, `Rule::ALL`, `code()`, `severity()`
|
||||
(`catalog::Severity`: Low, Medium, High), `family()`, `description()`. Codes are **kebab-case and family-prefixed**:
|
||||
`cue-key-longest`, `complete-distractor-design`, `evidence-stale`. `lint_catalog` runs the per-item rules and three
|
||||
bank-wide ones. `coursebank lint` filters with `--only`, `--ignore`, and `--min-severity`, prints by subject, and
|
||||
exits non-zero on any finding unless `--no-fail` is passed.
|
||||
|
||||
**Code mapping (to go in `CODES.md`).** E-codes become validation messages with stable wording, matching the
|
||||
repository's uncoded errors. W and I codes become lint rules in a new `glossary` family:
|
||||
|
||||
| prototype | coursebank | severity (proposed) |
|
||||
|---|---|---|
|
||||
| E2101–E2108 | validation messages (no code); `CODES.md` gives each one's message stem and test | error |
|
||||
| W2101 | `glossary-unused` | low |
|
||||
| W2102 | `glossary-no-textbook-name` | medium |
|
||||
| W2103 | `glossary-no-exam-wording` | low |
|
||||
| W2104 | `glossary-circular-definition` | low |
|
||||
| W2105 | `glossary-coined-in-item` | medium |
|
||||
| W2106 | `glossary-value-unsourced` | low |
|
||||
| W2107 | `glossary-value-conflict` | medium |
|
||||
| W2108 | `glossary-forbidden-target-wording` | medium |
|
||||
| W2109 | `glossary-unlisted-term` | low |
|
||||
| W2110 | `glossary-recall-value-printed` | medium |
|
||||
| W2111 | `glossary-equation-no-implications` | low |
|
||||
| I2112 | `glossary-defined-elsewhere` | note (see Q3) |
|
||||
| I2113 | `glossary-proposed` | note (see Q3) |
|
||||
|
||||
**Rendering.** The lecture pages are `src/export/lecture.rs`: `objectives_markdown` and `readings_markdown`, in
|
||||
Quarto or Plain Markdown, run by `coursebank lecture objectives|readings <id>` (`src/commands/lectures.rs`). There is
|
||||
no combined lecture page and no glossary page today. Student reports have two renderers, both fed by
|
||||
`analysis/students.rs::missed_items` (`MissedItem.misconception` and `.feedback`, taken from the chosen option):
|
||||
Markdown in `src/export/report.rs::student_markdown`, and Typst through `src/export/typst/payload.rs` and
|
||||
`templates/student-report.typ`. Student-visible exports are `export/qti.rs` (Canvas), `export/site.rs` (encrypted
|
||||
solutions), `export/practice.rs`, and the Typst exam and key.
|
||||
|
||||
## 4. Teaching order, and how braces and math are handled today
|
||||
|
||||
- **Teaching order** is computed in `Merge::resolve`: lectures in **id order** (a `BTreeMap`, not by date), then
|
||||
each lecture's `teaches` position, which sets `objective.order`. `objectives_in_order` sorts by unit rank, then
|
||||
`order`, then id. `targets(objective)` sorts by effective ceiling, then id: the 2.0 rule the spec keeps. The
|
||||
prototype sorts lectures by `(date, id)` instead. The spec says "the existing one", so I will use coursebank's id
|
||||
order (the example has one lecture, so the two agree on it).
|
||||
- **A term's first lecture** will walk lectures in that order, the objectives in `teaches` order, and the targets in
|
||||
`targets()` order (ceiling, then id). The prototype walks targets in file order. For the example both give the
|
||||
preview's new-terms order.
|
||||
- **Braces.** `CourseFile::expand_objective_refs` replaces `{id}` only when `id` is a declared objective or target,
|
||||
so `$U_\text{final}$` passes through. `readings_markdown` renders such a reference as `T n`, the target's position
|
||||
on the page, or as the target's text when it is not on the page. Only reading `summary`, `focus`, and `skip` are
|
||||
expanded today. 2.1 generalises this into one `expand_refs` that resolves the six prefixes and takes a renderer
|
||||
callback, keeping 2.0 behaviour for unprefixed ids. E2106 scans only the prefixed forms the prototype's `BRACE`
|
||||
pattern matches, so `10^{59}` and `\text{final}` are never references.
|
||||
- **Math.** `util/markup.rs` has no regex engine. `to_typst` hands `$...$` to mitex; `to_html` and `to_markdown`
|
||||
leave `$` alone, and Quarto renders it. The lecture pages pass prose through to Quarto unconverted, and the new
|
||||
page will too: `$d$` symbols stay inline math, and `tex` is emitted as `$$...$$`.
|
||||
|
||||
## 5. Tests and fixtures
|
||||
|
||||
Unit tests sit inline in each module (`#[cfg(test)] mod tests`) and build courses in `std::env::temp_dir()` or
|
||||
directly from structs. `tests/docs.rs` checks Markdown fences and guide coverage. There is **no fixture representing
|
||||
a real course**; the README's biosc1000 and biosc1540 pictures are prose. Plan: `docs/design/2.1/examples/` becomes
|
||||
the fixture, read by new integration tests through `CARGO_MANIFEST_DIR`, so the examples and the tests cannot drift.
|
||||
|
||||
## 6. Where the specification conflicts with the code, and what I propose
|
||||
|
||||
1. **The schemas are hand-written, not generated** (section 1). Resolution as described there.
|
||||
2. **No numeric codes** (section 3). Resolution: the mapping table above, kept in `CODES.md`.
|
||||
3. **There is no regex engine** (`Cargo.toml` is deliberately small; `markup.rs` and `lint.rs` scan by hand). E2107,
|
||||
W2107, and W2108 need real regular expressions → **Q1**. W2104 and W2105 (whole-word, case-insensitive) reuse
|
||||
`lint.rs::contains_word` and need no regex.
|
||||
4. **E2106 on 2.0 fields vs. "no new errors on a 2.0 course".** A 2.0 reading whose `focus` contains `{t-typo}`
|
||||
validates today and renders the braces literally. Under E2106 as written it becomes an error → **Q2**.
|
||||
5. **Info-level findings.** `Severity` has no note level, and `coursebank lint` exits non-zero on any finding. If
|
||||
I2113 is an ordinary finding, every course with a proposed entry fails lint, which breaks pipelines that call
|
||||
`coursebank lint` → **Q3**.
|
||||
6. **The example bank does not load in coursebank 2.0.** `Item.status` is required in Rust (the JSON Schema does not
|
||||
require it, which is an older mismatch), and the example item has none. With `status: approved`, validation
|
||||
requires `cognitive_process`, `sources`, and `design`. Proposal: add `status: draft` to the example item, the
|
||||
smallest change that keeps it a valid bank, and record it in the report. (I checked: the two 2.0 fragments
|
||||
validate unchanged under today's binary.)
|
||||
7. **`docs/design/2.1/` breaks `tests/docs.rs`.** Two tests fail now that the folder is in place: every `.md` under
|
||||
`docs/` must be included in the guide, and the design docs have fences without a language. Proposal: have both
|
||||
tests skip `docs/design/`, since design records are not guide pages.
|
||||
8. **Existing lint rules and `misconception_id`.** `complete-distractor-design` fires when an option has neither
|
||||
`misconception` nor `error_type`, and `complete-student-feedback` uses only the option's own text. Proposal: an
|
||||
option with a `misconception_id` counts as designed and as having feedback (the registry's correction).
|
||||
9. **Report wording.** Today a report prints "The option you chose is the one students pick when
|
||||
{misconception}", because the option text is written as a completion. A registry `belief` is a full sentence in
|
||||
the student's voice. Proposal: the option's own `misconception` keeps today's sentence; registry-only options
|
||||
print "This answer reflects a common mistake: *belief*" followed by the correction. Misses sharing a
|
||||
`misconception_id` are grouped under one heading, with their question numbers listed.
|
||||
10. **"Reports exactly W2101 and I2113".** On the example, the existing per-item rules also fire: the bank item has
|
||||
no explanations or feedback on some distractors. I will read the Phase 3 criterion as covering the 2.1 rules
|
||||
(`coursebank lint --only <glossary codes>`) and show the full output in the report.
|
||||
11. **The preview's reading line** reads `(T t-align-fill-matrix)` because the prototype prints the raw id.
|
||||
coursebank renders `T 3` (the target's position on the page, as today). The snapshot test will expect `T 3`.
|
||||
12. **`policy.blueprints` and `policy.exam_conditions`** are stored and validated but read by no command, because
|
||||
no phase asks for it. `assemble --blueprint <name>` reading `policy.blueprints` would be a small follow-up.
|
||||
13. **Registry id prefixes** (`propertyNames`) are enforced by the validator as an extra error, since no runtime
|
||||
JSON Schema check exists. It sits next to E2101 in `CODES.md` as an addition. It cannot fire on 2.0 data, which
|
||||
has no registry entries.
|
||||
|
||||
## 7. Questions (behaviour changes; I will wait for answers)
|
||||
|
||||
**Q1. Regular expressions.** Add the `regex` crate (pure Rust, no unsafe in our code, with features trimmed to
|
||||
`std` plus the Unicode classes `\b`, `\s`, and case-insensitivity need)? The patterns in the examples (`5\.7\s*kJ`,
|
||||
`\bcomput`, `10\^\{?100\}?`) are valid in both Python's `re` and Rust's `regex`. Rust's `regex` has no
|
||||
look-around or backreferences, which `CODES.md` would document. The alternative is a small hand-written matcher in
|
||||
`util/`, which keeps the dependency list but means maintaining a regex engine. **I recommend adding `regex`.**
|
||||
|
||||
**Q2. E2106 on existing 2.0 fields.** (a) E2106 applies to every student-facing field, so a typo'd `{t-...}` in an
|
||||
old reading becomes an error. This is stricter, and arguably a bug fix, but it breaks rule 1 for such a course.
|
||||
(b) E2106 applies to the four new prefixes everywhere, and to `{lo-}`/`{t-}` only in the new 2.1 fields; an
|
||||
unresolved `{lo-}`/`{t-}` in a 2.0 reading field draws a lint warning instead and still renders as today.
|
||||
**I recommend (b)**, since it keeps rule 1 exact.
|
||||
|
||||
**Q3. Note-level findings.** Add `Severity::Note` below `Low` for `glossary-defined-elsewhere` and
|
||||
`glossary-proposed`, printed as `[note]`, shown by default, selectable with `--min-severity note`, and **not**
|
||||
making `lint` exit non-zero? Or treat them as `Low` findings that fail lint like any other? **I recommend `Note`**,
|
||||
so a course with proposed entries can still pass a pipeline's lint gate.
|
||||
|
||||
## 8. Phases and the files each touches
|
||||
|
||||
All work on a new branch `schema-2.1` (from `prelim-use`), one commit per phase, nothing pushed.
|
||||
|
||||
**Phase 1: types and schemas.**
|
||||
- `src/model/course/glossary.rs` (new, so `course.rs` does not grow past 3,000 lines): `Term`, `TextbookName`,
|
||||
`TermKind`, `Equation`, `Symbol`, `Implication`, `RelationForm`, `Value`, `Misconception`, `Evidence`,
|
||||
`EvidenceKind`, `EntryStatus`, `SourceRef`, `Authoring`, `Material`, `MaterialKind`.
|
||||
- `src/model/course.rs`: `SCHEMA_VERSION = "2.1"`; the four maps on `CourseFile`. New fields on `Objective`
|
||||
(`description`), `Target` (`explanation`, the four id lists, `authoring`), `Lecture` (`description`,
|
||||
`materials`), and `Reading` (`minutes`, through its hand-written `Serialize`/`Deserialize`). `Policy` gains
|
||||
`exam_conditions`, `blueprints`, `levels`, and `target_text_forbid`. Every new field is optional, with
|
||||
`skip_serializing_if`, so a 2.0 file round-trips unchanged.
|
||||
- `src/model/course/fragment.rs`, `src/model/layout.rs`: sections, `Role::Glossary`, `glossary.yaml`.
|
||||
- `src/model/item.rs`: `Choice.misconception_id`.
|
||||
- `src/authoring/jsonschema.rs`: `Kind::Glossary`, the new builders, the changed descriptions.
|
||||
- Struct literals across the crate (`CourseFile::skeleton`, test fixtures in `export/lecture.rs` and elsewhere)
|
||||
gain the new fields, or `..Default::default()` where the type allows it.
|
||||
- `tests/docs.rs`: skip `docs/design/` (conflict 7). `docs/design/2.1/examples/v21/banks/alignment.yaml`:
|
||||
`status: draft` (conflict 6).
|
||||
- `tests/schema_2_1.rs` (new): schemas equal the targets apart from the allow-list; `examples/v20` (in a temporary
|
||||
course with a minimal `course.yaml` and the references it cites) and `examples/v21` load and pass the 2.0
|
||||
validation; `examples/migration/glossary.expected.yaml` loads as a glossary fragment.
|
||||
- Done when both example sets load and the existing suite passes (`pixi run check`).
|
||||
|
||||
**Phase 2: validation.**
|
||||
- `src/model/course/glossary.rs`: the `glossary_issues` checks, E2102–E2107 and the id prefixes. `course.rs`:
|
||||
`problems()` calls them, and the cross-registry form of E2101.
|
||||
- The brace scanner, plus the student-facing prose walk (the prototype's `prose()`), in `course.rs` beside
|
||||
`expand_refs`.
|
||||
- `src/model/bank.rs::validate_item`: E2108.
|
||||
- Tests: one per error, each built from the prototype's self-test mutation of the example, plus a check that the
|
||||
example validates clean.
|
||||
|
||||
**Phase 3: lint.**
|
||||
- `src/authoring/lint.rs`: the 13 rules and a course-level `lint_course` called from `lint_catalog`.
|
||||
- If Q3 is yes: `src/model/catalog.rs` (`Severity::Note`), `src/cli.rs` (`SeverityArg`), and `lint` in
|
||||
`src/commands/project.rs`, which does not fail on notes. Every `match` on `Severity` (in `seal.rs`,
|
||||
`commands/handlers.rs`, and `commands/analysis.rs`) is updated.
|
||||
- Changes to existing rules from conflict 8.
|
||||
- Tests: one per rule, each from the self-test mutation, plus the example reporting exactly
|
||||
`glossary-unused` on `val-per-decade` and `glossary-proposed` (two entries) among the glossary rules.
|
||||
|
||||
**Phase 4: derived data and rendering.**
|
||||
- `course.rs`: `first_lecture(entry)`, and `lecture_glossary(lecture)` returning the lecture's new terms, earlier
|
||||
terms, equations, values, and misconceptions, in page order.
|
||||
- `export/lecture.rs`: `page_markdown` (sections in the spec's order, ceilings labelled from `policy.levels` and
|
||||
otherwise from the generic level names) and `glossary_markdown`. Brace rendering follows the spec's table. A
|
||||
`{term-}` links to the term's anchor on the page when the term is new there, and otherwise to the course glossary
|
||||
page (`glossary.html#term-...`, overridable with `--glossary-href`). `objectives_markdown` and `readings_markdown`
|
||||
keep their output, except that the new prefixes now expand in reading prose.
|
||||
- `src/cli.rs` and `commands/lectures.rs`: `coursebank lecture page <id>` and `coursebank lecture glossary`.
|
||||
- Reports: `analysis/students.rs` (`MissedItem` gains the target explanations, `misconception_id`, and the
|
||||
correction), `export/report.rs` (the new text, and grouping by misconception),
|
||||
`export/typst/payload.rs` and `templates/student-report.typ` (the same, for the PDF).
|
||||
- Tests: a snapshot of the example lecture page compared with `lecture-page-preview.md` for content and order
|
||||
(conflict 11 aside). The rule-5 test renders the lecture page, the glossary page, a Markdown and a Typst student
|
||||
report, and the Canvas, site, and practice exports of the example bank, and asserts that no `authoring`,
|
||||
`notes_private`, `evidence`, `conflicts`, or `status` text appears in any of them.
|
||||
- Deliverable: the example lecture page rendered to HTML with Quarto (installed here), sent to you as the HTML
|
||||
file and as a private artifact link.
|
||||
|
||||
**Phase 5: migration.**
|
||||
- `src/migrate.rs`: `glossary(root, terms, values, write)`, which parses the 1.0 list form leniently and reports
|
||||
unknown fields as dropped. It adds the id prefixes, maps `course`→`term` and `meaning`→`definition`, and drops
|
||||
`first` and `lectures`, comparing them with the derived first lecture or lecture set. It reports entries no target
|
||||
lists. It writes `glossary.yaml` and skips ids already defined anywhere in the course, so a second run is a no-op
|
||||
that reports "already present" and writes nothing.
|
||||
- When `glossary.yaml` already exists with other entries, new ones are inserted as text under the right key, in
|
||||
the module's line-oriented style, so hand edits and comments survive.
|
||||
- `src/cli.rs` (`MigrateCommand::Glossary { terms, values, dry_run }`) and `commands/project.rs`.
|
||||
- Tests: the expected output (as values, so field order does not matter), the report, and the second-run no-op.
|
||||
|
||||
**Phase 6: documentation.**
|
||||
- `docs/guide/glossary.md` (new, wired into `src/guide.rs`): the registries and new fields with the example, the
|
||||
brace-reference table, the error and lint codes, and the migration.
|
||||
- `docs/guide/setup.md` and `README.md` (the files table, "What validate checks, and what lint checks", "Migrating
|
||||
an older course" with schema version 2.1, the lecture outputs, the command reference). Module docs in `lib.rs`,
|
||||
`model.rs`, `fragment.rs`, and `authoring.rs` (its rule count is already stale: it says 26, and there are 30).
|
||||
- `CHANGELOG.md` (Unreleased, Added). `docs/design/2.1/CODES.md` and `REPORT.md`.
|
||||
|
||||
## 9. Notes for the report (not blocking)
|
||||
|
||||
- `say: 'about 10^{59}'` renders literally as `10^{59}`, because it is not inside `$...$`. The example probably
|
||||
wants `about $10^{59}$`. That is a content edit to the spec's example, which I will not make unless you ask.
|
||||
- `pyv` cannot run `prototype/check21.py` on this machine: the vendored PyYAML it points at
|
||||
(`pipelines/mcq-pipeline/tools/vendor`) is not there, because `pipelines/` holds only zip files. The Rust tests
|
||||
reproduce every self-test case, so the port does not depend on it.
|
||||
Reference in new issue
Block a user