docs: schema 2.1 guide, codes, changelog, and report
A guide page for the four registries: where entries live, how targets point at them, the other new fields, the brace-reference table, what students never see, the lecture and glossary pages, every new error and lint rule, and the migration from terms.yaml and values.yaml. The setup guide points at it. The README covers glossary.yaml, the nine schemas, the 2.1 checks, the glossary lint family and the note severity, the new lecture commands and report sections, migrate glossary, and schema version 2.1. Module docs and the changelog follow. docs/design/2.1/CODES.md maps the specification's E, W, and I codes to coursebank's messages and lint codes, each with its test, and REPORT.md records what changed, the two remaining schema differences, every choice made where the specification left room, and suggested changes to it. A new test covers the validation messages the specification does not number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
11 files changed
+535
-29
No files matched your search
@@ -12,6 +12,23 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
- Student reports add bonus questions and curves to the score and show how the total adds up, so it matches Gradescope and Canvas. Class means, bands, and grade distributions use the same total.
|
||||
- A zero-point Gradescope question with no option columns is recorded as a curve.
|
||||
- `ingest` warns when a question dropped as removed still gives every student points on the platform.
|
||||
- Schema 2.1, which is additive: a course that validated under 2.0 validates with no new errors. Rerun `coursebank schema` to pick up the new and changed JSON Schemas, including the new `glossary.schema.json`.
|
||||
- Four registries beside objectives and targets: terms (`term-`), equations (`eq-`), values (`val-`), and misconceptions (`mis-`), each defined once in an objective file, a new `glossary.yaml`, or `course.yaml`. Ids are unique across every registry.
|
||||
- Targets list the entries they turn on (`terms`, `equations`, `values`, `misconceptions`), carry a student-facing `explanation`, and keep the item-writing judgments in `authoring`.
|
||||
- Objectives and lectures gain a `description`, lectures gain `materials`, readings gain `minutes`, and bank options gain `misconception_id`.
|
||||
- `policy` gains `levels` (the course's names for levels 1 to 5), `exam_conditions`, `blueprints`, and `target_text_forbid`. No command reads `exam_conditions` or `blueprints` yet.
|
||||
- Brace references such as `{term-gap-penalty}` in student-facing prose render as links, values, quoted beliefs, or target numbers.
|
||||
- `authoring`, `notes_private`, `evidence`, `conflicts`, and `status` never reach a student page, report, or export.
|
||||
- `validate` checks the registries: id prefixes, ids unique across registries, the entries each target lists, equation symbols, values, citation keys, brace references, that every regular expression compiles, and each option's `misconception_id`.
|
||||
- Fourteen lint rules in a `glossary` family, from `glossary-unused` to `glossary-proposed`, and a `note` severity for the two that report conventions rather than problems.
|
||||
- `lecture page ID` writes a whole lecture page: description, objectives and targets with explanations and named ceilings, new and earlier terms, equations, numbers to know, common mistakes, readings with minutes, and materials. `lecture glossary` writes the course glossary page.
|
||||
- Student reports show what meeting each missed target looks like, the correction for a registry misconception, and group misses that share one.
|
||||
- `migrate glossary --terms terms.yaml --values values.yaml` brings 1.0 side files into `glossary.yaml`, reports what it renamed and dropped, and is a no-op when run again.
|
||||
|
||||
### Changed
|
||||
|
||||
- `lint --min-severity` defaults to `note`, and `lint` fails only on findings above a note.
|
||||
- The crate depends on `regex`, built without its optional features.
|
||||
|
||||
### Fixed
|
||||
|
||||
|
||||
@@ -73,15 +73,16 @@ Writing comma-separated values (CSV) instead is available with `ingest --format
|
||||
The pooled statistics for each item (difficulty, discrimination, per-option selection rates, flags, and any item response theory (IRT) parameters) live in a fifth file, `analysis/calibration.yaml`, written only by `calibrate --apply`.
|
||||
A command that loads the course attaches those statistics to each item by id, so the statistics are available wherever the item is used, though the bank file never contains them.
|
||||
|
||||
The files `course.yaml`, `references.yaml`, `lectures/*.yaml`, and `objectives/*.yaml` are one course file split by subject, and a course that keeps everything in `course.yaml` loads unchanged.
|
||||
The files `course.yaml`, `references.yaml`, `glossary.yaml`, `lectures/*.yaml`, and `objectives/*.yaml` are one course file split by subject, and a course that keeps everything in `course.yaml` loads unchanged.
|
||||
The full directory layout is:
|
||||
|
||||
| Path | Role | Written by | Ignored by `init`'s `.gitignore` |
|
||||
|------|------|-----------|---------|
|
||||
| `course.yaml` | identity, policy, units | you | no |
|
||||
| `references.yaml` | works cited | you | no |
|
||||
| `lectures/*.yaml` | one lecture each: readings and what it teaches | you | no |
|
||||
| `objectives/*.yaml` | one learning objective each, with its learning targets | you | no |
|
||||
| `glossary.yaml` | terms, equations, values, and misconceptions no single objective owns | you, or `migrate glossary` | no |
|
||||
| `lectures/*.yaml` | one lecture each: readings, materials, and what it teaches | you | no |
|
||||
| `objectives/*.yaml` | one learning objective each, with its learning targets and the glossary entries they first need | you | no |
|
||||
| `banks/*.yaml` | item banks | you | no |
|
||||
| `assessments/*.yaml` | assessment records | `assemble`, then you | no |
|
||||
| `seals/*.yaml` | frozen copy of what was printed | `seal` | no |
|
||||
@@ -93,7 +94,7 @@ The full directory layout is:
|
||||
| `schema/` | JSON (JavaScript Object Notation) Schemas for editor validation | `init`, `schema` | no |
|
||||
| `templates/` | Typst templates and `typst.yaml` | `template dump`, `template config` | no |
|
||||
|
||||
`coursebank course files` lists the files the course is assembled from, `coursebank course build` prints the merged course, and `coursebank course where ID` says which file defines a given unit, lecture, objective, target, reference, or stimulus id.
|
||||
`coursebank course files` lists the files the course is assembled from, `coursebank course build` prints the merged course, and `coursebank course where ID` says which file defines a given unit, lecture, objective, target, reference, stimulus, term, equation, value, or misconception id.
|
||||
Every command merges the fragments in memory, and each id may be defined in only one place.
|
||||
`course build` is for reading and for other programs: nothing in coursebank reads what it writes.
|
||||
|
||||
@@ -291,10 +292,17 @@ An objective is what a report classifies, and a target is what items are tagged
|
||||
An id may exist in only one of the two registries, and an item that tags an objective that has targets is a validation error: tag the target.
|
||||
Lectures list the objectives they `teach`, and readings attach to lectures with a `ref`, a `locator`, a `role` (`assigned` or `supplemental`), and the targets they support.
|
||||
The `policy` block carries defaults such as `points_per_item` (1.0), `options_per_item` (4), `mastery_threshold` (0.75), `min_items_for_mastery` (2), `allow_partial_credit` (true), `bonus_levels`, and a `grade_scale`.
|
||||
Schema 2.1 adds the course's names for the five levels (`levels`), its `exam_conditions`, named `blueprints`, and `target_text_forbid`, patterns that objective and target text must not match; `exam_conditions` and `blueprints` are stored and validated, and no command reads them yet.
|
||||
|
||||
Schema 2.1 also adds four registries beside the objectives and targets: terms (`term-`), equations (`eq-`), values (`val-`), and misconceptions (`mis-`).
|
||||
Each entry is defined once, in the objective file whose targets first need it or in `glossary.yaml`, and a target lists the entries it turns on and carries a student-facing `explanation`.
|
||||
Student-facing prose names an entry in braces, such as `{term-gap-penalty}`, and every page renders the reference.
|
||||
A bank option names the misconception it encodes with `misconception_id`.
|
||||
The guide's `glossary` page has the fields, the brace table, and the example.
|
||||
|
||||
### Editor support
|
||||
|
||||
`init` and `schema` write eight JSON Schemas into `schema/`: `course`, `references`, `lecture`, `objective`, `calibration`, `measurements`, `bank`, and `assessment`.
|
||||
`init` and `schema` write nine JSON Schemas into `schema/`: `course`, `references`, `glossary`, `lecture`, `objective`, `calibration`, `measurements`, `bank`, and `assessment`.
|
||||
Put a modeline at the top of a YAML file and an editor with a YAML language server validates the file as you type:
|
||||
|
||||
```yaml
|
||||
@@ -309,10 +317,11 @@ Use the `../schema/` form shown above.
|
||||
|
||||
`coursebank validate` checks that every file is correct and consistent, and it exits 2 if it finds any problem.
|
||||
It covers the course (empty identity fields, grade scale, duplicate ids, reference fields, prerequisite cycles, level ceilings), every bank against the course (the rules above, unknown targets, lectures, stimuli, and references), every assessment record against the catalog, every seal's stem digests, and calibration against the banks.
|
||||
For the 2.1 registries it checks id prefixes, ids unique across every registry, the entries each target lists, equation symbols, values, citation keys, brace references in student-facing prose, that every `conflicts` and `target_text_forbid` pattern compiles, and each option's `misconception_id`.
|
||||
A bank item that carries a `calibration:` block fails validation.
|
||||
|
||||
`coursebank lint` checks craft.
|
||||
It has 30 rules in five families, and each rule has a code.
|
||||
It has 44 rules in six families, and each rule has a code.
|
||||
|
||||
| Family | Codes | What they flag |
|
||||
|--------|-------|----------------|
|
||||
@@ -321,11 +330,13 @@ It has 30 rules in five families, and each rule has a code.
|
||||
| Completeness (7) | `complete-distractor-design`, `complete-distractor-explanation`, `complete-student-feedback`, `complete-alt-text`, `complete-format-level`, `complete-bonus-policy`, `complete-design-conflict` | missing design intent, feedback, or alt text |
|
||||
| Pool (3) | `pool-thin`, `pool-unused`, `pool-unjustified-retirement` | too few usable distractors for a form |
|
||||
| Evidence (5) | `evidence-nonfunctioning`, `evidence-stale`, `evidence-difficulty`, `evidence-discrimination`, `evidence-duplicate-stem` | observed results that contradict the prediction or the text |
|
||||
| Glossary (14) | `glossary-unused`, `glossary-no-textbook-name`, `glossary-no-exam-wording`, `glossary-circular-definition`, `glossary-coined-in-item`, `glossary-value-unsourced`, `glossary-value-conflict`, `glossary-forbidden-target-wording`, `glossary-unlisted-term`, `glossary-recall-value-printed`, `glossary-equation-no-implications`, `glossary-unresolved-reference`, `glossary-defined-elsewhere`, `glossary-proposed` | entries nothing needs, coined terms in items, printed recall values, conflicting wordings of a number, and entries waiting for approval |
|
||||
|
||||
Each finding has a severity of low, medium, or high, and a finding's severity can differ from its rule's nominal one (for example, `cue-uneven-length` is medium when the spread is extreme).
|
||||
Each finding has a severity of note, low, medium, or high, and a finding's severity can differ from its rule's nominal one (for example, `cue-uneven-length` is medium when the spread is extreme).
|
||||
Two rules report notes: `glossary-defined-elsewhere` and `glossary-proposed`.
|
||||
Retired items are skipped, and findings are grouped by item.
|
||||
The options are `--list-rules` (list every code and severity, then exit), `--only CODES`, `--ignore CODES` (both comma-separated), `--min-severity low|medium|high` (default `low`), and `--no-fail`.
|
||||
`lint` exits 2 if it finds anything, unless `--no-fail` is given, which is why a pre-commit hook of `coursebank validate && coursebank lint` works.
|
||||
The options are `--list-rules` (list every code and severity, then exit), `--only CODES`, `--ignore CODES` (both comma-separated), `--min-severity note|low|medium|high` (default `note`), and `--no-fail`.
|
||||
`lint` exits 2 if it finds anything above a note, unless `--no-fail` is given, which is why a pre-commit hook of `coursebank validate && coursebank lint` works.
|
||||
To block a commit only on serious findings, add `--min-severity high`.
|
||||
|
||||
`coursebank catalog` summarizes the pool: item, bank, and assemblable counts, and counts by status and by level.
|
||||
@@ -553,6 +564,10 @@ The guide describes the lock as a deterrent, not a grading system of record (`do
|
||||
|
||||
`lecture readings ID` and `lecture objectives ID` write the readings block and the learning-objectives block for one lecture, as Quarto Pandoc definition lists (`--style quarto`, the default) or as plain Markdown bullets (`--style plain`).
|
||||
They print to standard output without `--out`.
|
||||
`lecture page ID` writes the whole page for one lecture: its description, each objective with its targets (each with its explanation and its ceiling named by `policy.levels`), new and earlier terms, equations, numbers to know, common mistakes, readings with minutes, and materials.
|
||||
`lecture glossary` writes the course glossary page: every term alphabetically, then every equation and value, each with the lecture that introduces it.
|
||||
Both take `--style` and `--out`, and `lecture page` takes `--glossary-href` for where links to earlier terms point (default `glossary.html`).
|
||||
None of these prints `authoring`, `notes_private`, `evidence`, `conflicts`, or `status`.
|
||||
`references export` writes the bibliography as `hayagriva` (the default, read by Typst), `csl-json` (read by Zotero and Pandoc), or `bibtex`.
|
||||
The `label`, `role`, `base_url`, and `note` fields do not survive into those formats.
|
||||
|
||||
@@ -769,6 +784,8 @@ The student report answers "what should I do next?"
|
||||
It includes the student's performance by level and by objective, strengths, the objectives to focus on, per-question results with the feedback and hint written for the option the student chose, dropped questions, lectures to review with readings, and a study list.
|
||||
It leaves out the correct answers, other students' data, and any numeric rank.
|
||||
Two optional switches change that: `--solutions` adds the worked solution for every missed question (the command warns about the effect on reusing the exam), and `--misconceptions` names the misconception each chosen distractor was written to catch, which is off by default.
|
||||
Each missed target shows what meeting it looks like, from the target's `explanation`.
|
||||
A chosen option with a `misconception_id` shows that misconception's correction, and misses that share one across questions are grouped under it.
|
||||
`--ability` adds the IRT ability estimate and `--no-comparison` omits the comparison with the class.
|
||||
Section switches turn individual sections off: `--no-questions`, `--no-feedback`, `--no-hints`, `--no-levels`, `--no-objectives`, `--no-strengths`, `--no-focus`, `--no-dropped-questions`, `--no-review-lectures`, and `--no-study`.
|
||||
Files go to `reports/ID/`, named for the student key, unless `--out DIR` says otherwise.
|
||||
@@ -792,7 +809,8 @@ A reference link is resolved in this order: an explicit `url`, then `base_url` p
|
||||
## Migrating an older course
|
||||
|
||||
Every `migrate` subcommand takes `--dry-run`, which shows what would change and writes nothing.
|
||||
The current schema version is `2.0`, and major versions 1 and 2 are readable, so a 1.0 course loads unchanged and migration is optional.
|
||||
The current schema version is `2.1`, and major versions 1 and 2 are readable, so a 1.0 or 2.0 course loads unchanged and migration is optional.
|
||||
2.1 is additive: a course that validated under 2.0 validates under 2.1 with no new errors.
|
||||
|
||||
| Command | What it does |
|
||||
|---------|-------------|
|
||||
@@ -804,20 +822,21 @@ The current schema version is `2.0`, and major versions 1 and 2 are readable, so
|
||||
| `migrate stems` | drop the ignored `version:` and `history:` fields |
|
||||
| `migrate options` | rewrite option letters as names derived from option text; run with `--dry-run` first; seals keep their letters on purpose; exits 2 if some items need hand-naming |
|
||||
| `migrate ids` | rewrite pre-2.0 `bank::item` ids as plain item ids in records, seals, and the response store |
|
||||
| `migrate glossary --terms FILE --values FILE` | bring a 1.0 `terms.yaml` and `values.yaml` into `glossary.yaml`: ids gain `term-` and `val-`, `course` becomes `term`, `meaning` becomes `definition`, and `first` and `lectures` are dropped because 2.1 derives them; reports renamed ids, every dropped `first` or `lectures` the targets now contradict, and entries no target lists yet; leaves the old files in place and skips entries the course already declares, so a second run is a no-op |
|
||||
|
||||
Run `--dry-run` first, and commit before applying.
|
||||
The hints that some `migrate` commands print afterwards name commands that do not exist (see the limits section).
|
||||
|
||||
## Command reference
|
||||
|
||||
coursebank has 21 top-level names and 47 runnable commands: 9 single commands and 12 groups holding 38 subcommands.
|
||||
coursebank has 21 top-level names and 50 runnable commands: 9 single commands and 12 groups holding 41 subcommands.
|
||||
Every command accepts `--course PATH` (or `-C PATH`) and `--quiet` (or `-q`).
|
||||
Identifiers are positional.
|
||||
|
||||
| Stage | Command | What it does |
|
||||
|-------|---------|--------------|
|
||||
| Project setup | `init --code C --title T --term TERM [--with-examples]` | create a course; refuses to overwrite `course.yaml` |
|
||||
| | `schema` | write the eight JSON Schemas to `schema/` |
|
||||
| | `schema` | write the nine JSON Schemas to `schema/` |
|
||||
| | `validate` | check every file for problems that must be fixed; exits 2 on any |
|
||||
| | `lint [flags]` | check item craft; see Authoring items |
|
||||
| | `catalog [--coverage] [--topics]` | summarize the pool and objective coverage |
|
||||
@@ -835,8 +854,8 @@ Identifiers are positional.
|
||||
| | `calibrate` | pool every stored administration into `analysis/calibration.yaml` |
|
||||
| | `report students ID`, `report cohort ID` | per-student and instructor reports |
|
||||
| | `data` | list the stored administrations |
|
||||
| Lectures | `lecture readings ID`, `lecture objectives ID`, `lecture coverage` | site partials; reading coverage |
|
||||
| Migration | `migrate split`, `counters`, `order`, `references`, `variants`, `stems`, `options`, `ids` | upgrade an older course; all take `--dry-run` |
|
||||
| Lectures | `lecture readings ID`, `lecture objectives ID`, `lecture page ID`, `lecture glossary`, `lecture coverage` | site partials; the whole lecture page; the course glossary page; reading coverage |
|
||||
| Migration | `migrate split`, `counters`, `order`, `references`, `variants`, `stems`, `options`, `ids`, `glossary` | upgrade an older course; all take `--dry-run` |
|
||||
|
||||
coursebank has no `import`, `guide`, or `completions` subcommand.
|
||||
The only help is `--help`, and the guide pages are rustdoc modules.
|
||||
@@ -1046,12 +1065,13 @@ Each item below states what the code does.
|
||||
- `export typst --form` takes one value or `all`, not a repeated flag.
|
||||
- `export typst --variant student-report|cohort-report` is accepted by the parser; its behavior was not verified, so use `report` for those documents.
|
||||
- `init --with-examples` writes only an empty bank, `banks/example.yaml`, and not an example assessment.
|
||||
- `init` writes 8 schema files, not the 4 that `docs/guide/setup.md` shows.
|
||||
- `init` writes 9 schema files, not the 4 that `docs/guide/setup.md` shows.
|
||||
- `policy.exam_conditions` and `policy.blueprints` are stored and validated, and no command reads them yet; `assemble` still takes its counts from the command line.
|
||||
- `schema` prints a modeline path (`../.coursebank/schema`) that differs from where it writes the files (`schema/`).
|
||||
- `--apply` exists only on `calibrate`, and it writes `analysis/calibration.yaml`, not a bank. The `src/main.rs` documentation says every command that modifies a bank prints what it would change and requires `--apply`; the `migrate` commands use `--dry-run` instead.
|
||||
- There is no `import` subcommand, although `src/main.rs` and a `bank` comment mention a legacy importer.
|
||||
- The guide pages use `analyze items --assessment exam-2` and similar forms. On the command line the id is positional, and `calibrate` has no assessment argument. The IRT model value is `two-pl`, not `2pl`.
|
||||
- `lint --rules`, as shown in two guide pages, is `lint --list-rules`. The lint rule count is 30, not the 26 in one doc comment.
|
||||
- `lint --rules`, as shown in two guide pages, is `lint --list-rules`.
|
||||
- `migrate counters` and `migrate ids` print follow-up hints that name `seal verify` and `analyze items --all`, neither of which exists. Use `seal --check` to verify.
|
||||
|
||||
### Analysis
|
||||
@@ -1064,6 +1084,8 @@ Each item below states what the code does.
|
||||
|
||||
- `#sym.minus`, used in an authoring example, is not in the symbol table.
|
||||
- Fenced code blocks in stems are not handled by the markup converter.
|
||||
- The lecture and glossary pages leave `$...$` math and an equation's `tex` for Quarto to render, and a value's `say` is printed as written, so a `say` like `about 10^{59}` shows its caret and braces unless it is written in math markup.
|
||||
- `conflicts` and `target_text_forbid` patterns use Rust's `regex` syntax, which has no look-around and no backreferences.
|
||||
- Item figures (`assets`) go into the Typst payload only, and the bundled Typst templates do not print them. The QTI, practice, and site exports do not emit assets.
|
||||
- The bundled Typst templates import `@preview` packages, which the Typst compiler resolves; whether they are cached or downloaded at compile time was not checked.
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# Codes: the specification's placeholders, mapped onto coursebank
|
||||
|
||||
coursebank's validation errors are plain messages with no code, and its lint rules have kebab-case codes prefixed by family (`cue-`, `clarity-`, `complete-`, `pool-`, `evidence-`).
|
||||
The specification's codes map onto those as below.
|
||||
An error is identified by its message, which starts with the entry it is about in backticks, so `validate` can prefix it with the file that defines that entry.
|
||||
The message wording is stable: the tests in `tests/validate_2_1.rs` assert it.
|
||||
|
||||
## Errors (`coursebank validate`, exit 2)
|
||||
|
||||
| Spec | Message (`x` stands for the id) | Where | Test |
|
||||
|---|---|---|---|
|
||||
| E2101 | ``term `x` is defined in two places: A and B. An id has one definition site; delete one or rename it.`` (two files, one registry; the 2.0 message, now for the four new registries too) | `fragment.rs`, `duplicate` | `schema_2_1.rs`: `a_glossary_id_defined_twice_names_both_files` |
|
||||
| E2101 | `` `x` is declared as both a term and a value; an id is unique across all registries `` (any two of objectives, targets, terms, equations, values, misconceptions) | `glossary.rs`, `registry_id_problems` | `e2101_an_id_in_two_registries` |
|
||||
| E2101 (added) | ``term `x`: a term id is `term-` followed by lowercase words joined by hyphens, such as `term-…` `` (the schemas' `propertyNames` patterns, enforced at load since nothing validates YAML against the JSON Schema) | `glossary.rs`, `registry_id_problems` | `e2101_a_registry_id_without_its_prefix` |
|
||||
| E2102 | ``target `t`: `terms` names `x`, which is not a declared term`` (also when the id belongs to another registry) | `glossary.rs`, `target_link_problems` | `e2102_a_target_lists_an_undeclared_entry`, `e2102_an_id_listed_under_the_wrong_registry` |
|
||||
| E2103 | ``equation `eq-x`: symbol `d` names the term `term-y`, which is not declared`` | `glossary.rs`, `entry_problems` | `e2103_a_symbol_naming_an_undeclared_term` |
|
||||
| E2103 | ``equation `eq-x`: symbol `d` needs a `term` or a `meaning` `` | `glossary.rs`, `entry_problems` | `e2103_a_symbol_with_neither_term_nor_meaning` |
|
||||
| E2104 | ``value `val-x`: give a `value` or a `range` `` | `glossary.rs`, `entry_problems` | `e2104_a_value_with_neither_value_nor_range` |
|
||||
| E2104 | ``value `val-x`: range runs from 9 down to 1; write the low end first`` | `glossary.rs`, `entry_problems` | `e2104_a_range_that_runs_high_to_low_and_an_undeclared_term` |
|
||||
| E2104 | ``value `val-x`: term `term-y` is not declared`` | `glossary.rs`, `entry_problems` | the same test |
|
||||
| E2105 | ``term `x`: textbook cites `KEY`, which is not in references`` (also `source` on equations and values, `evidence` on misconceptions) | `glossary.rs`, `entry_problems` | `e2105_a_citation_key_not_in_references` |
|
||||
| E2106 | ``lecture `l09`: description refers to `{term-x}`, which is not a declared term`` (owner and field vary; every student-facing prose field) | `glossary.rs`, `brace_problems` | `e2106_a_brace_reference_that_does_not_resolve`, `e2106_a_new_field_checks_every_kind` |
|
||||
| E2106 (narrowed) | an unresolved `{lo-…}` or `{t-…}` in a reading's `summary`, `focus`, or `skip` is not an error but the lint rule `glossary-unresolved-reference`, because 2.0 accepted it (Q2, option b) | `lint/glossary.rs` | `e2106_an_old_reading_keeps_its_2_0_behaviour_for_targets`, `an_unresolved_target_reference_in_an_old_reading_field_is_a_warning` |
|
||||
| E2107 | ``value `val-x`: conflicts pattern `…` does not compile: …`` | `glossary.rs`, `pattern_problems` | `e2107_a_pattern_that_does_not_compile` |
|
||||
| E2107 | ``policy.target_text_forbid: pattern `…` does not compile: …`` | `glossary.rs`, `pattern_problems` | the same test |
|
||||
| E2108 | ``option o-x: misconception_id `mis-x` is not a declared misconception`` | `bank.rs`, `validate_item` | `e2108_an_option_names_an_undeclared_misconception` |
|
||||
|
||||
Errors the specification does not number, added because the schemas imply them (required, non-empty, or in a range) and the loader would otherwise accept the file:
|
||||
|
||||
| Message | Test (`tests/validate_2_1.rs`) |
|
||||
|---|---|
|
||||
| ``term `x`: empty term``, `empty definition`, ``textbook name N is empty``; ``equation `eq-x`: empty name`` (or `tex`, `says`), ``a symbol has no `symbol` ``, ``a direction implication has no statement``; ``value `val-x`: empty name``, `empty say`; ``misconception `mis-x`: empty belief``, `empty correction` | `empty_required_text_and_out_of_range_numbers_are_errors` (one of each kind) |
|
||||
| ``lecture `l09`: material 2 has no url``; ``lecture `l09` reading 1: minutes must be at least 1`` | the same test |
|
||||
| ``policy.levels: `7` is not a level; levels run from 1 to 5``; ``policy.blueprints: `module` counts items at `6`, which is not a level`` | `policy_levels_and_blueprints_are_keyed_by_level` |
|
||||
| ``policy.levels: level 3 has an empty name``; ``policy.blueprints: `module` gives no time; minutes must be above 0``; `policy.exam_conditions: seconds_per_item must be above 0` | `empty_required_text_and_out_of_range_numbers_are_errors` |
|
||||
|
||||
## Lint (`coursebank lint`)
|
||||
|
||||
Every 2.1 rule is in a new `glossary` family.
|
||||
|
||||
| Spec | Code | Severity | Test (`tests/lint_2_1.rs`) |
|
||||
|---|---|---|---|
|
||||
| W2101 | `glossary-unused` | low | `w2101_an_entry_nothing_needs` |
|
||||
| W2102 | `glossary-no-textbook-name` | medium | `w2102_a_renamed_term_with_no_textbook_name` |
|
||||
| W2103 | `glossary-no-exam-wording` | low | `w2103_a_coined_term_with_no_exam_wording` |
|
||||
| W2104 | `glossary-circular-definition` | low | `w2104_a_definition_that_uses_its_term` |
|
||||
| W2105 | `glossary-coined-in-item` | medium | `w2105_a_coined_term_in_an_item` |
|
||||
| W2106 | `glossary-value-unsourced` | low | `w2106_a_value_with_no_source` |
|
||||
| W2107 | `glossary-value-conflict` | medium | `w2107_a_value_variant_in_student_prose_and_in_an_item` |
|
||||
| W2108 | `glossary-forbidden-target-wording` | medium | `w2108_target_text_the_policy_forbids` |
|
||||
| W2109 | `glossary-unlisted-term` | low | `w2109_an_explanation_using_a_term_its_target_does_not_list`, `w2109_a_term_inherited_from_a_prerequisite_target_is_listed` |
|
||||
| W2110 | `glossary-recall-value-printed` | medium | `w2110_an_item_prints_a_recall_value` |
|
||||
| W2111 | `glossary-equation-no-implications` | low | `w2111_a_deep_target_on_an_equation_with_no_implications` |
|
||||
| (added) | `glossary-unresolved-reference` | medium | `an_unresolved_target_reference_in_an_old_reading_field_is_a_warning` |
|
||||
| I2112 | `glossary-defined-elsewhere` | note | `i2112_an_entry_defined_away_from_the_objective_that_first_needs_it` |
|
||||
| I2113 | `glossary-proposed` | note | `i2113_counts_proposed_entries` |
|
||||
|
||||
`note` is a new severity below `low` (Q3). Notes print by default, `--min-severity low` hides them, and `lint` never exits non-zero for notes alone (`notes_are_printed_but_do_not_fail_lint`).
|
||||
On the example course the glossary family reports exactly `glossary-unused` on `val-per-decade` and one `glossary-proposed` finding counting two entries (`the_example_draws_exactly_the_two_expected_glossary_findings`).
|
||||
|
||||
Two existing rules changed with 2.1: `complete-distractor-design` accepts a `misconception_id` as the distractor's design, and `complete-student-feedback` accepts a `misconception_id` that resolves, since the registry's correction is the feedback (`a_registry_misconception_designs_a_distractor_and_gives_it_feedback`).
|
||||
|
||||
## Regular expressions
|
||||
|
||||
`conflicts` and `target_text_forbid` use the Rust `regex` crate (1.x, `std`, `unicode-perl`, `unicode-case`).
|
||||
Its syntax matches Python's `re` for everything the examples use (`\b`, `\s`, `\.`, `\^`, `\{?`), and it has no look-around and no backreferences; a pattern that uses them fails E2107 with the crate's reason.
|
||||
`target_text_forbid` matches case-insensitively, as the prototype's `re.I` does; `conflicts` matches as written.
|
||||
@@ -0,0 +1,98 @@
|
||||
# coursebank 2.1: implementation report
|
||||
|
||||
Branch `schema-2.1`, six commits, one per phase, nothing pushed.
|
||||
Every phase ended with `cargo fmt`, `cargo clippy --all-targets -- -D warnings`, `cargo test`, and `cargo doc --no-deps` clean.
|
||||
The 2.1 work adds 57 integration tests (schema 7, validate 18, lint 19, render 8, migrate 5) and 14 unit tests, and every existing test still passes unchanged.
|
||||
|
||||
## What changed, by phase
|
||||
|
||||
| Phase | Commit | Files |
|
||||
|---|---|---|
|
||||
| 1. Types and schemas | `51a2fd2` | `src/model/course/glossary.rs` (new: every entry type, `RefKind`, the brace scanner), `src/model/course.rs` (the four registries, new fields on objectives, targets, lectures, readings, policy; `SCHEMA_VERSION = "2.1"`), `src/model/course/fragment.rs` (four sections, `Role::Glossary`, `glossary.yaml`), `src/model/layout.rs`, `src/model/item.rs` (`misconception_id`), `src/authoring/jsonschema.rs` (`Kind::Glossary` and every changed builder), `tests/schema_2_1.rs`, `tests/docs.rs` |
|
||||
| 2. Validation | `960cf6c` | `src/model/course/glossary.rs` (`glossary_problems`: E2101 to E2107 and the prefix check), `src/model/bank.rs` (E2108), `Cargo.toml` (`regex`), `tests/validate_2_1.rs`, `tests/common/mod.rs` |
|
||||
| 3. Lint | `b8fdf4d` | `src/authoring/lint.rs` (14 rules, `glossary` family; two existing rules accept `misconception_id`), `src/authoring/lint/glossary.rs` (new), `src/model/catalog.rs` (`Severity::Note`), `src/cli.rs`, `src/commands/project.rs` (notes never fail lint), `tests/lint_2_1.rs` |
|
||||
| 4. Derived data and rendering | `50a1e08` | `src/model/course/glossary.rs` (first use, teaching order, `expand_refs`), `src/export/lecture.rs` (`page_markdown`, `glossary_markdown`, brace rendering), `src/cli.rs` and `src/commands/lectures.rs` (`lecture page`, `lecture glossary`), `src/analysis/students.rs`, `src/export/report.rs`, `src/analysis/diagnostic.rs`, `src/export/typst/diagnostic.rs`, `src/export/typst/templates/student-report.typ`, `tests/render_2_1.rs` |
|
||||
| 5. Migration | `6c07122` | `src/migrate/glossary.rs` (new), `src/migrate.rs`, `src/cli.rs`, `src/commands/project.rs`, `tests/migrate_2_1.rs` |
|
||||
| 6. Documentation | this commit | `docs/guide/glossary.md` (new) and `src/guide.rs`, `docs/guide/setup.md`, `README.md`, `CHANGELOG.md`, module docs in `src/lib.rs`, `src/model.rs`, `src/authoring.rs`, `docs/design/2.1/CODES.md`, this file, one more test in `tests/validate_2_1.rs` |
|
||||
|
||||
The rendered example lecture page was sent as a private artifact in Phase 4.
|
||||
|
||||
## Code mapping
|
||||
|
||||
In `CODES.md`. In short: E2101 to E2108 are validation messages without codes, as every coursebank error is; W2101 to W2111 are the `glossary-*` lint rules; I2112 and I2113 are `glossary-defined-elsewhere` and `glossary-proposed` at a new `note` severity.
|
||||
Two additions: a prefix error next to E2101, and the lint rule `glossary-unresolved-reference` that takes E2106's place in 2.0 reading fields.
|
||||
|
||||
## Generated versus target schemas
|
||||
|
||||
The schemas are not generated from the Rust types: `src/authoring/jsonschema.rs` writes each one by hand with `serde_json::json!`, and that was the case before 2.1.
|
||||
So rule 2 became: port every edit in `build_schemas.py` into those builders, with the descriptions copied word for word.
|
||||
Hand-written JSON can say `propertyNames`, so the id patterns are in the schemas; the validator also enforces them, because nothing checks YAML against the JSON Schema at load time.
|
||||
|
||||
`tests/schema_2_1.rs` compares each of the nine generated schemas with `docs/design/2.1/schemas/` as JSON values. Two differences remain, both older than 2.1 and both one word:
|
||||
|
||||
| File | Path | Generated | Target |
|
||||
|---|---|---|---|
|
||||
| `calibration.schema.json` | `/properties/items/additionalProperties/properties/variants/items/properties/option_stats/description` | behaviour | behavior |
|
||||
| `course.schema.json` | `/properties/policy/properties/grade_scale/items/properties/group/description` | Colour | Color |
|
||||
|
||||
`schemas-original/` has the American spellings, which no commit in this repository ever had.
|
||||
I kept the code's spelling; a test fails if either difference disappears or another appears.
|
||||
|
||||
## What I did differently from the specification, and why
|
||||
|
||||
1. **Lecture files and `references.yaml` cannot define glossary entries.** The specification says any fragment may; the target schemas add the registries only to `course`, `objective`, and `glossary`, and I followed the schemas. A lecture file that tries gets the existing "a section belongs to a kind of file" error.
|
||||
2. **Teaching order is coursebank's: lectures in id order.** The prototype sorts lectures by `(date, id)`. The specification says to use the existing order, which is the lecture map's id order. They agree on the example.
|
||||
3. **E2106 on 2.0 fields (Q2, option b).** The four new prefixes are errors everywhere. An unresolved `{lo-…}` or `{t-…}` in a reading's `summary`, `focus`, or `skip` stays a medium lint finding, because 2.0 accepted it and rule 1 says no new errors.
|
||||
4. **Note severity (Q3).** `glossary-defined-elsewhere` and `glossary-proposed` are notes. `lint --min-severity` now defaults to `note`, and `lint` exits 2 only when something is above a note, so a course with proposed entries passes a lint gate.
|
||||
5. **`regex` crate (Q1).** Built without default features (`std`, `unicode-perl`, `unicode-case`). No look-around and no backreferences; such a pattern fails E2107 with the crate's reason.
|
||||
6. **The preview's `(T t-align-fill-matrix)` renders as `(T 3)`**, the target's number on the page, as 2.0's readings page always did. The prototype prints the raw id.
|
||||
7. **Three lines the preview does not have.** The page prints "About 40 minutes of assigned reading." above the readings, and each reading's `summary` and `skip`, which the 2.0 readings page prints and the specification's "readings (with minutes)" does not rule out. The snapshot test allows exactly these three.
|
||||
8. **Anchors are `glossary-<id>`**, such as `#glossary-term-gap`, not `#term-gap`. Quarto reserves the `eq-` prefix for numbered equations, so `{#eq-global-recurrence}` would be read as an equation cross-reference.
|
||||
9. **Report wording.** An option's own `misconception` text keeps its 2.0 sentence. An option with only a `misconception_id` prints "This answer reflects a common mistake: *belief*", then the correction. Misses sharing one are grouped under "The same mistake, more than once" with their question numbers. The registry belief and correction are student-facing by design, so they print without `--misconceptions`; the option's own text keeps that switch.
|
||||
10. **Two existing lint rules changed.** `complete-distractor-design` accepts a `misconception_id` as design, and `complete-student-feedback` accepts one that resolves, since the correction is the feedback.
|
||||
11. **`policy.exam_conditions` and `policy.blueprints` are stored and validated, but nothing reads them.** No phase asked for it. `assemble --blueprint NAME` reading `policy.blueprints` is a small follow-up.
|
||||
12. **More errors than E2101 to E2108.** Required text that is present but empty, a material without a URL, `minutes: 0`, level keys outside 1 to 5, and non-positive durations. They follow from the schemas, cannot fire on 2.0 data, and are listed in `CODES.md`.
|
||||
13. **Proposed entries look approved to students.** Nothing on a student page says "proposed"; the rule-5 test checks the word never appears.
|
||||
14. **The example needed one edit.** The bank item had no `status`, which coursebank requires; I added `status: draft` with a comment. With `approved`, validation would also require `cognitive_process`, `sources`, and `design`.
|
||||
15. **The value `say: 'about 10^{59}'` renders literally**, caret and braces, because `say` is printed as written and the example does not put it in math markup.
|
||||
16. **The prototype was read, not run.** `pyv` has no vendored PyYAML on this machine, so `check21.py --selftest` could not run. Each self-test mutation became a test, built by reading the prototype.
|
||||
17. **Migration details the specification leaves open:**
|
||||
- a value's `term:` gains the prefix of the id it names, and is reported;
|
||||
- fields 2.1 has no place for are dropped and reported;
|
||||
- an existing `glossary.yaml` is added to as text, never rewritten;
|
||||
- entries declared anywhere in the course are skipped, so a second run is a no-op;
|
||||
- an id that would be invalid with its prefix, two entries that would share an id, and an id another registry already uses are refused before anything is written;
|
||||
- an entry written as a flow mapping is written by the serializer instead of copied.
|
||||
|
||||
On the example, `coursebank lint` reports the two expected glossary findings and one existing finding: `cue-absolute` on option `o-length`, which says "always".
|
||||
|
||||
## Running the migration on your course
|
||||
|
||||
From the course root, with the 1.0 files wherever they are:
|
||||
|
||||
```console
|
||||
$ coursebank migrate glossary --terms terms.yaml --values values.yaml --dry-run
|
||||
$ coursebank migrate glossary --terms terms.yaml --values values.yaml
|
||||
$ coursebank validate
|
||||
$ coursebank lint --only glossary-unused,glossary-defined-elsewhere,glossary-proposed
|
||||
```
|
||||
|
||||
The dry run prints the report and the file it would write.
|
||||
The report lists renamed ids, every dropped `first` or `lectures` that the targets now contradict, entries no target lists yet, and fields dropped.
|
||||
Before you link any targets, every entry will be under "no target lists these yet", and every old `first` and `lectures` will be reported as contradicted, because nothing lists the new ids yet.
|
||||
`terms.yaml` and `values.yaml` are left in place.
|
||||
Then add the ids to each target's `terms` and `values`, and move each entry into the objective file that first needs it; `glossary-defined-elsewhere` names those files.
|
||||
|
||||
## Suggested changes to the specification
|
||||
|
||||
1. Say which fragments may define entries: objective files, `glossary.yaml`, and `course.yaml`, as the schemas already do, rather than "any fragment".
|
||||
2. Define teaching order: lectures by id or by date. The prototype and the spec's "existing order" differ for a course whose lecture ids do not sort by date.
|
||||
3. Write down E2106's exemption for 2.0 reading fields, so rule 1 and E2106 do not contradict each other.
|
||||
4. Define the I-codes' effect on exit status: shown, never failing.
|
||||
5. Say whether `say` is plain text or markup. If markup, the example should write `about $10^{59}$`.
|
||||
6. Give the anchors a prefix of their own (`glossary-`), and note that Quarto reserves `eq-`, `fig-`, `tbl-`, and `sec-`.
|
||||
7. Make `lecture-page-preview.md` print target numbers (`T 3`) rather than raw ids, and decide whether a reading's `summary` and `skip` belong on the page.
|
||||
8. Add `status:` to the example bank item, since coursebank requires it.
|
||||
9. Name the regular-expression dialect as the subset Python's `re` and Rust's `regex` share: no look-around, no backreferences.
|
||||
10. In the migration table, add a value's `term:` (prefix it) and say what happens to unknown fields and to an existing `glossary.yaml`.
|
||||
11. Decide whether `policy.blueprints` should feed `assemble`, and say so; today it is data with no reader.
|
||||
@@ -0,0 +1,244 @@
|
||||
# Terms, equations, values, and misconceptions
|
||||
|
||||
Schema 2.1 gives a course four registries beside its objectives and targets: the terms students meet, the equations they use, the numbers they are taught, and the mistakes they commonly make.
|
||||
Each entry is defined once, keyed by id, and a target lists the entries it turns on.
|
||||
The lecture page, the course glossary page, and student reports are built from those lists, so a definition you correct in one place is corrected everywhere it is shown.
|
||||
|
||||
A 2.0 course needs none of this and loads unchanged.
|
||||
|
||||
## The four registries
|
||||
|
||||
| Registry | Id prefix | One entry is |
|
||||
|:--|:--|:--|
|
||||
| `terms` | `term-` | a word or phrase, its one-sentence definition, and what the assigned texts call it |
|
||||
| `equations` | `eq-` | a relation in LaTeX, what it says in words, its symbols, and what it implies |
|
||||
| `values` | `val-` | a number as taught, the one wording every material uses for it, and its source |
|
||||
| `misconceptions` | `mis-` | a belief students hold, in their voice, and its correction |
|
||||
|
||||
After the prefix, an id is lowercase words joined by hyphens, like every other id in the course.
|
||||
Ids are unique across all six registries, objectives and targets included, so `{term-gap}` can only ever mean one thing.
|
||||
Ids are also join keys: renaming one orphans every target, option, and brace reference that names it.
|
||||
|
||||
A term:
|
||||
|
||||
```yaml
|
||||
terms:
|
||||
term-gap-penalty:
|
||||
term: gap penalty
|
||||
symbol: 'd'
|
||||
definition: 'The score subtracted for each position where one sequence is aligned against a {term-gap}.'
|
||||
term-decade:
|
||||
term: decade
|
||||
definition: 'A factor of ten in a concentration ratio or an equilibrium constant.'
|
||||
kind: coined
|
||||
textbook:
|
||||
- { term: order of magnitude, ref: KKW, locator: '§10.10' }
|
||||
variants: [decades]
|
||||
exam_wording: ten-fold
|
||||
```
|
||||
|
||||
`kind` says whether a student can look the term up under this name: `standard` (the default), `renamed` (the course uses its own name for a standard idea), or `coined`.
|
||||
A renamed or coined term should give the `textbook` name, and a coined one the `exam_wording` that items write instead of the label, because an exam should not test vocabulary only your lectures use.
|
||||
|
||||
An equation, a value, and a misconception:
|
||||
|
||||
```yaml
|
||||
equations:
|
||||
eq-global-recurrence:
|
||||
name: the global alignment recurrence
|
||||
tex: 'F(i,j) = \max\begin{cases} F(i-1,j-1) + s(x_i, y_j) \\ F(i-1,j) - d \\ F(i,j-1) - d \end{cases}'
|
||||
says: >-
|
||||
The best score for the first i letters of one sequence against the first j of the other is the best of
|
||||
three moves into that cell.
|
||||
symbols:
|
||||
- { symbol: 'F(i,j)', meaning: 'the best score for aligning the two prefixes' }
|
||||
- { symbol: 'd', term: term-gap-penalty }
|
||||
implications:
|
||||
- { form: direction, statement: 'Raising {term-gap-penalty} makes the diagonal win more cells.' }
|
||||
assumes: ['Each gap position costs the same, whatever its neighbours.']
|
||||
fails_when: ['Gaps come in long runs, which a linear penalty charges too much for.']
|
||||
recall: true
|
||||
|
||||
values:
|
||||
val-per-decade:
|
||||
name: free energy per ten-fold concentration ratio at 37 °C
|
||||
value: 6
|
||||
units: kJ/mol
|
||||
say: about 6 kJ/mol
|
||||
exact: 5.9
|
||||
source: { ref: KKW, locator: '§10.10' }
|
||||
conflicts: ['5\.7\s*kJ', '5\.8\s*kJ']
|
||||
recall: true
|
||||
|
||||
misconceptions:
|
||||
mis-only-matches-count:
|
||||
belief: 'An alignment score counts the matching positions, so the alignment with the most matches scores highest.'
|
||||
correction: 'Every column counts: matches add, mismatches cost a little, and gaps cost more.'
|
||||
```
|
||||
|
||||
An equation's `implications` are the things a question can ask about a relation without evaluating it, each in one of six forms: `direction`, `limit`, `scale`, `dominant_term`, `held_fixed`, or `misuse`.
|
||||
A value's `say` is the wording the lecture page, slides, and items all use, and its `conflicts` are regular expressions for the wordings that must not appear anywhere students read.
|
||||
`recall: true` means students are expected to know the value, so no item may print it.
|
||||
|
||||
Any entry may carry `status: proposed`, which a pipeline sets when it adds one; delete the line to approve it.
|
||||
Proposed entries are checked like any other, and `coursebank lint` counts how many are waiting.
|
||||
|
||||
## Where entries live
|
||||
|
||||
An entry can be defined in three places:
|
||||
|
||||
| File | What goes there |
|
||||
|:--|:--|
|
||||
| `objectives/*.yaml` | the entries an objective's targets are the first in the course to need |
|
||||
| `glossary.yaml` | cross-cutting entries, and whatever `migrate glossary` brought in, until each one is moved |
|
||||
| `course.yaml` | anything, for a course that keeps everything in one file |
|
||||
|
||||
Lecture files and `references.yaml` cannot define entries.
|
||||
Wherever an entry is defined, it is course-wide: a target in any later lecture lists it by id, and nothing redefines it.
|
||||
The convention is the objective file whose targets first need the entry, and `lint` notes an entry that sits elsewhere (`glossary-defined-elsewhere`), without failing.
|
||||
|
||||
## Targets point at entries
|
||||
|
||||
A target lists the entries it turns on, and says in one student-facing paragraph what meeting it looks like:
|
||||
|
||||
```yaml
|
||||
learning_targets:
|
||||
t-align-gap-penalty-effect:
|
||||
text: 'Predict how raising the gap penalty changes the optimal path through the matrix.'
|
||||
objective: lo-alignment-recurrence
|
||||
level_ceiling: 3
|
||||
explanation: >-
|
||||
A larger {term-gap-penalty} makes every horizontal or vertical step cost more, so the diagonal wins more
|
||||
cells and the optimal path uses fewer gaps.
|
||||
terms: [term-gap-penalty]
|
||||
equations: [eq-global-recurrence]
|
||||
misconceptions: [mis-only-matches-count]
|
||||
authoring:
|
||||
ceiling_item: 'Show the optimal path under d = 1 and ask which path wins under d = 4.'
|
||||
supplied: ['the two sequences', 'the optimal path under the lower penalty']
|
||||
relation_form: direction
|
||||
```
|
||||
|
||||
The lists are what a lecture page draws its glossary sections from, and what decides each entry's first lecture: the first lecture, in teaching order, with a target that lists it.
|
||||
Teaching order is the course's own: lectures in id order, objectives in each lecture's `teaches` order, and an objective's targets by ceiling and then id.
|
||||
A term first needed in an earlier lecture appears on a later lecture's page as a link back, not as a new definition.
|
||||
|
||||
`authoring` holds the item-writing judgments that settle what a question at the target's ceiling looks like: `ceiling_item`, `supplied` data, `constraints`, `number_work`, `examples`, `relation_form`, and the evidence for the ceiling.
|
||||
It is for the item pipelines and for you; students never see it.
|
||||
|
||||
## Other fields 2.1 adds
|
||||
|
||||
| On | Field | What it does |
|
||||
|:--|:--|:--|
|
||||
| objective | `description` | a short student-facing paragraph under the objective on the lecture page |
|
||||
| lecture | `description` | the paragraph that opens the lecture page |
|
||||
| lecture | `materials` | slides, briefs, notebooks, and other links, each `{kind, url, title}` |
|
||||
| reading | `minutes` | the reading's length, summed at the top of the readings section |
|
||||
| bank option | `misconception_id` | the registry misconception the distractor encodes |
|
||||
| policy | `exam_conditions` | closed book or not, calculator, how equations are given, formats, seconds per item |
|
||||
| policy | `blueprints` | named level counts, bonus counts, and minutes |
|
||||
| policy | `levels` | the course's names for levels 1 to 5, used to label ceilings on the lecture page |
|
||||
| policy | `target_text_forbid` | regular expressions that objective and target text must not match |
|
||||
|
||||
An option's own `misconception` text, when it has one, still wins for that item.
|
||||
An option with only a `misconception_id` counts as designed, and its feedback is the registry's correction.
|
||||
`exam_conditions` and `blueprints` are stored and validated, but no command reads them yet.
|
||||
|
||||
## Brace references
|
||||
|
||||
Student-facing prose can name an entry in braces, and every page that shows the prose renders the reference:
|
||||
|
||||
| Reference | Renders as |
|
||||
|:--|:--|
|
||||
| `{term-gap-penalty}` | the term, linked to its definition |
|
||||
| `{eq-global-recurrence}` | the equation's name, linked to it |
|
||||
| `{val-per-decade}` | the value's `say`, such as about 6 kJ/mol |
|
||||
| `{mis-only-matches-count}` | the belief, in quotation marks |
|
||||
| `{t-align-fill-matrix}` | `T 3`, the target's number on the page, or the target's text where it has no number |
|
||||
| `{lo-alignment-recurrence}` | the objective's text |
|
||||
|
||||
On a Quarto lecture page, a link points at the definition on the same page when it is there (a term new in that lecture, or an equation its targets use).
|
||||
Every other link points at the course glossary page: `glossary.html`, unless `--glossary-href` says otherwise.
|
||||
Braces that are not a prefixed id, such as `10^{59}` or `\text{final}`, are left alone.
|
||||
|
||||
The fields that take references are the student-facing ones: definitions, `says`, implication statements, `assumes`, `fails_when`, corrections, objective and lecture descriptions, target explanations, and a reading's `summary`, `focus`, and `skip`.
|
||||
|
||||
## What students never see
|
||||
|
||||
`authoring`, `notes_private`, `evidence`, `conflicts`, and `status` are instructor-facing.
|
||||
No lecture page, glossary page, student report, or student export (Canvas, the practice worksheet and its solutions, the site) prints them, and a test renders the example course through every one of those and checks.
|
||||
A proposed entry is shown to students like an approved one, without saying it is proposed.
|
||||
|
||||
## Pages
|
||||
|
||||
```console
|
||||
$ coursebank lecture page l09 --out site/l09.qmd
|
||||
$ coursebank lecture glossary --out site/glossary.qmd
|
||||
```
|
||||
|
||||
`lecture page` writes, in order: the lecture's description; each objective with its description and its targets, each with its explanation and its ceiling named by `policy.levels`; the new terms; earlier terms, linked; the equations; the numbers to know; common mistakes; readings, with minutes; and materials.
|
||||
`lecture glossary` lists every term alphabetically with its definition and first lecture, then every equation and value.
|
||||
Both write Quarto Markdown by default (numbered targets as `(@)` example lists, anchors as `{#glossary-term-...}`), or plain Markdown with `--style plain`.
|
||||
|
||||
Student reports gain two things.
|
||||
Each missed target shows what meeting it looks like, from its `explanation`.
|
||||
A chosen option with a `misconception_id` shows the misconception and its correction, and misses that share one across questions are grouped under it, so a student who made the same mistake three times reads about it once.
|
||||
|
||||
## Checks
|
||||
|
||||
`coursebank validate` reports these as errors, each prefixed with the file that defines the entry:
|
||||
|
||||
| Problem | Message |
|
||||
|:--|:--|
|
||||
| an id lacks its registry's prefix | ``term `x`: a term id is `term-` followed by lowercase words`` |
|
||||
| an id is in two registries, or two files | `` `x` is declared as both a ``, or `is defined in two places` |
|
||||
| a target lists an undeclared entry | ``target `t`: `terms` names `x`, which is not a declared term`` |
|
||||
| an equation symbol names an undeclared term, or has neither `term` nor `meaning` | ``equation `eq-x`: symbol `d` `` |
|
||||
| a value has neither `value` nor `range`, its range runs backwards, or its `term` is undeclared | ``value `val-x`: `` |
|
||||
| a `source`, `textbook`, or `evidence` cites a key not in `references` | ``…: source cites `KEY`, which is not in references`` |
|
||||
| a brace reference does not resolve | ``…: description refers to `{term-x}`, which is not a declared term`` |
|
||||
| a `conflicts` or `target_text_forbid` pattern does not compile | ``value `val-x`: conflicts pattern `…` does not compile`` |
|
||||
| an option's `misconception_id` is undeclared | ``option o-x: misconception_id `mis-x` is not a declared misconception`` |
|
||||
|
||||
An unresolved `{lo-...}` or `{t-...}` in a reading's `summary`, `focus`, or `skip` stays a lint warning (`glossary-unresolved-reference`) rather than an error, because 2.0 accepted it.
|
||||
|
||||
`coursebank lint` adds fourteen rules in a `glossary` family:
|
||||
|
||||
| Code | Severity | Flags |
|
||||
|:--|:--|:--|
|
||||
| `glossary-unused` | low | an entry no target, equation symbol, value, or option needs |
|
||||
| `glossary-no-textbook-name` | medium | a renamed or coined term with no textbook name |
|
||||
| `glossary-no-exam-wording` | low | a coined term with no `exam_wording` |
|
||||
| `glossary-circular-definition` | low | a definition that uses the term it defines |
|
||||
| `glossary-coined-in-item` | medium | a coined term in an item's stem or options |
|
||||
| `glossary-value-unsourced` | low | a value with no `source` |
|
||||
| `glossary-value-conflict` | medium | a `conflicts` pattern that matches student-facing text or an item |
|
||||
| `glossary-forbidden-target-wording` | medium | objective or target text that matches `target_text_forbid` |
|
||||
| `glossary-unlisted-term` | low | an explanation that names a term its target neither lists nor inherits from a prerequisite |
|
||||
| `glossary-recall-value-printed` | medium | an item that prints a `recall: true` value |
|
||||
| `glossary-equation-no-implications` | low | a target at ceiling 3 or above that turns on an equation with no implications |
|
||||
| `glossary-unresolved-reference` | medium | an old prose field that refers to an undeclared objective or target |
|
||||
| `glossary-defined-elsewhere` | note | an entry outside the objective file whose targets first need it |
|
||||
| `glossary-proposed` | note | how many entries are waiting for approval |
|
||||
|
||||
A note is shown by default and never makes `lint` fail, so a course with proposed entries still passes a lint gate.
|
||||
`--min-severity low` hides notes.
|
||||
|
||||
Regular expressions use Rust's `regex` syntax, which has no look-around and no backreferences.
|
||||
`target_text_forbid` matches case-insensitively; `conflicts` matches as written.
|
||||
|
||||
## From `terms.yaml` and `values.yaml`
|
||||
|
||||
A course that kept its terms and numbers in 1.0 side files brings them in with one command:
|
||||
|
||||
```console
|
||||
$ coursebank migrate glossary --terms terms.yaml --values values.yaml --dry-run
|
||||
$ coursebank migrate glossary --terms terms.yaml --values values.yaml
|
||||
```
|
||||
|
||||
Ids gain their prefix (`decade` becomes `term-decade`), `course` becomes `term`, `meaning` becomes `definition`, and a value's `term:` gains the prefix of the id it names.
|
||||
`first` and `lectures` are dropped, because 2.1 derives them from the targets that list an entry; the command reports each one that disagrees with what the targets now say, each entry no target lists yet, and each field 2.1 has no place for.
|
||||
The entries go into `glossary.yaml`, added at the end of its sections if it exists, with comments and quoting copied from the old files.
|
||||
The old files are left alone, and an entry the course already declares is skipped, so a second run changes nothing.
|
||||
|
||||
Linking targets to the migrated entries, and moving each entry into the objective file that first needs it, is then yours or a pipeline's to do; `lint` lists what is left (`glossary-unused`, `glossary-defined-elsewhere`).
|
||||
@@ -208,6 +208,12 @@ stimuli:
|
||||
Items reference it with `stimulus: s-dock-poses`.
|
||||
Declaring it here rather than pasting it into four items means a correction to the table happens once.
|
||||
|
||||
### Terms, equations, values, and misconceptions
|
||||
|
||||
Four more registries hold what students study from besides the objectives: the terms they meet (`term-`), the equations they use (`eq-`), the numbers they are taught (`val-`), and the mistakes they commonly make (`mis-`).
|
||||
Each is defined once, in the objective file whose targets first need it or in a `glossary.yaml` beside `course.yaml`, and targets list the entries they turn on.
|
||||
They are optional, and [`glossary`](crate::guide::glossary) covers them, the lecture pages built from them, and the migration from 1.0 `terms.yaml` and `values.yaml`.
|
||||
|
||||
## Check it
|
||||
|
||||
```console
|
||||
@@ -229,6 +235,7 @@ Exit codes are meaningful.
|
||||
|
||||
## Where to go next
|
||||
|
||||
[`glossary`](crate::guide::glossary) adds terms, equations, values, and misconceptions, and builds lecture pages from them.
|
||||
[`authoring`](crate::guide::authoring) writes the first bank.
|
||||
[`first_exam`](crate::guide::first_exam) takes an exam from blueprint to student report.
|
||||
|
||||
|
||||
+2
-2
@@ -7,8 +7,8 @@
|
||||
//! The three modules here are what you interact with before an exam exists, and
|
||||
//! they divide by how much authority each one has.
|
||||
//!
|
||||
//! [`lint`] has none. Its 26 rules are advice with stable codes, every one
|
||||
//! silenceable with `--ignore`. It is deliberately separate from the validation in
|
||||
//! [`lint`] has none. Its 44 rules are advice with stable codes, every one
|
||||
//! silenceable with `--ignore`, and the two that report notes never fail. It is deliberately separate from the validation in
|
||||
//! [`crate::model::bank`], which enforces what must be true and fails. Conflating
|
||||
//! the two produces a tool that either blocks you on style or lets real errors
|
||||
//! through.
|
||||
|
||||
+11
-5
@@ -16,14 +16,16 @@
|
||||
//!
|
||||
//! 1. [`setup`] builds a course directory from nothing and explains what each file
|
||||
//! is for.
|
||||
//! 2. [`authoring`] writes items, with the design fields that make an item worth
|
||||
//! 2. [`glossary`] adds the terms, equations, values, and misconceptions a
|
||||
//! course's targets turn on, and the lecture pages built from them.
|
||||
//! 3. [`authoring`] writes items, with the design fields that make an item worth
|
||||
//! reusing.
|
||||
//! 3. [`first_exam`] runs one exam end to end: assemble, export, administer,
|
||||
//! 4. [`first_exam`] runs one exam end to end: assemble, export, administer,
|
||||
//! ingest, analyze, report.
|
||||
//! 4. [`typst_export`] covers printed output, template markers, and render config.
|
||||
//! 5. [`assignment`] publishes a homework to the course website, with solutions
|
||||
//! 5. [`typst_export`] covers printed output, template markers, and render config.
|
||||
//! 6. [`assignment`] publishes a homework to the course website, with solutions
|
||||
//! gated behind a password.
|
||||
//! 6. [`recipes`] holds short answers to specific questions, for when you already
|
||||
//! 7. [`recipes`] holds short answers to specific questions, for when you already
|
||||
//! know the shape of the tool.
|
||||
//!
|
||||
//! ## Why the tutorials are in here rather than a wiki
|
||||
@@ -45,6 +47,10 @@
|
||||
#[doc = include_str!("../docs/guide/setup.md")]
|
||||
pub mod setup {}
|
||||
|
||||
/// Terms, equations, values, and misconceptions, and the pages built from them.
|
||||
#[doc = include_str!("../docs/guide/glossary.md")]
|
||||
pub mod glossary {}
|
||||
|
||||
/// Writing items that are worth keeping.
|
||||
#[doc = include_str!("../docs/guide/authoring.md")]
|
||||
pub mod authoring {}
|
||||
|
||||
+8
-5
@@ -20,13 +20,14 @@
|
||||
//! |:--|:--|:--|
|
||||
//! | `course.yaml` | identity, policy, units | you |
|
||||
//! | `references.yaml` | the works the course cites | you |
|
||||
//! | `lectures/*.yaml` | one lecture: readings, and what it teaches | you |
|
||||
//! | `objectives/*.yaml` | one objective and its learning targets | you |
|
||||
//! | `glossary.yaml` | terms, equations, values, and misconceptions no one objective owns | you, or `migrate glossary` |
|
||||
//! | `lectures/*.yaml` | one lecture: readings, materials, and what it teaches | you |
|
||||
//! | `objectives/*.yaml` | one objective, its learning targets, and the glossary entries they first need | you |
|
||||
//! | `banks/*.yaml` | items, with design intent and pooled statistics | you, then `calibrate` |
|
||||
//! | `assessments/*.yaml` | what was given, to whom, when | `assemble`, then you |
|
||||
//! | `data/*.parquet` | one row per student per item | `ingest` |
|
||||
//!
|
||||
//! The first four are one course file split by subject; `coursebank course build`
|
||||
//! The first five are one course file split by subject; `coursebank course build`
|
||||
//! prints the merged result, and a course that keeps everything in `course.yaml`
|
||||
//! still loads unchanged. See [`course::fragment`].
|
||||
//!
|
||||
@@ -75,8 +76,10 @@
|
||||
//!
|
||||
//! ## Dependency posture
|
||||
//!
|
||||
//! Deliberately small: serde, a YAML parser, clap, thiserror, and csv, plus arrow
|
||||
//! and parquet behind a default-on feature that can be switched off. Dates, PRNG,
|
||||
//! Deliberately small: serde, a YAML parser, clap, thiserror, csv, and regex
|
||||
//! (built without its optional features, for the glossary's `conflicts` and
|
||||
//! `target_text_forbid` patterns), plus arrow and parquet behind a default-on
|
||||
//! feature that can be switched off. Dates, PRNG,
|
||||
//! hashing, ZIP writing, and the psychometrics are implemented here rather than
|
||||
//! pulled in — see [`date`], [`rng`], [`hash`], [`zipfile`], [`irt`]. For a tool
|
||||
//! whose job is to still open a course repository in five years, that tradeoff
|
||||
|
||||
+3
-1
@@ -13,8 +13,10 @@
|
||||
//! taxonomy levels, cognitive processes, error types, status, flags
|
||||
//! │
|
||||
//! course identity, policy, objectives, lectures, stimuli
|
||||
//! │ ├─ course::glossary terms, equations, values, misconceptions
|
||||
//! │ └─ course::fragment merges course.yaml, references.yaml,
|
||||
//! │ lectures/*.yaml, objectives/*.yaml
|
||||
//! │ glossary.yaml, lectures/*.yaml,
|
||||
//! │ objectives/*.yaml
|
||||
//! │
|
||||
//! item one question: stem, options, design intent, calibration
|
||||
//! │
|
||||
|
||||
@@ -267,3 +267,43 @@ fn policy_levels_and_blueprints_are_keyed_by_level() {
|
||||
reports(&issues, "policy.levels: `6` is not a level");
|
||||
reports(&issues, "policy.blueprints: `module` counts items at `7`");
|
||||
}
|
||||
|
||||
/// The checks the specification does not number: required text that is
|
||||
/// present but empty, and numbers outside the range the schema gives.
|
||||
#[test]
|
||||
fn empty_required_text_and_out_of_range_numbers_are_errors() {
|
||||
let mut c = course("required");
|
||||
c.terms.get_mut("term-gap").unwrap().definition = " ".into();
|
||||
c.equations.get_mut("eq-global-recurrence").unwrap().says = String::new();
|
||||
c.values.get_mut("val-per-decade").unwrap().say = String::new();
|
||||
c.misconceptions
|
||||
.get_mut("mis-only-matches-count")
|
||||
.unwrap()
|
||||
.correction = String::new();
|
||||
let lecture = c.lectures.get_mut("l09").unwrap();
|
||||
lecture.materials[0].url = String::new();
|
||||
lecture.readings[0].minutes = Some(0);
|
||||
c.policy.levels.get_mut("3").unwrap().name = String::new();
|
||||
c.policy.blueprints.get_mut("module").unwrap().minutes = Some(0.0);
|
||||
c.policy.exam_conditions.as_mut().unwrap().seconds_per_item = Some(0.0);
|
||||
|
||||
let issues = c.validate();
|
||||
reports(&issues, "term `term-gap`: empty definition");
|
||||
reports(&issues, "equation `eq-global-recurrence`: empty says");
|
||||
reports(&issues, "value `val-per-decade`: empty say");
|
||||
reports(
|
||||
&issues,
|
||||
"misconception `mis-only-matches-count`: empty correction",
|
||||
);
|
||||
reports(&issues, "lecture `l09`: material 1 has no url");
|
||||
reports(
|
||||
&issues,
|
||||
"lecture `l09` reading 1: minutes must be at least 1",
|
||||
);
|
||||
reports(&issues, "policy.levels: level 3 has an empty name");
|
||||
reports(&issues, "policy.blueprints: `module` gives no time");
|
||||
reports(
|
||||
&issues,
|
||||
"policy.exam_conditions: seconds_per_item must be above 0",
|
||||
);
|
||||
}
|
||||
Reference in new issue
Block a user