Compare commits

...
15 Commits
Author SHA1 Message Date
alexm abae39bd54 chore: relicense under Apache-2.0
Pipeline / check (pull_request) Failing after 12m11s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
Replace the Prosperity Public License 3.0.0 with the Apache License 2.0:
LICENSE.md, a new NOTICE.md, the Cargo license field, every SPDX header,
and the README's license and contributing sections.
2026-11-18 18:57:08 -05:00
alexm 18c48ac3e3 chore: UK -> US
Pipeline / check (pull_request) Failing after 12m31s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-11-16 13:10:36 -05:00
alexm 67528817f5 feat: clean
Pipeline / check (pull_request) Failing after 11m14s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-11-16 13:01:41 -05:00
alexmandClaude 7af1cd5b7e fix(lecture): leave prerequisites off a lecture's pages
Pipeline / check (pull_request) Successful in 13m38s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
A target taught in one lecture can name a later lecture that builds on
it, and every page printed whatever named the lecture, so L4.2 opened
with five objectives from L3.2, L3.5, and L4.1. A lecture's objectives,
readings, and page now show an entry only when the lecture lists its
objective in teaches, or when no earlier lecture covers it. The
lectures: link is unchanged, so assembly, coverage, and the reports'
review lists still read it.

Reading prose that cites an objective or target from another lecture
now renders as its number on that lecture's page (L3.5's T 1) rather
than the target's full text. The trailing (T ...) list stays this
page's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 13:00:24 -05:00
alexmandClaude b5cae6afac docs: schema 2.1 guide, codes, changelog, and report
A guide page for the four registries: where entries live, how targets
point at them, the other new fields, the brace-reference table, what
students never see, the lecture and glossary pages, every new error and
lint rule, and the migration from terms.yaml and values.yaml. The setup
guide points at it.

The README covers glossary.yaml, the nine schemas, the 2.1 checks, the
glossary lint family and the note severity, the new lecture commands and
report sections, migrate glossary, and schema version 2.1. Module docs
and the changelog follow.

docs/design/2.1/CODES.md maps the specification's E, W, and I codes to
coursebank's messages and lint codes, each with its test, and REPORT.md
records what changed, the two remaining schema differences, every choice
made where the specification left room, and suggested changes to it. A
new test covers the validation messages the specification does not
number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:31:01 -05:00
alexmandClaude 6c07122fc5 feat(migrate): glossary from 1.0 terms.yaml and values.yaml
`coursebank migrate glossary --terms terms.yaml --values values.yaml`
writes glossary.yaml. Ids gain their term- and val- prefixes, course
becomes term and meaning becomes definition, a value's term: gains the
prefix of the id it names, and first and lectures are dropped; each one
the targets now contradict is reported, as is each entry no target lists
yet and each field 2.1 has no place for.

Like the other migrations it copies lines, so comments, block scalars,
and flow mappings survive. Every entry is read back and compared with the
same conversion done on the parsed source, falling back to the serializer
when the copy differs. An existing glossary.yaml is added to, never
rewritten, and an entry the course already declares is skipped, so a
second run is a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:23:39 -05:00
alexmandClaude 50a1e083c1 feat(render): 2.1 lecture page, glossary page, and student reports
Derive each glossary entry's first lecture from teaching order and split
a lecture's terms into new and earlier. Add `coursebank lecture page`
(objectives with descriptions, targets with explanations and ceilings
named by policy.levels, terms, equations, numbers, common mistakes,
readings with minutes, materials) and `coursebank lecture glossary`.
Render brace references in student prose as links, values, quoted
beliefs, or target numbers. Student reports, Markdown and Typst, show
what meeting each target looks like, the correction for a registry
misconception, and group misses that share one.

A test renders every student-facing output from the example and checks
that no authoring, notes_private, evidence, conflicts, or status text
reaches it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:15:15 -05:00
alexmandClaude b8fdf4d6a8 feat(lint): glossary rules for terms, equations, values, misconceptions
Fourteen rules in a new `glossary` family (src/authoring/lint/glossary.rs):
unused entries, renamed or coined terms with no textbook name, coined
terms with no exam wording, circular definitions, coined terms in items,
unsourced values, value variants in prose or items, policy-forbidden
target wording, explanation terms a target does not list, recall values
printed in items, deep targets on equations with no implications, and
unresolved {lo-}/{t-} references in 2.0 prose fields. Two of them are
notes: entries defined away from the objective that first needs them,
and the count of proposed entries.

Adds Severity::Note. `coursebank lint` prints notes by default and does
not fail on them, so a course with proposed entries still passes a lint
gate. A distractor that names a registry misconception counts as designed
and as having student feedback.

On the example the glossary rules report exactly glossary-unused on
val-per-decade and glossary-proposed for two entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 12:02:50 -05:00
alexmandClaude 960cf6cbd3 feat(validate): 2.1 errors for the glossary registries
Validation now checks what 2.1 adds: registry ids carry their prefix and
are unique across all registries (E2101); a target's terms, equations,
values, and misconceptions name declared entries (E2102); equation
symbols resolve (E2103); values give a value or an ordered range and a
declared term (E2104); citation keys in the new registries resolve
(E2105); brace references in student-facing prose resolve (E2106);
`conflicts` and `target_text_forbid` patterns compile (E2107); and a bank
option's misconception_id is declared (E2108).

An unresolved {lo-...} or {t-...} in a field 2.0 already had (objective
and target text, reading notes) is left to the linter, because 2.0
accepted it and rendered it literally; the new reference kinds and the
new fields are checked everywhere.

Adds the `regex` crate, with default features off, for the course's own
patterns. Each error has a test built from the prototype's self-test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:58:53 -05:00
alexmandClaude 51a2fd235f feat(schema): 2.1 types, fragments, and JSON Schemas
Add the term, equation, value, and misconception registries
(src/model/course/glossary.rs) and the new optional fields on objectives,
targets, lectures, readings, bank options, and policy. Load registry
entries from objective files, glossary.yaml, and course.yaml, with one
definition site per id as for every other registry. Port every edit in
docs/design/2.1/build_schemas.py into the hand-written schema builders,
add glossary.schema.json, and test the generated schemas against the
targets; the only differences are two 2.0 spellings the code already had.

The 2.0 example fragments load and validate unchanged. The 2.1 example
bank gains `status: draft`, which every coursebank item requires.
tests/docs.rs skips docs/design/, which holds design records rather than
guide pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 11:55:18 -05:00
alexm 15175f64fd feat: include bonus
Pipeline / check (pull_request) Successful in 12m11s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-11-02 08:22:07 -05:00
alexmandClaude c8d6f4a5dd Name Scienting as the copyright holder and contributor
Pipeline / check (pull_request) Successful in 12m21s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
Replace 'Scientific Computing Studio' in the source headers, the Cargo.toml
authors, and the LICENSE contributor line. pixi.toml still has the old
authors line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014zNf4shD4zaCKLucZWPYL9
2026-10-20 00:21:19 -04:00
alexm 8f08ed6d91 chore: add more detailed readme
Pipeline / check (pull_request) Successful in 5m24s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-10-17 12:02:11 -04:00
alexm e9e5584ab9 fix: exam typst template
Pipeline / check (pull_request) Successful in 4m8s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-10-05 14:45:02 -04:00
alexm c6d6ee10b6 feat: improvement
Pipeline / check (pull_request) Successful in 5m0s
Pipeline / docs (pull_request) Skipped
Pipeline / nightly (pull_request) Skipped
Pipeline / release (pull_request) Skipped
2026-09-27 01:11:09 -04:00
89 changed files with 12156 additions and 500 deletions

No files matched your search

+29
View File
@@ -9,3 +9,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
### Added
- First public release.
- Student reports add bonus questions and curves to the score and show how the total adds up, so it matches Gradescope and Canvas. Class means, bands, and grade distributions use the same total.
- A zero-point Gradescope question with no option columns is recorded as a curve.
- `ingest` warns when a question dropped as removed still gives every student points on the platform.
- Schema 2.1, which is additive: a course that validated under 2.0 validates with no new errors. Rerun `coursebank schema` to pick up the new and changed JSON Schemas, including the new `glossary.schema.json`.
- Four registries beside objectives and targets: terms (`term-`), equations (`eq-`), values (`val-`), and misconceptions (`mis-`), each defined once in an objective file, a new `glossary.yaml`, or `course.yaml`. Ids are unique across every registry.
- Targets list the entries they turn on (`terms`, `equations`, `values`, `misconceptions`), carry a student-facing `explanation`, and keep the item-writing judgments in `authoring`.
- Objectives and lectures gain a `description`, lectures gain `materials`, readings gain `minutes`, and bank options gain `misconception_id`.
- `policy` gains `levels` (the course's names for levels 1 to 5), `exam_conditions`, `blueprints`, and `target_text_forbid`. No command reads `exam_conditions` or `blueprints` yet.
- Brace references such as `{term-gap-penalty}` in student-facing prose render as links, values, quoted beliefs, or target numbers.
- `authoring`, `notes_private`, `evidence`, `conflicts`, and `status` never reach a student page, report, or export.
- `validate` checks the registries: id prefixes, ids unique across registries, the entries each target lists, equation symbols, values, citation keys, brace references, that every regular expression compiles, and each option's `misconception_id`.
- Fourteen lint rules in a `glossary` family, from `glossary-unused` to `glossary-proposed`, and a `note` severity for the two that report conventions rather than problems.
- `lecture page ID` writes a whole lecture page: description, objectives and targets with explanations and named ceilings, new and earlier terms, equations, numbers to know, common mistakes, readings with minutes, and materials. `lecture glossary` writes the course glossary page.
- Student reports show what meeting each missed target looks like, the correction for a registry misconception, and group misses that share one.
- `migrate glossary --terms terms.yaml --values values.yaml` brings 1.0 side files into `glossary.yaml`, reports what it renamed and dropped, and is a no-op when run again.
### Changed
- `lint --min-severity` defaults to `note`, and `lint` fails only on findings above a note.
- Lecture pages (`lecture objectives`, `lecture readings`, `lecture page`) leave off prerequisites. An objective or target covered in an earlier lecture is no longer printed on a later lecture just because it lists that lecture in `lectures:`, unless the later lecture lists its objective in `teaches`. The `lectures:` link itself is unchanged, so assembly, coverage, and the reports' review lists still use it. Target numbers on those pages close up accordingly.
- Reading prose that cites an objective or target from another lecture renders as its number on that lecture's page, such as `L3.5's T 1`, instead of the full text of the target.
- The crate depends on `regex`, built without its optional features.
### Fixed
- A record with `points: 0` on a bonus question no longer erases the points the platform awarded.
- A curve whose position matches a recorded question number is no longer discarded at ingest.
- Gradescope files named with the question's title, such as `39_Curve.csv`, are now read. They used to be skipped without a word, which left titled questions and curves out of the totals. Any other CSV file in the directory now draws a warning, and two files for one question are refused. Re-ingest to pick this up.
- The assessment JSON Schema now accepts `dropped_as`, `dropped_before_printing`, `stem_digest`, and `learning_objectives` on a placement, all of which the loader already read. Rerun `coursebank schema` to pick this up.
+3 -2
View File
@@ -4,8 +4,8 @@ version = "0.0.0"
edition = "2024"
rust-version = "1.85"
description = "Author, assemble, administer, and analyze leveled course assessments from YAML item banks"
authors = ["Scientific Computing Studio <opensource@scient.ing>"]
license = "Prosperity-3.0.0"
authors = ["Scienting <opensource@scient.ing>"]
license = "Apache-2.0"
readme = "README.md"
publish = false
@@ -33,6 +33,7 @@ pbkdf2 = { version = "0.12", default-features = false, features = ["hmac"]}
sha2 = { version = "0.10"}
base64 = { version = "0.22"}
getrandom = { version = "0.2"}
regex = { version = "1.11", default-features = false, features = ["std", "unicode-perl", "unicode-case"] }
[profile.release]
opt-level = 3
+139 -32
View File
@@ -1,60 +1,167 @@
# The Prosperity Public License 3.0.0
# Apache License
Contributor: Scientific Computing Studio
_Version 2.0, January 2004_
_&lt;<http://www.apache.org/licenses/>&gt;_
Source Code: https://git.scient.ing/education/coursebank
## Terms and Conditions for use, reproduction, and distribution
## Purpose
### 1. Definitions
This license allows you to use and share this software for noncommercial purposes for free and to try this software for commercial purposes for thirty days.
“License” shall mean the terms and conditions for use, reproduction, and
distribution as defined by Sections 1 through 9 of this document.
## Agreement
“Licensor” shall mean the copyright owner or entity authorized by the copyright
owner that is granting the License.
In order to receive this license, you have to agree to its rules.
Those rules are both obligations under that agreement and conditions to your license.
Don't do anything with this software that triggers a rule you can't or won't follow.
“Legal Entity” shall mean the union of the acting entity and all other entities
that control, are controlled by, or are under common control with that entity.
For the purposes of this definition, “control” means **(i)** the power, direct or
indirect, to cause the direction or management of such entity, whether by
contract or otherwise, or **(ii)** ownership of fifty percent (50%) or more of the
outstanding shares, or **(iii)** beneficial ownership of such entity.
## Notices
“You” (or “Your”) shall mean an individual or Legal Entity exercising
permissions granted by this License.
Make sure everyone who gets a copy of any part of this software from you, with or without changes, also gets the text of this license and the contributor and source code lines above.
“Source” form shall mean the preferred form for making modifications, including
but not limited to software source code, documentation source, and configuration
files.
## Commercial Trial
“Object” form shall mean any form resulting from mechanical transformation or
translation of a Source form, including but not limited to compiled object code,
generated documentation, and conversions to other media types.
Limit your use of this software for commercial purposes to a thirty-day trial period.
If you use this software for work, your company gets one trial period for all personnel, not one trial per person.
“Work” shall mean the work of authorship, whether in Source or Object form, made
available under the License, as indicated by a copyright notice that is included
in or attached to the work (an example is provided in the Appendix below).
## Contributions Back
“Derivative Works” shall mean any work, whether in Source or Object form, that
is based on (or derived from) the Work and for which the editorial revisions,
annotations, elaborations, or other modifications represent, as a whole, an
original work of authorship. For the purposes of this License, Derivative Works
shall not include works that remain separable from, or merely link (or bind by
name) to the interfaces of, the Work and Derivative Works thereof.
Developing feedback, changes, or additions that you contribute back to the contributor on the terms of a standardized public software license such as [the Blue Oak Model License 1.0.0](https://blueoakcouncil.org/license/1.0.0), [the Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html), [the MIT license](https://spdx.org/licenses/MIT.html), or [the two-clause BSD license](https://spdx.org/licenses/BSD-2-Clause.html) doesn't count as use for a commercial purpose.
“Contribution” shall mean any work of authorship, including the original version
of the Work and any modifications or additions to that Work or Derivative Works
thereof, that is intentionally submitted to Licensor for inclusion in the Work
by the copyright owner or by an individual or Legal Entity authorized to submit
on behalf of the copyright owner. For the purposes of this definition,
“submitted” means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems, and
issue tracking systems that are managed by, or on behalf of, the Licensor for
the purpose of discussing and improving the Work, but excluding communication
that is conspicuously marked or otherwise designated in writing by the copyright
owner as “Not a Contribution.”
## Personal Uses
“Contributor” shall mean Licensor and any individual or Legal Entity on behalf
of whom a Contribution has been received by Licensor and subsequently
incorporated within the Work.
Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious observance, without any anticipated commercial application, doesn't count as use for a commercial purpose.
### 2. Grant of Copyright License
## Noncommercial Organizations
Subject to the terms and conditions of this License, each Contributor hereby
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
irrevocable copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the Work and such
Derivative Works in Source or Object form.
Use by any charitable organization, educational institution, public research organization, public safety or health organization, environmental protection organization, or government institution doesn't count as use for a commercial purpose regardless of the source of funding or obligations resulting from the funding.
### 3. Grant of Patent License
## Defense
Subject to the terms and conditions of this License, each Contributor hereby
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
irrevocable (except as stated in this section) patent license to make, have
made, use, offer to sell, sell, import, and otherwise transfer the Work, where
such license applies only to those patent claims licensable by such Contributor
that are necessarily infringed by their Contribution(s) alone or by combination
of their Contribution(s) with the Work to which such Contribution(s) was
submitted. If You institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work or a
Contribution incorporated within the Work constitutes direct or contributory
patent infringement, then any patent licenses granted to You under this License
for that Work shall terminate as of the date such litigation is filed.
Don't make any legal claim against anyone accusing this software, with or without changes, alone or with other technology, of infringing any patent.
### 4. Redistribution
## Copyright
You may reproduce and distribute copies of the Work or Derivative Works thereof
in any medium, with or without modifications, and in Source or Object form,
provided that You meet the following conditions:
The contributor licenses you to do everything with this software that would otherwise infringe their copyright in it.
* **(a)** You must give any other recipients of the Work or Derivative Works a copy of
this License; and
* **(b)** You must cause any modified files to carry prominent notices stating that You
changed the files; and
* **(c)** You must retain, in the Source form of any Derivative Works that You distribute,
all copyright, patent, trademark, and attribution notices from the Source form
of the Work, excluding those notices that do not pertain to any part of the
Derivative Works; and
* **(d)** If the Work includes a “NOTICE” text file as part of its distribution, then any
Derivative Works that You distribute must include a readable copy of the
attribution notices contained within such NOTICE file, excluding those notices
that do not pertain to any part of the Derivative Works, in at least one of the
following places: within a NOTICE text file distributed as part of the
Derivative Works; within the Source form or documentation, if provided along
with the Derivative Works; or, within a display generated by the Derivative
Works, if and wherever such third-party notices normally appear. The contents of
the NOTICE file are for informational purposes only and do not modify the
License. You may add Your own attribution notices within Derivative Works that
You distribute, alongside or as an addendum to the NOTICE text from the Work,
provided that such additional attribution notices cannot be construed as
modifying the License.
## Patent
You may add Your own copyright statement to Your modifications and may provide
additional or different license terms and conditions for use, reproduction, or
distribution of Your modifications, or for any such Derivative Works as a whole,
provided Your use, reproduction, and distribution of the Work otherwise complies
with the conditions stated in this License.
The contributor licenses you to do everything with this software that would otherwise infringe any patents they can license or become able to license.
### 5. Submission of Contributions
## Reliability
Unless You explicitly state otherwise, any Contribution intentionally submitted
for inclusion in the Work by You to the Licensor shall be under the terms and
conditions of this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify the terms of
any separate license agreement you may have executed with Licensor regarding
such Contributions.
The contributor can't revoke this license.
### 6. Trademarks
## Excuse
This License does not grant permission to use the trade names, trademarks,
service marks, or product names of the Licensor, except as required for
reasonable and customary use in describing the origin of the Work and
reproducing the content of the NOTICE file.
You're excused for unknowingly breaking [Notices](#notices) if you take all practical steps to comply within thirty days of learning you broke the rule.
### 7. Disclaimer of Warranty
## No Liability
Unless required by applicable law or agreed to in writing, Licensor provides the
Work (and each Contributor provides its Contributions) on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
including, without limitation, any warranties or conditions of TITLE,
NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
solely responsible for determining the appropriateness of using or
redistributing the Work and assume any risks associated with Your exercise of
permissions under this License.
***As far as the law allows, this software comes as is, without any warranty or condition, and the contributor won't be liable to anyone for any damages related to this software or this license, under any kind of legal claim.***
### 8. Limitation of Liability
In no event and under no legal theory, whether in tort (including negligence),
contract, or otherwise, unless required by applicable law (such as deliberate
and grossly negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special, incidental,
or consequential damages of any character arising as a result of this License or
out of the use or inability to use the Work (including but not limited to
damages for loss of goodwill, work stoppage, computer failure or malfunction, or
any and all other commercial damages or losses), even if such Contributor has
been advised of the possibility of such damages.
### 9. Accepting Warranty or Additional Liability
While redistributing the Work or Derivative Works thereof, You may choose to
offer, and charge a fee for, acceptance of support, warranty, indemnity, or
other liability obligations and/or rights consistent with this License. However,
in accepting such obligations, You may act only on Your own behalf and on Your
sole responsibility, not on behalf of any other Contributor, and only if You
agree to indemnify, defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason of your
accepting any such warranty or additional liability.
+8
View File
@@ -0,0 +1,8 @@
# coursebank
Copyright 2026 Scientific Computing Studio, LLC
This product includes software developed by Scienting ([scient.ing](https://scient.ing/)).
Licensed under the Apache License, Version 2.0.
See the `LICENSE.md` file for the full license text.
+1087 -16
View File
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Chooses the version string compiled into the binary.
+1 -1
View File
@@ -79,7 +79,7 @@ Keys *under* `extra` are yours and reach Typst verbatim, so they conventionally
### `extra` is the escape hatch
Anything under `extra` is carried through untouched and arrives as `extra` in the payload.
Tier colours, a font stack, a `show-solutions` flag, a watermark, column counts.
Tier colors, a font stack, a `show-solutions` flag, a watermark, column counts.
Put it there and read it in the template.
Nothing about appearance needs to be added to this crate.
+245
View File
@@ -0,0 +1,245 @@
# Terms, equations, values, and misconceptions
Schema 2.1 gives a course four registries beside its objectives and targets: the terms students meet, the equations they use, the numbers they are taught, and the mistakes they commonly make.
Each entry is defined once, keyed by id, and a target lists the entries it turns on.
The lecture page, the course glossary page, and student reports are built from those lists, so a definition you correct in one place is corrected everywhere it is shown.
A 2.0 course needs none of this and loads unchanged.
## The four registries
| Registry | Id prefix | One entry is |
|:--|:--|:--|
| `terms` | `term-` | a word or phrase, its one-sentence definition, and what the assigned texts call it |
| `equations` | `eq-` | a relation in LaTeX, what it says in words, its symbols, and what it implies |
| `values` | `val-` | a number as taught, the one wording every material uses for it, and its source |
| `misconceptions` | `mis-` | a belief students hold, in their voice, and its correction |
After the prefix, an id is lowercase words joined by hyphens, like every other id in the course.
Ids are unique across all six registries, objectives and targets included, so `{term-gap}` can only ever mean one thing.
Ids are also join keys: renaming one orphans every target, option, and brace reference that names it.
A term:
```yaml
terms:
term-gap-penalty:
term: gap penalty
symbol: 'd'
definition: 'The score subtracted for each position where one sequence is aligned against a {term-gap}.'
term-decade:
term: decade
definition: 'A factor of ten in a concentration ratio or an equilibrium constant.'
kind: coined
textbook:
- { term: order of magnitude, ref: KKW, locator: '§10.10' }
variants: [decades]
exam_wording: ten-fold
```
`kind` says whether a student can look the term up under this name: `standard` (the default), `renamed` (the course uses its own name for a standard idea), or `coined`.
A renamed or coined term should give the `textbook` name, and a coined one the `exam_wording` that items write instead of the label, because an exam should not test vocabulary only your lectures use.
An equation, a value, and a misconception:
```yaml
equations:
eq-global-recurrence:
name: the global alignment recurrence
tex: 'F(i,j) = \max\begin{cases} F(i-1,j-1) + s(x_i, y_j) \\ F(i-1,j) - d \\ F(i,j-1) - d \end{cases}'
says: >-
The best score for the first i letters of one sequence against the first j of the other is the best of
three moves into that cell.
symbols:
- { symbol: 'F(i,j)', meaning: 'the best score for aligning the two prefixes' }
- { symbol: 'd', term: term-gap-penalty }
implications:
- { form: direction, statement: 'Raising {term-gap-penalty} makes the diagonal win more cells.' }
assumes: ['Each gap position costs the same, whatever its neighbors.']
fails_when: ['Gaps come in long runs, which a linear penalty charges too much for.']
recall: true
values:
val-per-decade:
name: free energy per ten-fold concentration ratio at 37 °C
value: 6
units: kJ/mol
say: about 6 kJ/mol
exact: 5.9
source: { ref: KKW, locator: '§10.10' }
conflicts: ['5\.7\s*kJ', '5\.8\s*kJ']
recall: true
misconceptions:
mis-only-matches-count:
belief: 'An alignment score counts the matching positions, so the alignment with the most matches scores highest.'
correction: 'Every column counts: matches add, mismatches cost a little, and gaps cost more.'
```
An equation's `implications` are the things a question can ask about a relation without evaluating it, each in one of six forms: `direction`, `limit`, `scale`, `dominant_term`, `held_fixed`, or `misuse`.
A value's `say` is the wording the lecture page, slides, and items all use, and its `conflicts` are regular expressions for the wordings that must not appear anywhere students read.
`recall: true` means students are expected to know the value, so no item may print it.
Any entry may carry `status: proposed`, which a pipeline sets when it adds one; delete the line to approve it.
Proposed entries are checked like any other, and `coursebank lint` counts how many are waiting.
## Where entries live
An entry can be defined in three places:
| File | What goes there |
|:--|:--|
| `objectives/*.yaml` | the entries an objective's targets are the first in the course to need |
| `glossary.yaml` | cross-cutting entries, and whatever `migrate glossary` brought in, until each one is moved |
| `course.yaml` | anything, for a course that keeps everything in one file |
Lecture files and `references.yaml` cannot define entries.
Wherever an entry is defined, it is course-wide: a target in any later lecture lists it by id, and nothing redefines it.
The convention is the objective file whose targets first need the entry, and `lint` notes an entry that sits elsewhere (`glossary-defined-elsewhere`), without failing.
## Targets point at entries
A target lists the entries it turns on, and says in one student-facing paragraph what meeting it looks like:
```yaml
learning_targets:
t-align-gap-penalty-effect:
text: 'Predict how raising the gap penalty changes the optimal path through the matrix.'
objective: lo-alignment-recurrence
level_ceiling: 3
explanation: >-
A larger {term-gap-penalty} makes every horizontal or vertical step cost more, so the diagonal wins more
cells and the optimal path uses fewer gaps.
terms: [term-gap-penalty]
equations: [eq-global-recurrence]
misconceptions: [mis-only-matches-count]
authoring:
ceiling_item: 'Show the optimal path under d = 1 and ask which path wins under d = 4.'
supplied: ['the two sequences', 'the optimal path under the lower penalty']
relation_form: direction
```
The lists are what a lecture page draws its glossary sections from, and what decides each entry's first lecture: the first lecture, in teaching order, with a target that lists it.
Teaching order is the course's own: lectures in id order, objectives in each lecture's `teaches` order, and an objective's targets by ceiling and then id.
A term first needed in an earlier lecture appears on a later lecture's page as a link back, not as a new definition.
`authoring` holds the item-writing judgments that settle what a question at the target's ceiling looks like: `ceiling_item`, `supplied` data, `constraints`, `number_work`, `examples`, `relation_form`, and the evidence for the ceiling.
It is for the item pipelines and for you; students never see it.
## Other fields 2.1 adds
| On | Field | What it does |
|:--|:--|:--|
| objective | `description` | a short student-facing paragraph under the objective on the lecture page |
| lecture | `description` | the paragraph that opens the lecture page |
| lecture | `materials` | slides, briefs, notebooks, and other links, each `{kind, url, title}` |
| reading | `minutes` | the reading's length, summed at the top of the readings section |
| bank option | `misconception_id` | the registry misconception the distractor encodes |
| policy | `exam_conditions` | closed book or not, calculator, how equations are given, formats, seconds per item |
| policy | `blueprints` | named level counts, bonus counts, and minutes |
| policy | `levels` | the course's names for levels 1 to 5, used to label ceilings on the lecture page |
| policy | `target_text_forbid` | regular expressions that objective and target text must not match |
An option's own `misconception` text, when it has one, still wins for that item.
An option with only a `misconception_id` counts as designed, and its feedback is the registry's correction.
`exam_conditions` and `blueprints` are stored and validated, but no command reads them yet.
## Brace references
Student-facing prose can name an entry in braces, and every page that shows the prose renders the reference:
| Reference | Renders as |
|:--|:--|
| `{term-gap-penalty}` | the term, linked to its definition |
| `{eq-global-recurrence}` | the equation's name, linked to it |
| `{val-per-decade}` | the value's `say`, such as about 6 kJ/mol |
| `{mis-only-matches-count}` | the belief, in quotation marks |
| `{t-align-fill-matrix}` | `T 3`, the target's number on the page, or the target's text where it has no number |
| `{lo-alignment-recurrence}` | the objective's text |
On a Quarto lecture page, a link points at the definition on the same page when it is there (a term new in that lecture, or an equation its targets use).
Every other link points at the course glossary page: `glossary.html`, unless `--glossary-href` says otherwise.
Braces that are not a prefixed id, such as `10^{59}` or `\text{final}`, are left alone.
The fields that take references are the student-facing ones: definitions, `says`, implication statements, `assumes`, `fails_when`, corrections, objective and lecture descriptions, target explanations, and a reading's `summary`, `focus`, and `skip`.
## What students never see
`authoring`, `notes_private`, `evidence`, `conflicts`, and `status` are instructor-facing.
No lecture page, glossary page, student report, or student export (Canvas, the practice worksheet and its solutions, the site) prints them, and a test renders the example course through every one of those and checks.
A proposed entry is shown to students like an approved one, without saying it is proposed.
## Pages
```console
$ coursebank lecture page l09 --out site/l09.qmd
$ coursebank lecture glossary --out site/glossary.qmd
```
`lecture page` writes, in order: the lecture's description; each objective with its description and its targets, each with its explanation and its ceiling named by `policy.levels`; the new terms; earlier terms, linked; the equations; the numbers to know; common mistakes; readings, with minutes; and materials.
A lecture's pages show what it teaches, not what it builds on: an objective or target already covered in an earlier lecture is left off, even when it names this lecture in `lectures:`, unless the lecture lists its objective in `teaches`.
`lecture glossary` lists every term alphabetically with its definition and first lecture, then every equation and value.
Both write Quarto Markdown by default (numbered targets as `(@)` example lists, anchors as `{#glossary-term-...}`), or plain Markdown with `--style plain`.
Student reports gain two things.
Each missed target shows what meeting it looks like, from its `explanation`.
A chosen option with a `misconception_id` shows the misconception and its correction, and misses that share one across questions are grouped under it, so a student who made the same mistake three times reads about it once.
## Checks
`coursebank validate` reports these as errors, each prefixed with the file that defines the entry:
| Problem | Message |
|:--|:--|
| an id lacks its registry's prefix | ``term `x`: a term id is `term-` followed by lowercase words`` |
| an id is in two registries, or two files | `` `x` is declared as both a ``, or `is defined in two places` |
| a target lists an undeclared entry | ``target `t`: `terms` names `x`, which is not a declared term`` |
| an equation symbol names an undeclared term, or has neither `term` nor `meaning` | ``equation `eq-x`: symbol `d` `` |
| a value has neither `value` nor `range`, its range runs backwards, or its `term` is undeclared | ``value `val-x`: `` |
| a `source`, `textbook`, or `evidence` cites a key not in `references` | ``…: source cites `KEY`, which is not in references`` |
| a brace reference does not resolve | ``…: description refers to `{term-x}`, which is not a declared term`` |
| a `conflicts` or `target_text_forbid` pattern does not compile | ``value `val-x`: conflicts pattern `…` does not compile`` |
| an option's `misconception_id` is undeclared | ``option o-x: misconception_id `mis-x` is not a declared misconception`` |
An unresolved `{lo-...}` or `{t-...}` in a reading's `summary`, `focus`, or `skip` stays a lint warning (`glossary-unresolved-reference`) rather than an error, because 2.0 accepted it.
`coursebank lint` adds fourteen rules in a `glossary` family:
| Code | Severity | Flags |
|:--|:--|:--|
| `glossary-unused` | low | an entry no target, equation symbol, value, or option needs |
| `glossary-no-textbook-name` | medium | a renamed or coined term with no textbook name |
| `glossary-no-exam-wording` | low | a coined term with no `exam_wording` |
| `glossary-circular-definition` | low | a definition that uses the term it defines |
| `glossary-coined-in-item` | medium | a coined term in an item's stem or options |
| `glossary-value-unsourced` | low | a value with no `source` |
| `glossary-value-conflict` | medium | a `conflicts` pattern that matches student-facing text or an item |
| `glossary-forbidden-target-wording` | medium | objective or target text that matches `target_text_forbid` |
| `glossary-unlisted-term` | low | an explanation that names a term its target neither lists nor inherits from a prerequisite |
| `glossary-recall-value-printed` | medium | an item that prints a `recall: true` value |
| `glossary-equation-no-implications` | low | a target at ceiling 3 or above that turns on an equation with no implications |
| `glossary-unresolved-reference` | medium | an old prose field that refers to an undeclared objective or target |
| `glossary-defined-elsewhere` | note | an entry outside the objective file whose targets first need it |
| `glossary-proposed` | note | how many entries are waiting for approval |
A note is shown by default and never makes `lint` fail, so a course with proposed entries still passes a lint gate.
`--min-severity low` hides notes.
Regular expressions use Rust's `regex` syntax, which has no look-around and no backreferences.
`target_text_forbid` matches case-insensitively; `conflicts` matches as written.
## From `terms.yaml` and `values.yaml`
A course that kept its terms and numbers in 1.0 side files brings them in with one command:
```console
$ coursebank migrate glossary --terms terms.yaml --values values.yaml --dry-run
$ coursebank migrate glossary --terms terms.yaml --values values.yaml
```
Ids gain their prefix (`decade` becomes `term-decade`), `course` becomes `term`, `meaning` becomes `definition`, and a value's `term:` gains the prefix of the id it names.
`first` and `lectures` are dropped, because 2.1 derives them from the targets that list an entry; the command reports each one that disagrees with what the targets now say, each entry no target lists yet, and each field 2.1 has no place for.
The entries go into `glossary.yaml`, added at the end of its sections if it exists, with comments and quoting copied from the old files.
The old files are left alone, and an entry the course already declares is skipped, so a second run changes nothing.
Linking targets to the migrated entries, and moving each entry into the objective file that first needs it, is then yours or a pipeline's to do; `lint` lists what is left (`glossary-unused`, `glossary-defined-elsewhere`).
+9 -1
View File
@@ -56,7 +56,15 @@ See [`typst_export`](crate::guide::typst_export).
**Gradescope.**
`coursebank ingest gradescope grading/exam-2/ --assessment exam-2`.
Point it at the directory holding the per-question CSVs.
Point it at the directory holding the per-question CSVs, named as Gradescope names them: `7.csv`, or `39_Curve.csv` for a question with a title.
Give every form its own directory, including its own copy of a curve.
**Bonus questions and curves.**
Leave them in the Gradescope export, worth zero points.
A zero-point question with option columns is a bonus question; give it `bonus: true` in the record.
A zero-point question with no option columns, such as a final `Curve` question worth 3.6/0, stays out of the record and is treated as a curve.
Reports add both to the total without raising the maximum, so their percentage matches Gradescope's and can pass 100%.
If you drop a question by giving everyone the points, set `dropped_as: full_credit` on its placement, or reports will take it out of the total that Gradescope still counts.
**Canvas.**
`coursebank ingest canvas export.csv --assessment exam-2`.
+7
View File
@@ -208,6 +208,12 @@ stimuli:
Items reference it with `stimulus: s-dock-poses`.
Declaring it here rather than pasting it into four items means a correction to the table happens once.
### Terms, equations, values, and misconceptions
Four more registries hold what students study from besides the objectives: the terms they meet (`term-`), the equations they use (`eq-`), the numbers they are taught (`val-`), and the mistakes they commonly make (`mis-`).
Each is defined once, in the objective file whose targets first need it or in a `glossary.yaml` beside `course.yaml`, and targets list the entries they turn on.
They are optional, and [`glossary`](crate::guide::glossary) covers them, the lecture pages built from them, and the migration from 1.0 `terms.yaml` and `values.yaml`.
## Check it
```console
@@ -229,6 +235,7 @@ Exit codes are meaningful.
## Where to go next
[`glossary`](crate::guide::glossary) adds terms, equations, values, and misconceptions, and builds lecture pages from them.
[`authoring`](crate::guide::authoring) writes the first bank.
[`first_exam`](crate::guide::first_exam) takes an exam from blueprint to student report.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Psychometrics: what the responses say about the items and about the students.
+107 -43
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Writing statistics back onto the items.
@@ -28,18 +28,19 @@
use std::collections::{BTreeMap, BTreeSet};
use std::path::PathBuf;
use crate::SCHEMA_VERSION;
use crate::assessment::AssessmentFile;
use crate::bank::BankFile;
use crate::calibration::{CalibrationFile, Measurement, MeasurementFile, MeasurementMeta};
use crate::catalog::Catalog;
use crate::classical::{self, Analysis, ItemAnalysis, Thresholds};
use crate::date::Date;
use crate::error::{Error, Result};
use crate::irt::{self, Fit};
use crate::item::{Calibration, IrtParams, Item, OptionStat, VariantCalibration};
use crate::layout::Layout;
use crate::responses::ResponseSet;
use crate::store::Store;
use crate::taxonomy::Flag;
use crate::yaml;
/// What calibration would change about one item.
#[derive(Debug, Clone)]
@@ -615,60 +616,123 @@ fn diff_calibration(previous: Option<&Calibration>, next: &Calibration) -> Vec<S
out
}
/// Applies a plan, rewriting the affected bank files.
/// Applies a plan, rewriting the calibration store.
///
/// Files are rewritten one at a time and each is re-read before editing, so a plan
/// built against a bank that has since changed on disk fails loudly rather than
/// clobbering the newer version.
/// One file, `analysis/calibration.yaml`, and never a bank. A bank is reviewed
/// for what it asks; its history should be a record of wording decisions, not
/// of every grading run. The store is re-read immediately before editing, so a
/// plan built against a store that has since changed on disk fails loudly
/// rather than clobbering the newer version.
///
/// # Arguments
///
/// * `layout` - the course layout, for where the store lives.
/// * `plan` - the plan to apply.
///
/// # Returns
///
/// The bank files rewritten.
/// The file written.
///
/// # Errors
///
/// Returns [`Error::Unresolved`] when an item in the plan is no longer in its bank,
/// and [`Error::Io`] on a write failure.
pub fn apply(plan: &Plan) -> Result<Vec<PathBuf>> {
// Group by file so each is read and written once.
let mut by_file: BTreeMap<&PathBuf, Vec<&Change>> = BTreeMap::new();
/// Returns [`Error::Io`] on a write failure and [`Error::Yaml`] if the existing
/// store does not parse.
pub fn apply(layout: &Layout, plan: &Plan) -> Result<PathBuf> {
let path = layout.calibration_file();
let mut store = CalibrationFile::load(&path)?;
for change in &plan.changes {
by_file.entry(&change.path).or_default().push(change);
store
.items
.insert(change.uid.clone(), change.calibration.clone());
}
let mut written = Vec::new();
for (path, changes) in by_file {
let mut bank: BankFile = yaml::read(path)?;
for change in changes {
// The uid is `bank::item`; match on the item part.
let item_id = change
.uid
.split_once("::")
.map(|(_, id)| id)
.unwrap_or(&change.uid);
let target = bank.items.iter_mut().find(|i| i.id == item_id);
match target {
Some(item) => item.calibration = Some(change.calibration.clone()),
None => {
return Err(Error::Unresolved {
kind: "item",
id: change.uid.clone(),
context: Some(format!(
"{} — the bank changed since the plan was built; re-run calibration",
path.display()
)),
});
}
}
}
yaml::write(path, &bank)?;
written.push(path.clone());
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).map_err(|e| Error::io(parent, e))?;
}
Ok(written)
store.save(&path)?;
Ok(path)
}
/// Records what one administration measured, as a file that is never rewritten.
///
/// The audit trail under the pooled store: these are the numbers one exam
/// produced, on a day, under a named model. Keeping them means the history
/// survives losing `data/`, which is ignored by git precisely because every row
/// of it carries a student.
///
/// # Arguments
///
/// * `layout` - the course layout.
/// * `administration` - the administration id, which names the file.
/// * `analysis` - the classical analysis of that administration.
/// * `catalog` - the loaded course, for the digests each item was measured
/// against.
/// * `record` - the assessment record, for the term and date.
///
/// # Returns
///
/// The two files written: one row per question, one row per question and
/// option.
///
/// # Errors
///
/// Returns [`Error::Usage`] when a record for this administration already
/// exists, since an administration happened once.
pub fn record_measurements(
layout: &Layout,
administration: &str,
analysis: &Analysis,
catalog: &Catalog,
record: Option<&AssessmentFile>,
) -> Result<Vec<PathBuf>> {
let mut items = Vec::new();
for item in &analysis.items {
let Some(uid) = &item.item_ref else { continue };
let entry = catalog.get(uid);
items.push(Measurement {
item: uid.clone(),
number: item.number,
variant: item.variant.clone(),
stem_digest: entry.map(|e| e.item.stem_digest()),
n: item.n,
p_value: Some(round4(item.p_value)),
point_biserial: item.point_biserial.map(round4),
discrimination_index: item.discrimination_index.map(round4),
key: item.key.clone(),
option_stats: item
.options
.iter()
.map(|(id, o)| (id.clone(), o.to_option_stat()))
.collect(),
irt: None,
flags: item.flags.clone(),
});
}
let file = MeasurementFile {
schema_version: SCHEMA_VERSION.to_string(),
administration: MeasurementMeta {
id: administration.to_string(),
assessment: record
.map(|r| r.assessment.id.clone())
.unwrap_or_else(|| administration.to_string()),
term: record.and_then(|r| r.assessment.term.clone()),
date: record.and_then(|r| r.assessment.date),
forms: record
.map(|r| r.forms.iter().map(|f| f.id.clone()).collect())
.unwrap_or_default(),
// The cohort, taken as the largest per-item n: a student who
// skipped question 7 still sat the exam.
n_examinees: analysis.items.iter().map(|i| i.n).max().unwrap_or(0),
model: None,
generated: Some(Date::today()),
coursebank: Some(crate::VERSION.to_string()),
},
items,
};
file.write_csv(&layout.measurements())
}
/// Builds a plan for a single administration, from an in-memory analysis.
+3 -3
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Classical item analysis.
@@ -191,7 +191,7 @@ pub struct Prediction {
/// The discrimination band the author expected, as `(low, high)`.
pub expected_band: Option<(f64, f64)>,
/// Whether the expectation rests on a calibration with enough examinees
/// behind it, rather than on the author's judgement alone.
/// behind it, rather than on the author's judgment alone.
pub calibrated: bool,
/// Signed difficulty error, observed minus expected. Positive means the item
/// was easier than predicted.
+158 -7
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! What to tell a student, and what to tell yourself.
@@ -106,7 +106,7 @@ pub struct Options {
/// Whether to include the worked solution for a missed question.
///
/// Off by default. [`crate::item::Solution::explanation`] is the derivation,
/// the estimate, and the argument for the key over its neighbours: the body of
/// the estimate, and the argument for the key over its neighbors: the body of
/// the solutions document. Turning this on hands that to every student who
/// missed the question, which is the right call for a question you will not
/// use again and the wrong one for a bank you reuse each term.
@@ -181,6 +181,10 @@ pub struct StudentDiagnostic {
/// One row per question, with no question in it.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub questions: Vec<QuestionRow>,
/// Registry mistakes behind more than one missed question, so the report
/// can group those misses under the one thing to fix.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub mistakes: Vec<MistakeGroup>,
/// Questions thrown out, in number order.
///
/// Reported separately from the question rows so the document can say once,
@@ -206,16 +210,33 @@ pub struct Score {
pub points: f64,
/// Points available on scored items.
pub points_possible: f64,
/// Percentage on scored items.
/// Every point earned, bonus and curve included: the total on the platform.
pub total_points: f64,
/// `total_points` as a percentage of `points_possible`, the same percentage
/// the platform shows. It can pass 100.
pub percent: f64,
/// Bonus points earned.
pub bonus_points: f64,
/// Points a curve added.
pub curve_points: f64,
/// The bonus questions on the exam, so the total can be added up by hand.
pub bonus_questions: Vec<BonusQuestion>,
/// Items answered correctly.
pub correct: usize,
/// Scored items administered.
pub n_items: usize,
}
/// One bonus question's contribution to a student's total.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "kebab-case")]
pub struct BonusQuestion {
/// The recorded question number.
pub number: u32,
/// Points the student earned on it.
pub points: f64,
}
/// Where a score sits in the class.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "kebab-case")]
@@ -320,6 +341,41 @@ pub struct Measured {
pub objective: Option<String>,
/// The target the question was written against.
pub target: String,
/// The target's explanation, with references rendered as words: what
/// meeting it looks like, which is what to practice toward.
#[serde(skip_serializing_if = "Option::is_none")]
pub explanation: Option<String>,
}
/// A named mistake behind a student's answer, from the course's misconception
/// registry. Student-facing: the belief and its correction are written for the
/// student.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "kebab-case")]
pub struct Mistake {
/// The misconception id.
pub id: String,
/// The wrong idea as a student holds it. Absent when the chosen option
/// carries its own misconception text, which overrides the registry's for
/// that item and already reaches the student as feedback.
#[serde(skip_serializing_if = "Option::is_none")]
pub belief: Option<String>,
/// Why it fails and what is true instead.
pub correction: String,
}
/// A mistake that more than one missed question drew on.
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "kebab-case")]
pub struct MistakeGroup {
/// The misconception id.
pub id: String,
/// The registry's belief.
pub belief: String,
/// The registry's correction.
pub correction: String,
/// The questions, by recorded number.
pub questions: Vec<u32>,
}
/// One question, described without being reproduced.
@@ -340,7 +396,7 @@ pub struct QuestionRow {
///
/// Both, because each answers a different question a student has in front of
/// a missed item. The target says what this question actually asked of them,
/// which is the specific thing to go and practise. The objective says which
/// which is the specific thing to go and practice. The objective says which
/// row of the table above the mark landed in, which is how they tell whether
/// one slip cost them a claim or whether it was one of several. Printing the
/// target alone left them unable to connect the note to the table; printing
@@ -380,6 +436,10 @@ pub struct QuestionRow {
/// The worked solution, when [`Options::solutions`] is on.
#[serde(skip_serializing_if = "Option::is_none")]
pub worked: Option<String>,
/// The registry misconception the chosen option encodes, with its
/// correction, when [`Options::feedback`] is on.
#[serde(skip_serializing_if = "Option::is_none")]
pub mistake: Option<Mistake>,
/// Where the material was taught.
#[serde(skip_serializing_if = "Vec::is_empty")]
pub taught_in: Vec<String>,
@@ -406,7 +466,7 @@ pub struct DroppedQuestion {
pub struct ItemReading {
/// A short citation, e.g. `KKW §2.1`.
pub citation: String,
/// The work's full title, for a student who does not recognise the label.
/// The work's full title, for a student who does not recognize the label.
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
/// A link, when the citation resolves to one.
@@ -585,14 +645,26 @@ pub fn student(
.map(|a| a.items.iter().map(|i| (i.number, i.p_value)).collect())
.unwrap_or_default();
// A curve is on the total but was never a question on the paper, so a box for
// it would send the student looking for something that was not there.
let questions = if opts.questions {
rows.iter()
.filter(|row| !row.is_curve())
.map(|row| question_row(row, catalog, &class_rates, opts))
.collect()
} else {
Vec::new()
};
let bonus_questions = rows
.iter()
.filter(|r| r.bonus && !r.dropped && !r.is_curve())
.map(|r| BonusQuestion {
number: r.item_number,
points: r.score,
})
.collect();
let mut dropped_questions: Vec<DroppedQuestion> = rows
.iter()
.filter(|r| r.dropped)
@@ -630,8 +702,11 @@ pub fn student(
score: Score {
points: summary.points,
points_possible: summary.points_possible,
total_points: summary.total_points,
percent: summary.percent,
bonus_points: summary.bonus_points,
curve_points: summary.curve_points,
bonus_questions,
correct: summary.correct,
n_items: summary.n_items,
},
@@ -646,6 +721,7 @@ pub fn student(
objectives,
strengths,
focus,
mistakes: shared_mistakes(&questions, course),
questions,
dropped_questions,
review_lectures,
@@ -674,6 +750,7 @@ fn question_row(
let mut hint = None;
let mut misconception = None;
let mut worked = None;
let mut mistake = None;
let mut taught_in = Vec::new();
let mut review = Vec::new();
@@ -682,6 +759,20 @@ fn question_row(
if missed {
if let Some(letter) = row.chosen().first() {
if let Some(choice) = entry.item.option(letter) {
if opts.feedback {
let course = &catalog.course;
mistake = choice.misconception_id.as_deref().and_then(|id| {
let registered = course.misconceptions.get(id)?;
Some(Mistake {
id: id.to_string(),
belief: choice
.misconception
.is_none()
.then(|| course.plain_refs(&registered.belief)),
correction: course.plain_refs(&registered.correction),
})
});
}
if opts.feedback {
// `student_text` falls back to `explanation`, which is
// written for a grader and often names the right
@@ -751,6 +842,11 @@ fn question_row(
// objective is left out.
objective: (objective != id).then(|| course.text_for(objective)),
target: course.text_for(id),
explanation: course
.learning_targets
.get(id)
.and_then(|t| t.explanation.as_deref())
.map(|e| course.plain_refs(e)),
}
})
.collect()
@@ -772,11 +868,47 @@ fn question_row(
hint,
misconception,
worked,
mistake,
taught_in,
review,
}
}
/// The registry mistakes that more than one of a student's missed questions
/// drew on, so a report can name the thing to fix once.
///
/// # Arguments
///
/// * `questions` - the student's question rows.
/// * `course` - the course, for the registry's wording.
///
/// # Returns
///
/// One group per shared misconception, in order of its first question.
pub fn shared_mistakes(questions: &[QuestionRow], course: &CourseFile) -> Vec<MistakeGroup> {
let mut groups: Vec<MistakeGroup> = Vec::new();
for q in questions {
let Some(mistake) = &q.mistake else {
continue;
};
if let Some(group) = groups.iter_mut().find(|g| g.id == mistake.id) {
group.questions.push(q.number);
continue;
}
let Some(entry) = course.misconceptions.get(&mistake.id) else {
continue;
};
groups.push(MistakeGroup {
id: mistake.id.clone(),
belief: course.plain_refs(&entry.belief),
correction: course.plain_refs(&entry.correction),
questions: vec![q.number],
});
}
groups.retain(|g| g.questions.len() > 1);
groups
}
/// Resolves an item's `review` citations against the course reference registry.
///
/// # Arguments
@@ -972,6 +1104,10 @@ fn readings_for(course: &CourseFile, objective: &str, limit: usize) -> Vec<Study
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "kebab-case")]
pub struct CohortDiagnostic {
/// Mean bonus points per student, which the distribution includes.
pub mean_bonus: f64,
/// Mean curve points per student, which the distribution includes.
pub mean_curve: f64,
/// How many students sat it.
pub n_students: usize,
/// How many scored items.
@@ -1288,7 +1424,7 @@ pub struct GradeRow {
/// The attainment word, when the scale records one.
#[serde(skip_serializing_if = "Option::is_none")]
pub attainment: Option<String>,
/// The colour group, so A, A- and A+ can be tinted together.
/// The color group, so A, A- and A+ can be tinted together.
pub group: String,
/// How many students landed in the band.
pub count: usize,
@@ -1758,6 +1894,8 @@ pub fn cohort(
n_students: cohort.students.len(),
n_items: analysis.reliability.n_items,
distribution: distribution(&percents),
mean_bonus: mean_of(cohort, |s| s.bonus_points),
mean_curve: mean_of(cohort, |s| s.curve_points),
grades,
lectures,
dropped_questions,
@@ -2419,6 +2557,15 @@ fn form_rows(set: &ResponseSet, cohort: &Cohort) -> Vec<FormRow> {
.collect()
}
/// The class mean of one per-student number.
fn mean_of(cohort: &Cohort, value: impl Fn(&StudentSummary) -> f64) -> f64 {
if cohort.students.is_empty() {
0.0
} else {
cohort.students.iter().map(value).sum::<f64>() / cohort.students.len() as f64
}
}
/// Bins a set of percentages into a distribution.
fn distribution(percents: &[f64]) -> Distribution {
let mut sorted: Vec<f64> = percents.to_vec();
@@ -2504,8 +2651,11 @@ mod tests {
score: Score {
points: 1.0,
points_possible: 2.0,
total_points: 1.0,
percent: 50.0,
bonus_points: 0.0,
curve_points: 0.0,
bonus_questions: Vec::new(),
correct: 1,
n_items: 2,
},
@@ -2515,6 +2665,7 @@ mod tests {
strengths: Vec::new(),
focus: Vec::new(),
questions: Vec::new(),
mistakes: Vec::new(),
dropped_questions: Vec::new(),
review_lectures: Vec::new(),
study: Vec::new(),
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Item response theory: marginal maximum likelihood estimation by EM.
+19 -4
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Turning responses into something you can say to a student.
@@ -184,6 +184,9 @@ pub struct MissedItem {
pub learning_targets: Vec<String>,
/// The misconception the chosen distractor was written to detect.
pub misconception: Option<String>,
/// The registry misconception the chosen distractor encodes, when it names
/// one. Misses that share it are grouped in the report.
pub misconception_id: Option<String>,
/// Feedback written for a student who chose that option.
pub feedback: Option<String>,
/// Where to go back to: lecture titles and slide numbers.
@@ -203,10 +206,15 @@ pub struct StudentSummary {
pub points: f64,
/// Points available on scored items.
pub points_possible: f64,
/// Percentage on scored items.
/// Every point earned, bonus and curve included: the total on the platform.
pub total_points: f64,
/// `total_points` as a percentage of `points_possible`, the same percentage
/// the platform shows. It can pass 100.
pub percent: f64,
/// Bonus points earned.
pub bonus_points: f64,
/// Points a curve added.
pub curve_points: f64,
/// Items answered correctly.
pub correct: usize,
/// Items administered.
@@ -355,7 +363,7 @@ pub fn summarize(
let percents: Vec<f64> = students
.iter()
.map(|key| {
let earned = set.scored_total(key);
let earned = set.points_of_record(key);
let possible = set.points_available();
if possible > 0.0 {
100.0 * earned / possible
@@ -513,8 +521,10 @@ pub fn summarize(
sid: rows.first().and_then(|r| r.sid.clone()),
points,
points_possible: possible,
total_points: set.points_of_record(key),
percent,
bonus_points: set.bonus_total(key),
curve_points: set.curve_total(key),
correct,
n_items,
band: band_for(percent, &percents),
@@ -668,6 +678,7 @@ fn missed_items(
continue;
}
let mut misconception = None;
let mut misconception_id = None;
let mut feedback = None;
let mut study = Vec::new();
@@ -678,6 +689,7 @@ fn missed_items(
if let Some(letter) = r.chosen().first() {
if let Some(choice) = entry.item.option(letter) {
misconception = choice.misconception.clone();
misconception_id = choice.misconception_id.clone();
feedback = choice.student_text().map(|s| s.to_string());
}
}
@@ -709,6 +721,7 @@ fn missed_items(
level: r.level,
learning_targets: r.learning_targets.clone(),
misconception,
misconception_id,
feedback,
study,
});
@@ -1389,8 +1402,10 @@ learning_objectives:
sid: None,
points: 0.0,
points_possible: 0.0,
total_points: 0.0,
percent: 0.0,
bonus_points: 0.0,
curve_points: 0.0,
correct: 0,
n_items: 0,
band: String::new(),
+5 -5
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Support for writing items and building assessments from them.
@@ -7,8 +7,8 @@
//! The three modules here are what you interact with before an exam exists, and
//! they divide by how much authority each one has.
//!
//! [`lint`] has none. Its 26 rules are advice with stable codes, every one
//! silenceable with `--ignore`. It is deliberately separate from the validation in
//! [`lint`] has none. Its 44 rules are advice with stable codes, every one
//! silenceable with `--ignore`, and the two that report notes never fail. It is deliberately separate from the validation in
//! [`crate::model::bank`], which enforces what must be true and fails. Conflating
//! the two produces a tool that either blocks you on style or lets real errors
//! through.
@@ -16,7 +16,7 @@
//! [`select`] draws an assessment from a blueprint. It places objective minimums
//! before level quotas, because a coverage requirement is the constraint most
//! likely to become unsatisfiable, and it prefers least-recently-used items so a
//! bank rotates rather than converging on your favourites.
//! bank rotates rather than converging on your favorites.
//!
//! [`jsonschema`] emits JSON Schema so an editor autocompletes the YAML. That is
//! a better authoring experience than any validator: catching `cognitve_process`
File diff suppressed because it is too large. Load diff
+147 -5
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Authoring-quality checks: is this item *well written*?
@@ -35,6 +35,10 @@ use crate::course::CourseFile;
use crate::item::Item;
use crate::taxonomy::{Discrimination, Format, Level, Status};
mod glossary;
pub use glossary::lint_glossary;
/// A lint rule.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum Rule {
@@ -107,6 +111,36 @@ pub enum Rule {
DiscriminationMissed,
/// Two items in the course have nearly the same stem.
DuplicateStem,
// --- glossary ---
/// A glossary entry nothing needs.
GlossaryUnused,
/// A renamed or coined term with no textbook name.
NoTextbookName,
/// A coined term with no exam wording.
NoExamWording,
/// A definition that uses the term it defines.
CircularDefinition,
/// A coined term in an item's stem or option.
CoinedTermInItem,
/// A taught value with no source.
UnsourcedValue,
/// A variant of a taught value in student-facing text.
ValueConflict,
/// Objective or target text the course's policy forbids.
ForbiddenTargetWording,
/// A term an explanation uses that its target does not list.
UnlistedTerm,
/// An item that prints a value students must recall.
RecallValuePrinted,
/// A ceiling-3 target turning on an equation with no implications.
EquationWithoutImplications,
/// A 2.0 prose field refers to an objective or target that does not exist.
UnresolvedReference,
/// A glossary entry defined away from the objective that first needs it.
DefinedElsewhere,
/// Glossary entries waiting for the instructor's approval.
ProposedEntries,
}
impl Rule {
@@ -114,7 +148,7 @@ impl Rule {
///
/// Used by `--list-rules`, and by the test that keeps this list in step with
/// the enum.
pub const ALL: [Rule; 30] = [
pub const ALL: [Rule; 44] = [
Rule::KeyIsLongest,
Rule::UnevenOptionLength,
Rule::WordRepeatCue,
@@ -145,6 +179,20 @@ impl Rule {
Rule::DifficultyMissed,
Rule::DiscriminationMissed,
Rule::DuplicateStem,
Rule::GlossaryUnused,
Rule::NoTextbookName,
Rule::NoExamWording,
Rule::CircularDefinition,
Rule::CoinedTermInItem,
Rule::UnsourcedValue,
Rule::ValueConflict,
Rule::ForbiddenTargetWording,
Rule::UnlistedTerm,
Rule::RecallValuePrinted,
Rule::EquationWithoutImplications,
Rule::UnresolvedReference,
Rule::DefinedElsewhere,
Rule::ProposedEntries,
];
/// The rule's nominal severity, for `--list-rules` and `--min-severity`.
@@ -202,6 +250,20 @@ impl Rule {
// Both are tidiness: the item still works, but its pool is
// carrying something nobody has accounted for.
R::UnusedOption | R::UnjustifiedRetirement => Severity::Low,
R::GlossaryUnused
| R::NoExamWording
| R::CircularDefinition
| R::UnsourcedValue
| R::UnlistedTerm
| R::EquationWithoutImplications => Severity::Low,
R::NoTextbookName
| R::CoinedTermInItem
| R::ValueConflict
| R::ForbiddenTargetWording
| R::RecallValuePrinted
| R::UnresolvedReference => Severity::Medium,
R::DefinedElsewhere | R::ProposedEntries => Severity::Note,
}
}
@@ -238,6 +300,20 @@ impl Rule {
Rule::DifficultyMissed => "evidence-difficulty",
Rule::DiscriminationMissed => "evidence-discrimination",
Rule::DuplicateStem => "evidence-duplicate-stem",
Rule::GlossaryUnused => "glossary-unused",
Rule::NoTextbookName => "glossary-no-textbook-name",
Rule::NoExamWording => "glossary-no-exam-wording",
Rule::CircularDefinition => "glossary-circular-definition",
Rule::CoinedTermInItem => "glossary-coined-in-item",
Rule::UnsourcedValue => "glossary-value-unsourced",
Rule::ValueConflict => "glossary-value-conflict",
Rule::ForbiddenTargetWording => "glossary-forbidden-target-wording",
Rule::UnlistedTerm => "glossary-unlisted-term",
Rule::RecallValuePrinted => "glossary-recall-value-printed",
Rule::EquationWithoutImplications => "glossary-equation-no-implications",
Rule::UnresolvedReference => "glossary-unresolved-reference",
Rule::DefinedElsewhere => "glossary-defined-elsewhere",
Rule::ProposedEntries => "glossary-proposed",
}
}
@@ -272,6 +348,20 @@ impl Rule {
| Rule::DiscriminationMissed
| Rule::NonfunctioningDistractor
| Rule::DuplicateStem => "evidence",
Rule::GlossaryUnused
| Rule::NoTextbookName
| Rule::NoExamWording
| Rule::CircularDefinition
| Rule::CoinedTermInItem
| Rule::UnsourcedValue
| Rule::ValueConflict
| Rule::ForbiddenTargetWording
| Rule::UnlistedTerm
| Rule::RecallValuePrinted
| Rule::EquationWithoutImplications
| Rule::UnresolvedReference
| Rule::DefinedElsewhere
| Rule::ProposedEntries => "glossary",
}
}
@@ -308,6 +398,20 @@ impl Rule {
Rule::DifficultyMissed,
Rule::DiscriminationMissed,
Rule::DuplicateStem,
Rule::GlossaryUnused,
Rule::NoTextbookName,
Rule::NoExamWording,
Rule::CircularDefinition,
Rule::CoinedTermInItem,
Rule::UnsourcedValue,
Rule::ValueConflict,
Rule::ForbiddenTargetWording,
Rule::UnlistedTerm,
Rule::RecallValuePrinted,
Rule::EquationWithoutImplications,
Rule::UnresolvedReference,
Rule::DefinedElsewhere,
Rule::ProposedEntries,
]
}
@@ -346,6 +450,32 @@ impl Rule {
Rule::DifficultyMissed => "observed difficulty was far from predicted",
Rule::DiscriminationMissed => "observed discrimination contradicted the prediction",
Rule::DuplicateStem => "two items have nearly the same stem",
Rule::GlossaryUnused => "a glossary entry no target, equation, value, or option needs",
Rule::NoTextbookName => "a renamed or coined term gives no textbook name to look up",
Rule::NoExamWording => "a coined term gives no exam_wording for items to use",
Rule::CircularDefinition => "a definition uses the term it defines",
Rule::CoinedTermInItem => "a coined term appears in an item stem or option",
Rule::UnsourcedValue => "a taught value has no source",
Rule::ValueConflict => {
"a value's conflicts pattern matches student-facing text or an item"
}
Rule::ForbiddenTargetWording => {
"objective or target text matches policy.target_text_forbid"
}
Rule::UnlistedTerm => {
"an explanation names a term its target neither lists nor inherits"
}
Rule::RecallValuePrinted => "an item stem or option prints a recall value",
Rule::EquationWithoutImplications => {
"a target at ceiling 3 or above turns on an equation with no implications"
}
Rule::UnresolvedReference => {
"an old prose field refers to an undeclared objective or target"
}
Rule::DefinedElsewhere => {
"an entry is defined in an objective file other than the first to need it"
}
Rule::ProposedEntries => "proposed glossary entries are waiting for approval",
}
}
}
@@ -427,6 +557,7 @@ pub fn lint_catalog(catalog: &Catalog, t: &Thresholds) -> Vec<Finding> {
out.extend(lint_key_positions(catalog, t));
out.extend(lint_option_counts(catalog));
out.extend(lint_duplicate_stems(catalog, t));
out.extend(lint_glossary(catalog));
out.sort_by(|a, b| {
b.severity
@@ -705,7 +836,12 @@ pub fn lint_item(entry: &Entry, course: &CourseFile, t: &Thresholds) -> Vec<Find
if is_ready {
for o in &it.options {
if !o.correct {
if o.misconception.is_none() && o.error_type.is_none() {
// A registry misconception named by id designs the distractor
// as well as its own text does.
if o.misconception.is_none()
&& o.misconception_id.is_none()
&& o.error_type.is_none()
{
push(
Rule::UndesignedDistractor,
Severity::Medium,
@@ -725,7 +861,13 @@ pub fn lint_item(entry: &Entry, course: &CourseFile, t: &Thresholds) -> Vec<Find
);
}
}
if o.student_text().is_none() {
// The registry's correction is what a report shows for an option
// that names a declared misconception and says nothing itself.
let registry_feedback = o
.misconception_id
.as_ref()
.is_some_and(|id| course.misconceptions.contains_key(id));
if o.student_text().is_none() && !registry_feedback {
push(
Rule::NoStudentFeedback,
Severity::Low,
+493
View File
@@ -0,0 +1,493 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The glossary rules: advice about terms, equations, values, and
//! misconceptions, and about how prose and items use them.
//!
//! These replace checks the authoring pipelines each ran on their own copy of
//! `terms.yaml` and `values.yaml` (the slide linter's W374 and W375, the bank
//! linter's W334, W804, and W805, the brief linter's terms and values checks),
//! so every pipeline can call `coursebank lint` instead.
//!
//! `docs/design/2.1/CODES.md` maps each rule to the specification's code.
use std::collections::BTreeSet;
use std::path::Path;
use super::{Finding, Rule};
use crate::catalog::Catalog;
use crate::course::CourseFile;
use crate::course::glossary::{RefKind, TermKind, brace_refs, compile, contains_word};
use crate::taxonomy::{Level, Status};
/// One piece of item text a student reads, for the rules that scan items.
struct ItemText<'a> {
/// The item's global id.
uid: &'a str,
/// Which part: `stem`, ``option `o-x` ``, ``option `o-x` feedback``,
/// `solution`.
field: String,
/// Whether it is the stem or an option's text, the two places a printed
/// value gives an answer away.
stem_or_option: bool,
/// The text.
text: &'a str,
}
/// Every student-visible piece of every live item.
fn item_texts(catalog: &Catalog) -> Vec<ItemText<'_>> {
let mut out = Vec::new();
for entry in &catalog.entries {
let it = &entry.item;
if it.status == Status::Retired {
continue;
}
let uid = entry.uid.as_str();
out.push(ItemText {
uid,
field: "stem".into(),
stem_or_option: true,
text: &it.stem,
});
for o in &it.options {
out.push(ItemText {
uid,
field: format!("option `{}`", o.id),
stem_or_option: true,
text: &o.text,
});
if let Some(feedback) = &o.feedback_student {
out.push(ItemText {
uid,
field: format!("option `{}` feedback", o.id),
stem_or_option: false,
text: feedback,
});
}
}
if let Some(explanation) = it.solution.as_ref().and_then(|s| s.explanation.as_ref()) {
out.push(ItemText {
uid,
field: "solution".into(),
stem_or_option: false,
text: explanation,
});
}
}
out
}
/// A finding at its rule's own severity.
fn finding(rule: Rule, subject: &str, message: String) -> Finding {
Finding {
subject: subject.to_string(),
rule,
severity: rule.severity(),
message,
}
}
/// Runs every glossary rule over a course and its banks.
///
/// # Arguments
///
/// * `catalog` - the loaded course.
///
/// # Returns
///
/// Findings, in rule order.
pub fn lint_glossary(catalog: &Catalog) -> Vec<Finding> {
let course = &catalog.course;
let items = item_texts(catalog);
let mut out = Vec::new();
out.extend(unused(catalog));
out.extend(terms(course));
out.extend(coined_in_items(course, &items));
out.extend(values(course, &items));
out.extend(forbidden_wording(course));
out.extend(unlisted_terms(course));
out.extend(equations_without_implications(course));
out.extend(unresolved_references(course));
out.extend(defined_elsewhere(course));
out.extend(proposed(course));
out
}
/// `glossary-unused`: an entry no target, equation symbol, value, or item
/// option needs (W2101).
fn unused(catalog: &Catalog) -> Vec<Finding> {
let course = &catalog.course;
let mut used: BTreeSet<&str> = BTreeSet::new();
for target in course.learning_targets.values() {
for list in [
&target.terms,
&target.equations,
&target.values,
&target.misconceptions,
] {
used.extend(list.iter().map(String::as_str));
}
}
for equation in course.equations.values() {
used.extend(equation.symbols.iter().filter_map(|s| s.term.as_deref()));
}
for value in course.values.values() {
used.extend(value.term.as_deref());
}
for entry in &catalog.entries {
for option in &entry.item.options {
used.extend(option.misconception_id.as_deref());
}
}
course
.glossary_ids()
.into_iter()
.filter(|id| !used.contains(id))
.map(|id| {
let noun = RefKind::of(id).map(RefKind::noun).unwrap_or("entry");
finding(
Rule::GlossaryUnused,
id,
format!(
"no target lists this {noun}, and no equation symbol, value, or item option \
names it; list it on the targets that need it, or remove it"
),
)
})
.collect()
}
/// The rules about one term at a time: `glossary-no-textbook-name` (W2102),
/// `glossary-no-exam-wording` (W2103), and `glossary-circular-definition`
/// (W2104).
fn terms(course: &CourseFile) -> Vec<Finding> {
let mut out = Vec::new();
for (id, term) in &course.terms {
if matches!(term.kind, TermKind::Renamed | TermKind::Coined) && term.textbook.is_empty() {
out.push(finding(
Rule::NoTextbookName,
id,
format!(
"a {} term with no textbook name, so a student has nothing a search will \
find; add `textbook`",
term.kind.as_str()
),
));
}
if term.kind == TermKind::Coined && term.exam_wording.is_none() {
out.push(finding(
Rule::NoExamWording,
id,
"a coined term with no `exam_wording`, so an item writer has no plain wording to \
use instead"
.into(),
));
}
if let Some(form) = term
.forms()
.into_iter()
.find(|form| contains_word(&term.definition, form))
{
out.push(finding(
Rule::CircularDefinition,
id,
format!(
"the definition uses '{form}', the term it defines; define it in other words"
),
));
}
}
out
}
/// `glossary-coined-in-item`: a coined label, a teaching aid rather than tested
/// vocabulary, in what an item shows a student (W2105).
fn coined_in_items(course: &CourseFile, items: &[ItemText<'_>]) -> Vec<Finding> {
let mut out = Vec::new();
for (id, term) in course
.terms
.iter()
.filter(|(_, t)| t.kind == TermKind::Coined)
{
for text in items {
if let Some(form) = term
.forms()
.into_iter()
.find(|form| contains_word(text.text, form))
{
let instead = term
.exam_wording
.as_deref()
.map(|w| format!("write '{w}'"))
.unwrap_or_else(|| "write its textbook name".into());
out.push(finding(
Rule::CoinedTermInItem,
text.uid,
format!(
"{} uses the coined term '{form}' ({id}); {instead}",
text.field
),
));
}
}
}
out
}
/// The rules about taught values: `glossary-value-unsourced` (W2106),
/// `glossary-value-conflict` (W2107), and `glossary-recall-value-printed`
/// (W2110).
fn values(course: &CourseFile, items: &[ItemText<'_>]) -> Vec<Finding> {
let mut out = Vec::new();
for (id, value) in &course.values {
if value.source.is_none() {
out.push(finding(
Rule::UnsourcedValue,
id,
"no `source`, so nobody can check the number against the text it came from".into(),
));
}
}
// Patterns that do not compile are a validation error already.
let patterns: Vec<(&str, regex::Regex)> = course
.values
.iter()
.flat_map(|(id, v)| {
v.conflicts
.iter()
.filter_map(|p| compile(p, false).ok())
.map(move |re| (id.as_str(), re))
})
.collect();
for prose in course.student_prose() {
for (id, re) in &patterns {
if let Some(m) = re.find(prose.text) {
out.push(finding(
Rule::ValueConflict,
prose.id,
format!(
"{} writes '{}', a variant of {id}; write {{{id}}} so every page says it \
one way",
prose.field,
m.as_str()
),
));
}
}
}
for text in items {
for (id, re) in &patterns {
if let Some(m) = re.find(text.text) {
out.push(finding(
Rule::ValueConflict,
text.uid,
format!(
"{} writes '{}', a variant of {id}; state it as '{}'",
text.field,
m.as_str(),
course.values[*id].say
),
));
}
}
}
for (id, value) in course.values.iter().filter(|(_, v)| v.recall == Some(true)) {
let say = value.say.trim().to_lowercase();
if say.is_empty() {
continue;
}
for text in items.iter().filter(|t| t.stem_or_option) {
if text.text.to_lowercase().contains(&say) {
out.push(finding(
Rule::RecallValuePrinted,
text.uid,
format!(
"{} prints '{}' ({id}), a value students are expected to recall",
text.field, value.say
),
));
}
}
}
out
}
/// `glossary-forbidden-target-wording`: objective or target text that matches
/// one of the course's `target_text_forbid` patterns (W2108).
fn forbidden_wording(course: &CourseFile) -> Vec<Finding> {
let patterns: Vec<regex::Regex> = course
.policy
.target_text_forbid
.iter()
.filter_map(|p| compile(p, true).ok())
.collect();
let texts = course
.learning_objectives
.iter()
.map(|(id, o)| (id, &o.text))
.chain(course.learning_targets.iter().map(|(id, t)| (id, &t.text)));
let mut out = Vec::new();
for (id, text) in texts {
for re in &patterns {
if let Some(m) = re.find(text) {
out.push(finding(
Rule::ForbiddenTargetWording,
id,
format!(
"the text says '{}', which the course's policy forbids (`{}`)",
m.as_str(),
re.as_str()
),
));
}
}
}
out
}
/// `glossary-unlisted-term`: an explanation names a term in braces that its
/// target neither lists nor inherits from a prerequisite target (W2109).
fn unlisted_terms(course: &CourseFile) -> Vec<Finding> {
let mut out = Vec::new();
for (id, target) in &course.learning_targets {
let Some(explanation) = &target.explanation else {
continue;
};
let mut listed: BTreeSet<&str> = target.terms.iter().map(String::as_str).collect();
for prerequisite in &target.prerequisites {
if let Some(p) = course.learning_targets.get(prerequisite) {
listed.extend(p.terms.iter().map(String::as_str));
}
}
for (_, term) in brace_refs(explanation) {
if RefKind::of(term) == Some(RefKind::Term) && !listed.contains(term) {
out.push(finding(
Rule::UnlistedTerm,
id,
format!(
"the explanation uses {{{term}}}, which the target does not list in \
`terms` and no prerequisite target lists either"
),
));
}
}
}
out
}
/// `glossary-equation-no-implications`: a target at ceiling 3 or above turns on
/// an equation that lists nothing a question could test (W2111).
fn equations_without_implications(course: &CourseFile) -> Vec<Finding> {
let mut out = Vec::new();
for (id, target) in &course.learning_targets {
let deep = course
.effective_level_ceiling(id)
.is_some_and(|l| l >= Level::Apply);
if !deep {
continue;
}
for eq in &target.equations {
if course
.equations
.get(eq)
.is_some_and(|e| e.implications.is_empty())
{
out.push(finding(
Rule::EquationWithoutImplications,
id,
format!(
"turns on {eq}, which lists no `implications`, so a level 3 item has \
nothing written down to test"
),
));
}
}
}
out
}
/// `glossary-unresolved-reference`: a `{lo-...}` or `{t-...}` in a field 2.0
/// already had that names nothing. Validation leaves it alone because 2.0
/// accepted it; it still renders as literal braces.
fn unresolved_references(course: &CourseFile) -> Vec<Finding> {
let mut out = Vec::new();
for prose in course.student_prose().into_iter().filter(|p| p.legacy) {
for (_, id) in brace_refs(prose.text) {
if !course.resolves(id) {
out.push(finding(
Rule::UnresolvedReference,
prose.id,
format!(
"{} refers to {{{id}}}, which is not a declared objective or target; a \
student sees the braces",
prose.field
),
));
}
}
}
out
}
/// `glossary-defined-elsewhere`: an entry defined in an objective file other
/// than the one whose targets first need it (I2112). A convention, reported as
/// a note.
fn defined_elsewhere(course: &CourseFile) -> Vec<Finding> {
let mut out = Vec::new();
for (id, (lecture, target)) in course.glossary_first_use() {
let Some((_, defined)) = course.origin(id) else {
continue;
};
if !defined.starts_with("objectives") {
continue;
}
let objective = course.objective_for(target);
let Some((_, home)) = course.origin(objective) else {
continue;
};
if defined != home {
out.push(finding(
Rule::DefinedElsewhere,
id,
format!(
"defined in {}; first needed by {target} in {lecture}, whose objective is in \
{}",
shown(defined),
shown(home)
),
));
}
}
out
}
/// `glossary-proposed`: how many entries are waiting for approval (I2113).
fn proposed(course: &CourseFile) -> Vec<Finding> {
let waiting: Vec<&str> = course
.glossary_ids()
.into_iter()
.filter(|id| course.is_proposed(id))
.collect();
if waiting.is_empty() {
return Vec::new();
}
vec![finding(
Rule::ProposedEntries,
"glossary",
format!(
"{} proposed entr{} waiting for the instructor: {}",
waiting.len(),
if waiting.len() == 1 { "y" } else { "ies" },
waiting.join(", ")
),
)]
}
/// A path as a message shows it.
fn shown(path: &Path) -> String {
path.display().to_string()
}
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Drawing an assessment from the item pool.
+99 -7
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The command-line argument model.
@@ -137,6 +137,37 @@ pub(crate) enum MigrateCommand {
#[arg(long)]
dry_run: bool,
},
/// Drop the trailing counter from every item id.
///
/// A rename, not a normalization: `q-x-001` and `q-x` are unrelated
/// strings, so banks, records, seals, and the response store are rewritten
/// in one pass or not at all. Two ids that would collide abort it.
Counters {
/// Show the renames, and write nothing.
#[arg(long)]
dry_run: bool,
},
/// Replace the `order:` integers with ordered declarations.
///
/// Objective order comes from each lecture's `teaches` list, target order
/// from a `targets:` list this writes onto each objective. Run
/// `migrate split` first — without `teaches`, objective order has no
/// source.
Order {
/// Show what would change, and write nothing.
#[arg(long)]
dry_run: bool,
},
/// Turn citations written into `note:` fields into real fields.
///
/// Journal, volume, pages, and DOI parsed out of the prose, with whatever
/// the note still says left in it. A field the entry already declares is
/// never overwritten; a disagreement is reported instead.
References {
/// Show what would change, and write nothing.
#[arg(long)]
dry_run: bool,
},
/// Fill in the stored `variant` column from the assessment records.
///
/// Nothing in the tool needs it — a variant is derived from the placement
@@ -169,6 +200,25 @@ pub(crate) enum MigrateCommand {
#[arg(long)]
dry_run: bool,
},
/// Bring a 1.0 terms.yaml and values.yaml into the 2.1 glossary.
///
/// Writes glossary.yaml. Ids gain their `term-` and `val-` prefixes,
/// `course` becomes `term` and `meaning` becomes `definition`, and `first`
/// and `lectures` are dropped, since 2.1 derives them from the targets
/// that list an entry; each one that disagrees is reported. The old files
/// are left in place, and an entry the course already declares is
/// skipped, so a second run changes nothing.
Glossary {
/// The 1.0 terms file.
#[arg(long, value_name = "FILE")]
terms: Option<PathBuf>,
/// The 1.0 values file.
#[arg(long, value_name = "FILE")]
values: Option<PathBuf>,
/// Show what would be written, and write nothing.
#[arg(long)]
dry_run: bool,
},
/// Rewrite pre-2.0 `bank::item` ids as the item ids they name.
///
/// Touches assessment records, seals, and the response store. Everything
@@ -251,8 +301,9 @@ pub(crate) struct LintArgs {
/// Skip these rule codes.
#[arg(long, value_delimiter = ',')]
pub(crate) ignore: Vec<String>,
/// Only report findings at this severity or above.
#[arg(long, value_enum, default_value = "low")]
/// Only report findings at this severity or above. Notes are printed but
/// never make lint fail.
#[arg(long, value_enum, default_value = "note")]
pub(crate) min_severity: SeverityArg,
/// Exit 0 even when findings exist.
#[arg(long)]
@@ -270,7 +321,7 @@ pub(crate) enum LectureCommand {
Readings {
/// Lecture id, e.g. L1.1.
id: String,
/// Which flavour of Markdown to write.
/// Which flavor of Markdown to write.
#[arg(long, value_enum, default_value = "quarto")]
style: StyleArg,
/// Output path; prints to stdout when omitted.
@@ -288,7 +339,38 @@ pub(crate) enum LectureCommand {
Objectives {
/// Lecture id, e.g. L1.1.
id: String,
/// Which flavour of Markdown to write.
/// Which flavor of Markdown to write.
#[arg(long, value_enum, default_value = "quarto")]
style: StyleArg,
/// Output path; prints to stdout when omitted.
#[arg(long)]
out: Option<PathBuf>,
},
/// Write the whole page for one lecture.
///
/// Its description; each objective with its targets, their ceilings, and
/// their explanations; the terms new here and those from earlier lectures;
/// the equations, numbers, and common mistakes its targets turn on; the
/// readings with their times; and the materials. Only student-facing
/// fields are read.
Page {
/// Lecture id, e.g. L1.1.
id: String,
/// Which flavor of Markdown to write.
#[arg(long, value_enum, default_value = "quarto")]
style: StyleArg,
/// Where the course glossary page lives, for links to entries this page
/// does not define.
#[arg(long, default_value = coursebank::lecture::DEFAULT_GLOSSARY_HREF)]
glossary_href: String,
/// Output path; prints to stdout when omitted.
#[arg(long)]
out: Option<PathBuf>,
},
/// Write the course glossary page: every term, equation, and value, with
/// the lecture that introduces it.
Glossary {
/// Which flavor of Markdown to write.
#[arg(long, value_enum, default_value = "quarto")]
style: StyleArg,
/// Output path; prints to stdout when omitted.
@@ -323,6 +405,7 @@ impl StyleArg {
#[derive(Debug, Clone, Copy, ValueEnum)]
pub(crate) enum SeverityArg {
Note,
Low,
Medium,
High,
@@ -332,6 +415,7 @@ impl SeverityArg {
/// Converts the CLI value into the library's [`Severity`].
pub(crate) fn as_severity(self) -> Severity {
match self {
SeverityArg::Note => Severity::Note,
SeverityArg::Low => Severity::Low,
SeverityArg::Medium => Severity::Medium,
SeverityArg::High => Severity::High,
@@ -698,7 +782,7 @@ pub(crate) enum IngestCommand {
#[command(flatten)]
common: IngestCommon,
/// Ingest even when a directory's graded keys do not match the form it
/// was labelled with.
/// was labeled with.
#[arg(long)]
allow_mismatch: bool,
/// The export numbers questions by recorded number rather than by
@@ -724,6 +808,14 @@ pub(crate) enum AnalyzeCommand {
/// Pool every stored administration of this assessment.
#[arg(long)]
pooled: bool,
/// Also write the administration record under analysis/.
///
/// Two CSVs, one row per question and one per question-and-option,
/// written once and never rewritten. Cohort aggregates only, so unlike
/// the response data they are meant to be committed — which is what
/// keeps the history when `data/` rotates.
#[arg(long = "record")]
record: bool,
},
/// Fit an IRT model.
Irt {
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Command handlers, grouped by workflow stage.
+49 -11
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The responses-to-report pipeline.
@@ -195,8 +195,9 @@ pub(crate) fn ingest(cli: &Cli, sub: &IngestCommand) -> Result<Outcome> {
for source in &parsed {
if !intake::looks_like_export(&source.dir) {
return Err(Error::usage(format!(
"{} holds no files named `1.csv` … `N.csv`. Gradescope writes one file \
per question; point at the directory those were unzipped into",
"{} holds no files named `1.csv` … `N.csv` or `N_Title.csv`. Gradescope \
writes one file per question; point at the directory those were \
unzipped into",
source.dir.display()
)));
}
@@ -294,12 +295,46 @@ pub(crate) fn analyze(cli: &Cli, sub: &AnalyzeCommand) -> Result<Outcome> {
let store = Store::open(catalog.layout.data())?;
match sub {
AnalyzeCommand::Items { id, pooled } => {
AnalyzeCommand::Items {
id,
pooled,
record: write_record,
} => {
let record = load_record(&catalog, id)?;
let set = responses_for(&store, &catalog, &record, *pooled)?;
let analysis =
classical::analyze(&set, &Thresholds::default(), Some(&record), Some(&catalog));
if *write_record {
// The full administration id, not the assessment id: `e1` is
// given again next year, and a record named after it would
// collide with this cohort's — which `write_csv` would report
// as "already recorded" when it is a different exam entirely.
let administration = coursebank::responses::administration_id(
&catalog.course.course.code,
record
.assessment
.term
.as_deref()
.unwrap_or(&catalog.course.course.term),
&record.assessment.id,
);
let written = calibrate::record_measurements(
&catalog.layout,
&administration,
&analysis,
&catalog,
Some(&record),
)?;
for path in &written {
println!("wrote {}", path.display());
}
println!(
"\nCohort aggregates only, so these are committed. Review with\n git diff \
analysis/\n"
);
}
for w in &analysis.warnings {
println!("! {w}\n");
}
@@ -463,17 +498,20 @@ pub(crate) fn calibrate(cli: &Cli, args: &CalibrateArgs) -> Result<Outcome> {
}
if !args.apply {
println!(
"Nothing written. Re-run with --apply to write these {} change(s) into the bank \
files, then review the git diff.",
"Nothing written. Re-run with --apply to write these {} change(s) into \
analysis/calibration.yaml, then review the git diff.",
plan.changes.len()
);
return Ok(Outcome::Ok);
}
for path in calibrate::apply(&plan)? {
println!("updated {}", path.display());
}
println!("\nReview the diff before committing: git diff banks/");
let path = calibrate::apply(&catalog.layout, &plan)?;
println!("updated {}", path.display());
println!(
"\nReview the diff before committing: git diff analysis/\n\nThe banks are untouched. \
Statistics are cohort aggregates with no student in\n them, which is why analysis/ is \
committed and data/ is not."
);
Ok(Outcome::Ok)
}
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Managing items and building assessments from them.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Turning an assembled assessment into deliverables.
+5 -4
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Replacement handlers for `src/commands/analysis.rs`.
@@ -207,8 +207,9 @@ pub(crate) fn ingest(cli: &Cli, sub: &IngestCommand) -> Result<Outcome> {
for source in &parsed {
if !intake::looks_like_export(&source.dir) {
return Err(Error::usage(format!(
"{} holds no files named `1.csv` … `N.csv`. Gradescope writes one file \
per question; point at the directory those were unzipped into",
"{} holds no files named `1.csv` … `N.csv` or `N_Title.csv`. Gradescope \
writes one file per question; point at the directory those were \
unzipped into",
source.dir.display()
)));
}
+24 -3
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Rendering lecture pages, and checking what backs each learning target.
@@ -10,7 +10,9 @@
use coursebank::course::CourseFile;
use coursebank::error::Result;
use coursebank::lecture::{objectives_markdown, readings_markdown};
use coursebank::lecture::{
PageOptions, glossary_markdown, objectives_markdown, page_markdown, readings_markdown,
};
use coursebank::yaml;
use crate::cli::{Cli, LectureCommand};
@@ -29,6 +31,25 @@ pub(crate) fn lecture(cli: &Cli, sub: &LectureCommand) -> Result<Outcome> {
objectives_markdown(&course, id, style.as_style())?,
out.as_deref(),
),
LectureCommand::Page {
id,
style,
glossary_href,
out,
} => {
let options = PageOptions {
style: style.as_style(),
glossary_href: glossary_href.clone(),
};
emit(page_markdown(&course, id, &options)?, out.as_deref())
}
LectureCommand::Glossary { style, out } => {
let options = PageOptions {
style: style.as_style(),
..PageOptions::default()
};
emit(glossary_markdown(&course, &options), out.as_deref())
}
LectureCommand::Coverage { lecture: only } => coverage(&course, only.as_deref(), cli.quiet),
}
}
+180 -5
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Setting up a course and checking it stays well-formed.
@@ -16,6 +16,7 @@ use std::path::Path;
use coursebank::assessment::AssessmentFile;
use coursebank::bank::BankFile;
use coursebank::catalog::Severity;
use coursebank::course::fragment::{self, Section};
use coursebank::course::{COURSE_FILE, CourseFile};
use coursebank::error::{Error, Result};
@@ -35,10 +36,15 @@ use crate::helpers::{load, truncate};
/// The `.gitignore` written by `init`.
pub(crate) const GITIGNORE: &str = "\
# Generated output: exports, rendered exams, reports.
# Generated output: exports, rendered exams, reports. A student report carries
# names, so it belongs here rather than in the repository.
build/
reports/
# Response data: every row carries a student. The statistics derived from it are
# cohort aggregates and live in analysis/, which is committed on purpose.
data/
# Typst and PDF artifacts.
*.pdf
@@ -109,9 +115,170 @@ pub(crate) fn migrate(cli: &Cli, sub: &MigrateCommand) -> Result<Outcome> {
MigrateCommand::Options { dry_run } => migrate_options(cli, *dry_run),
MigrateCommand::Stems { dry_run } => migrate_stems(cli, *dry_run),
MigrateCommand::Variants { dry_run } => migrate_variants(cli, *dry_run),
MigrateCommand::References { dry_run } => migrate_references(cli, *dry_run),
MigrateCommand::Order { dry_run } => migrate_order(cli, *dry_run),
MigrateCommand::Counters { dry_run } => migrate_counters(cli, *dry_run),
MigrateCommand::Glossary {
terms,
values,
dry_run,
} => migrate_glossary(cli, terms.as_deref(), values.as_deref(), *dry_run),
}
}
/// Brings a 1.0 terms.yaml and values.yaml into glossary.yaml.
fn migrate_glossary(
cli: &Cli,
terms: Option<&Path>,
values: Option<&Path>,
dry_run: bool,
) -> Result<Outcome> {
let report = migrate::glossary(&cli.course, terms, values, !dry_run)?;
let list = |heading: &str, lines: &[String]| {
if !lines.is_empty() {
println!("{heading}");
for line in lines {
println!(" {line}");
}
println!();
}
};
let renamed: Vec<String> = report
.renamed
.iter()
.map(|(old, new)| format!("{old} -> {new}"))
.collect();
list("renamed:", &renamed);
list(
"dropped `first` and `lectures` that the targets now contradict:",
&report.derived,
);
list("dropped, no 2.1 field:", &report.dropped);
list("relinked:", &report.relinked);
list("no target lists these yet:", &report.unlisted);
let present: Vec<String> = report
.present
.iter()
.map(|(id, path)| format!("{id} ({})", path.display()))
.collect();
list("already declared, skipped:", &present);
let Some(text) = &report.text else {
println!("nothing to do: the course already declares every entry");
return Ok(Outcome::Ok);
};
if dry_run {
println!(
"{} would read:\n\n{text}\nnothing written",
report.path.display()
);
return Ok(Outcome::Ok);
}
println!(
"wrote {} entr{} to {}\n\nNext:\n coursebank validate\n coursebank lint \
# glossary-unused lists entries nothing points at yet\n\n\
The old files are untouched; delete them once the pipelines read the registry.",
report.added.len(),
if report.added.len() == 1 { "y" } else { "ies" },
report.path.display()
);
Ok(Outcome::Ok)
}
/// Drops the trailing counter from every item id.
fn migrate_counters(cli: &Cli, dry_run: bool) -> Result<Outcome> {
let (rename, touched) = migrate::counters(&cli.course, !dry_run)?;
if rename.is_empty() {
println!("nothing to do: no item id ends in a counter");
return Ok(Outcome::Ok);
}
for (old, new) in &rename {
println!(" {old} -> {new}");
}
println!();
for (path, n) in &touched {
println!(" {:<44} {n:>5} reference(s)", path.display());
}
if dry_run {
println!("\nnothing written");
return Ok(Outcome::Ok);
}
let sealed = touched
.iter()
.filter(|(p, _)| p.starts_with("seals"))
.count();
println!(
"\nrenamed {} item(s) across {} file(s)",
rename.len(),
touched.len()
);
if sealed > 0 {
println!(
"\n{sealed} seal(s) were rewritten. The ids are inside the digest, so each one was \
recomputed\n and the previous digest recorded under `superseded_digests`. A seal \
that has been\n rewritten says so rather than looking untouched."
);
}
println!(
"\nNext:\n coursebank validate\n coursebank seal verify\n coursebank analyze items \
--all # the join key moved; check the data still lands"
);
Ok(Outcome::Ok)
}
/// Replaces the order integers with ordered declarations.
fn migrate_order(cli: &Cli, dry_run: bool) -> Result<Outcome> {
let touched = migrate::order(&cli.course, !dry_run)?;
if touched.is_empty() {
println!("nothing to do: no `order:` left to derive");
return Ok(Outcome::Ok);
}
let total: usize = touched.iter().map(|(_, n)| n).sum();
for (path, n) in &touched {
println!(" {:<44} {n:>5} order(s) dropped", path.display());
}
if dry_run {
println!("\nnothing written");
return Ok(Outcome::Ok);
}
println!(
"\nrewrote {} file(s), {total} integer(s) gone\n\nNext:\n coursebank validate\n \
coursebank lecture objectives L1.2 # check the order still reads right",
touched.len()
);
Ok(Outcome::Ok)
}
/// Takes the citations out of the notes and puts them in fields.
fn migrate_references(cli: &Cli, dry_run: bool) -> Result<Outcome> {
let (touched, notes) = migrate::references(&cli.course, !dry_run)?;
for (path, n) in &touched {
println!(" {:<44} {n:>5} note(s) taken apart", path.display());
}
for note in &notes {
println!(" note: {note}");
}
if touched.is_empty() {
println!("nothing to do: no note is carrying a citation");
return Ok(Outcome::Ok);
}
if dry_run {
println!("\nnothing written");
return Ok(Outcome::Ok);
}
println!(
"\nrewrote {} file(s)\n\nNext:\n coursebank validate\n coursebank references list\n\n\
An issue number is never inferred, not even from a DOI that encodes one, so add those \
by hand.",
touched.len()
);
Ok(Outcome::Ok)
}
/// Fills in the stored variant column.
fn migrate_variants(cli: &Cli, dry_run: bool) -> Result<Outcome> {
let touched = migrate::store_variants(&cli.course, !dry_run)?;
@@ -590,6 +757,10 @@ pub(crate) fn validate(cli: &Cli) -> Result<Outcome> {
let seals = coursebank::seal::SealFile::load_all(&catalog.layout.seals())?;
all.extend(catalog.validate_seals(&seals));
// The statistics are kept in a different file from the questions they
// describe, so the link between them is worth checking rather than assuming.
all.extend(catalog.calibration.validate(&catalog));
if all.is_empty() {
if !cli.quiet {
println!(
@@ -612,7 +783,8 @@ pub(crate) fn validate(cli: &Cli) -> Result<Outcome> {
/// `lint`: run item-writing rules, grouped by subject, honoring the filter flags.
///
/// With `--list-rules` it prints the rule table and stops. Otherwise it returns
/// [`Outcome::Findings`] when anything fires, unless `--no-fail` was given.
/// [`Outcome::Findings`] when anything above a note fires, unless `--no-fail`
/// was given.
pub(crate) fn lint(cli: &Cli, args: &LintArgs) -> Result<Outcome> {
if args.list_rules {
println!("{:<32} {:<8} WHAT IT CATCHES", "CODE", "SEVERITY");
@@ -672,7 +844,10 @@ pub(crate) fn lint(cli: &Cli, args: &LintArgs) -> Result<Outcome> {
);
println!("Silence a rule with --ignore <code>, or see them all with --list-rules.");
if args.no_fail {
// A note is information, not a problem: a course with entries waiting for
// approval still passes a pipeline's lint gate.
let problems = findings.iter().any(|f| f.severity > Severity::Note);
if args.no_fail || !problems {
Ok(Outcome::Ok)
} else {
Ok(Outcome::Findings)
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Getting grading data in, and keeping it.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Reading Canvas's "Student Analysis" quiz export.
+46 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Turning what a student marked into what a student chose.
@@ -592,6 +592,7 @@ pub fn apply(set: &mut ResponseSet, decoder: &FormDecoder, numbering: Numbering)
// something else entirely. Nothing downstream can tell them apart, so the
// collision has to be caught here.
let recorded: BTreeSet<u32> = decoder.numbers().collect();
let last_recorded = recorded.iter().next_back().copied().unwrap_or(0);
for row in &mut set.rows {
let belongs = row
@@ -604,6 +605,17 @@ pub fn apply(set: &mut ResponseSet, decoder: &FormDecoder, numbering: Numbering)
}
let Some(map) = decoder.lookup(row.item_number, numbering) else {
// A curve was never on the paper, so being off the form is expected.
// It still needs a number of its own: the positions past the end of
// the form can be numbers the record uses once a question was pulled
// before printing, and discarding it would lose points from the total.
if row.is_curve() {
if recorded.contains(&row.item_number) {
row.item_number =
last_recorded + row.item_number.saturating_sub(decoder.len() as u32);
}
continue;
}
unmapped_positions.insert(row.item_number);
if recorded.contains(&row.item_number) {
colliding_positions.insert(row.item_number);
@@ -802,6 +814,38 @@ mod tests {
assert_eq!(set.rows[0].item_number, 7, "left as found");
}
fn curve(number: u32) -> Response {
let mut r = row(number, "A");
r.selected = Vec::new();
r.correct = None;
r.bonus = true;
r.score = 3.6;
r
}
#[test]
fn a_curve_off_the_end_of_the_form_is_kept_quietly() {
let mut set = ResponseSet::new();
set.rows.push(row(1, "A"));
set.rows.push(curve(3));
let warnings = apply(&mut set, &decoder(), Numbering::Printed);
assert!(warnings.is_empty(), "{warnings:?}");
assert_eq!(set.rows[1].item_number, 3);
}
#[test]
fn a_curve_on_a_recorded_number_moves_past_the_record() {
// Two printed questions recorded as 1 and 9, and a curve as the ninth file
// of the export: position 9 is off the form, but number 9 is a real question.
let mut set = ResponseSet::new();
set.rows.push(row(1, "A"));
set.rows.push(curve(9));
let warnings = apply(&mut set, &decoder(), Numbering::Printed);
assert!(warnings.is_empty(), "{warnings:?}");
assert_eq!(set.rows.len(), 2, "the points are not discarded");
assert_eq!(set.rows[1].item_number, 16);
}
#[test]
fn a_perfect_fit_says_nothing() {
let fits = vec![FormFit {
+266 -37
View File
@@ -1,13 +1,14 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Reading Gradescope's per-question CSV exports.
//!
//! Gradescope exports one file per question, named `1.csv` through `N.csv`, in
//! wide form. Every assumption encoded here was checked against a real 32-question,
//! 24-student export rather than inferred from documentation, because several of
//! them are not what the format suggests.
//! wide form, with the title appended when the question has one: `39_Curve.csv`.
//! Every assumption encoded here was checked against a real 32-question, 24-student
//! export rather than inferred from documentation, because several of them are not
//! what the format suggests.
//!
//! The layout is:
//!
@@ -43,6 +44,12 @@
//! *Bonus questions have `max_points` of zero* while a rubric column still
//! carries points. Key detection therefore uses the largest value across the
//! maximum *and* every column, not the maximum alone.
//!
//! *A curve is a question too.* It is usually the last file, worth zero points,
//! with no option columns: a rubric row such as `Curve` worth 3.6, or nothing but
//! a point adjustment on each submission. It was never printed, so it is recorded
//! as a bonus row with nothing marked, which is what [`Response::is_curve`] looks
//! for.
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
@@ -185,9 +192,26 @@ impl Question {
///
/// # Returns
///
/// `true` when the maximum is zero but a column still awards points.
/// `true` when the maximum is zero but a column still awards points, or when
/// a question with no option columns awarded points by adjustment alone.
pub fn looks_like_bonus(&self) -> bool {
self.max_points <= 0.0 && self.top_value() > 0.0
self.max_points <= 0.0
&& (self.top_value() > 0.0
|| (!self.has_options() && self.rows.iter().any(|r| r.score != 0.0)))
}
/// Whether this looks like a curve rather than a question on the paper.
///
/// # Returns
///
/// `true` for a bonus question whose rubric names no option.
pub fn looks_like_curve(&self) -> bool {
self.looks_like_bonus() && !self.has_options()
}
/// Whether any rubric column names an option letter.
fn has_options(&self) -> bool {
self.columns.iter().any(|c| c.letter.is_some())
}
/// Whether elimination scoring was in use.
@@ -307,6 +331,27 @@ pub fn classify(label: &str) -> (ColumnKind, Option<String>, Option<String>) {
(kind, Some(upper.to_string()), note)
}
/// Reads the question number from a file name.
///
/// # Arguments
///
/// * `path` - the file, named like `12.csv` or `12_Title.csv`.
///
/// # Returns
///
/// The number, or `None` when the file is not named like a question export.
pub fn question_number(path: &Path) -> Option<u32> {
if path.extension().and_then(|e| e.to_str()) != Some("csv") {
return None;
}
let stem = path.file_stem()?.to_str()?;
let digits = stem.split_once('_').map_or(stem, |(number, _)| number);
if !digits.chars().all(|c| c.is_ascii_digit()) {
return None;
}
digits.parse().ok()
}
/// Parses one question file.
///
/// # Arguments
@@ -322,16 +367,13 @@ pub fn classify(label: &str) -> (ColumnKind, Option<String>, Option<String>) {
/// Returns [`Error::Csv`] on a malformed file and [`Error::Invalid`] when the
/// header lacks the `Submission Time` column that delimits the rubric.
pub fn parse_question(path: &Path) -> Result<Question> {
let number = path
.file_stem()
.and_then(|s| s.to_str())
.and_then(|s| s.parse::<u32>().ok())
.ok_or_else(|| {
Error::Invalid(vec![format!(
"{} is not named like a Gradescope question export (expected `12.csv`)",
path.display()
)])
})?;
let number = question_number(path).ok_or_else(|| {
Error::Invalid(vec![format!(
"{} is not named like a Gradescope question export (expected `12.csv` or \
`12_Title.csv`)",
path.display()
)])
})?;
// Flexible mode is required: the trailing `Point Values` rows are shorter
// than the header.
@@ -503,53 +545,68 @@ pub fn parse_question(path: &Path) -> Result<Question> {
///
/// # Arguments
///
/// * `dir` - the directory of `N.csv` files.
/// * `dir` - the directory of `N.csv` and `N_Title.csv` files.
/// * `ctx` - identifying information the export does not carry.
///
/// # Returns
///
/// Normalized responses and the parsed questions.
/// Normalized responses and the parsed questions, with a warning for any CSV file
/// that was not read.
///
/// # Errors
///
/// Returns [`Error::Io`] when the directory cannot be read and [`Error::Invalid`]
/// when it holds no question files.
/// when it holds no question files or two files for one question.
pub fn ingest_dir(dir: &Path, ctx: &Context) -> Result<Import> {
let mut paths: Vec<PathBuf> = std::fs::read_dir(dir)
.map_err(|e| Error::io(dir, e))?
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| {
p.extension().and_then(|e| e.to_str()) == Some("csv")
&& p.file_stem()
.and_then(|s| s.to_str())
.map(|s| s.chars().all(|c| c.is_ascii_digit()))
.unwrap_or(false)
})
.filter(|p| p.extension().and_then(|e| e.to_str()) == Some("csv"))
.collect();
// Numeric order, so `10.csv` does not sort before `2.csv`. Unnumbered files
// come out of the list here and are reported below, since a file left out
// quietly is points missing from every student's total.
paths.sort();
let (numbered, unread): (Vec<PathBuf>, Vec<PathBuf>) = paths
.into_iter()
.partition(|p| question_number(p).is_some());
let mut paths = numbered;
paths.sort_by_key(|p| question_number(p));
if paths.is_empty() {
return Err(Error::Invalid(vec![format!(
"{} holds no numbered CSV files; Gradescope exports one file per question, \
named `1.csv` through `N.csv`",
named `1.csv` through `N.csv` or `N_Title.csv`",
dir.display()
)]));
}
// Numeric order, so `10.csv` does not sort before `2.csv`.
paths.sort_by_key(|p| {
p.file_stem()
.and_then(|s| s.to_str())
.and_then(|s| s.parse::<u32>().ok())
.unwrap_or(u32::MAX)
});
for pair in paths.windows(2) {
if question_number(&pair[0]) == question_number(&pair[1]) {
return Err(Error::Invalid(vec![format!(
"{} and {} are both exports of the same question; remove the one that is out of \
date",
pair[0].display(),
pair[1].display()
)]));
}
}
let mut questions = Vec::new();
for p in &paths {
questions.push(parse_question(p)?);
}
Ok(to_responses(&questions, ctx))
let mut import = to_responses(&questions, ctx);
for path in &unread {
import.responses.warnings.push(format!(
"{} was not read: its name does not start with a question number",
path.display()
));
}
Ok(import)
}
/// Normalizes parsed questions into responses.
@@ -572,7 +629,14 @@ pub fn to_responses(questions: &[Question], ctx: &Context) -> Import {
let points = q.points_possible();
let bonus = q.looks_like_bonus();
if keyed.is_empty() {
if q.looks_like_curve() {
set.warnings.push(format!(
"question {} is worth zero points and has no option rubric, so it is recorded as \
a curve: its points go on each student's total and it is not treated as a \
question",
q.number
));
} else if keyed.is_empty() {
set.warnings.push(format!(
"question {}: no keyed option could be identified from the point values, so \
correctness is unknown; scores are still recorded",
@@ -779,6 +843,63 @@ mod tests {
assert_eq!(question.points_possible(), 1.5);
}
fn awarded(score: f64) -> StudentRow {
StudentRow {
sid: Some("1234567".to_string()),
name: None,
email: None,
section: None,
score,
submission_time: None,
marks: Vec::new(),
}
}
#[test]
fn a_zero_point_question_with_no_options_is_a_curve() {
let mut rubric = q(0.0, &[("Curve", 3.6)]);
rubric.rows = vec![awarded(3.6)];
assert!(rubric.looks_like_curve());
assert_eq!(rubric.points_possible(), 3.6);
// No rubric value at all: the points were a per-submission adjustment.
let mut adjusted = q(0.0, &[("Applied", 0.0)]);
adjusted.rows = vec![awarded(3.6)];
assert!(adjusted.looks_like_curve());
let ctx = Context {
course: "BIOSC1000".into(),
term: "2026f".into(),
assessment_id: "e3".into(),
date: None,
form: None,
};
let import = to_responses(&[adjusted], &ctx);
assert!(import.responses.rows[0].is_curve());
assert_eq!(import.responses.curve_total("1234567"), 3.6);
assert!(
import.responses.warnings[0].contains("curve"),
"{:?}",
import.responses.warnings
);
}
#[test]
fn a_bonus_question_with_options_is_not_a_curve() {
let question = q(0.0, &[("A", 0.0), ("B", 1.0)]);
assert!(question.looks_like_bonus());
assert!(!question.looks_like_curve());
}
#[test]
fn an_export_without_point_values_is_not_a_bonus() {
// The maximum defaults to zero when the row is absent; awarded points on
// lettered columns must not turn every question into a bonus.
let mut question = q(0.0, &[("A", 0.0), ("B", 0.0)]);
question.rows = vec![awarded(1.0)];
assert!(!question.looks_like_bonus());
}
#[test]
fn partial_credit_to_a_distractor_flags_ambiguity() {
// Real case: q14, where B earned 1.49 of 1.5.
@@ -854,6 +975,114 @@ mod tests {
std::fs::remove_dir_all(&dir).ok();
}
#[test]
fn reads_the_question_number_from_a_titled_file_name() {
assert_eq!(question_number(Path::new("7.csv")), Some(7));
assert_eq!(question_number(Path::new("39_Curve.csv")), Some(39));
assert_eq!(question_number(Path::new("39_Curve_v2.csv")), Some(39));
assert_eq!(question_number(Path::new("_Curve.csv")), None);
assert_eq!(question_number(Path::new("Curve.csv")), None);
assert_eq!(question_number(Path::new("39_Curve.txt")), None);
assert_eq!(question_number(Path::new("+39.csv")), None);
}
fn context() -> Context {
Context {
course: "BIOSC1000".into(),
term: "2026f".into(),
assessment_id: "e3".into(),
date: None,
form: Some("A".into()),
}
}
const CURVE_HEADER: &str = "Assignment Submission ID,Question Submission ID,First Name,\
Last Name,SID,Email,Sections,Score,Submission Time,Curve,Adjustment,Comments,Grader,Tags\n";
#[test]
fn a_titled_curve_file_reaches_the_totals() {
let dir = std::env::temp_dir().join(format!("cb-gs-curve-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(
dir.join("1.csv"),
"Assignment Submission ID,Question Submission ID,First Name,Last Name,SID,Email,\
Sections,Score,Submission Time,A,B,Adjustment,Comments,Grader,Tags\n\
1,11,Ada,Lovelace,1234567,ada@x.edu,L01,10.0,2026-10-28 10:51:39 -0700,true,false,,,,\"\"\n\
2,12,Alan,Turing,7654321,alan@x.edu,L01,0.0,2026-10-28 10:51:39 -0700,false,true,,,,\"\"\n\
Point Values,,,,,10.0,10.0,0.0\n\
Rubric Numbers,,,,,,1.1,1.2\n\
Scoring Method,positive\n",
)
.unwrap();
// The second student's curve was taken away by an adjustment, which leaves
// a score of zero with the rubric row still marked.
std::fs::write(
dir.join("39_Curve.csv"),
format!(
"{CURVE_HEADER}\
1,21,Ada,Lovelace,1234567,ada@x.edu,L01,4.7,2026-10-28 10:51:39 -0700,true,,,,\"\"\n\
2,22,Alan,Turing,7654321,alan@x.edu,L01,0.0,2026-10-28 10:51:39 -0700,true,-5.0,removed,,\"\"\n\
Point Values,,,,,0.0,4.7\n\
Rubric Numbers,,,,,,39.1\n\
Scoring Method,positive\n"
),
)
.unwrap();
let import = ingest_dir(&dir, &context()).unwrap();
std::fs::remove_dir_all(&dir).ok();
let numbers: Vec<u32> = import.questions.iter().map(|q| q.number).collect();
assert_eq!(numbers, vec![1, 39]);
assert!(import.questions[1].looks_like_curve());
let set = import.responses;
assert_eq!(set.curve_total("1234567"), 4.7);
assert_eq!(set.curve_total("7654321"), 0.0);
assert_eq!(set.points_of_record("1234567"), 14.7);
assert_eq!(set.points_of_record("7654321"), 0.0);
}
#[test]
fn warns_about_a_csv_file_it_did_not_read() {
let dir = std::env::temp_dir().join(format!("cb-gs-unread-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(
dir.join("1.csv"),
format!("{CURVE_HEADER}Point Values,,,,,0.0,4.7\n"),
)
.unwrap();
std::fs::write(dir.join("Curve.csv"), "not a question\n").unwrap();
let import = ingest_dir(&dir, &context()).unwrap();
std::fs::remove_dir_all(&dir).ok();
assert_eq!(import.questions.len(), 1);
assert!(
import
.responses
.warnings
.iter()
.any(|w| w.contains("Curve.csv") && w.contains("not read")),
"{:?}",
import.responses.warnings
);
}
#[test]
fn refuses_two_files_for_one_question() {
let dir = std::env::temp_dir().join(format!("cb-gs-twice-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let curve = format!("{CURVE_HEADER}Point Values,,,,,0.0,4.7\n");
std::fs::write(dir.join("39.csv"), &curve).unwrap();
std::fs::write(dir.join("39_Curve.csv"), &curve).unwrap();
let err = ingest_dir(&dir, &context()).unwrap_err();
std::fs::remove_dir_all(&dir).ok();
assert!(err.to_string().contains("same question"), "{err}");
}
#[test]
fn rejects_a_file_without_the_rubric_delimiter() {
let dir = std::env::temp_dir().join(format!("cb-gs-bad-{}", std::process::id()));
+7 -13
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Reading several forms of one exam back in at once.
@@ -511,20 +511,14 @@ pub fn describe(intake: &FormIntake) -> String {
///
/// # Returns
///
/// `true` when it holds at least one file named like `1.csv`.
/// `true` when it holds at least one file named like `1.csv` or `1_Title.csv`.
pub fn looks_like_export(dir: &Path) -> bool {
let Ok(entries) = std::fs::read_dir(dir) else {
return false;
};
entries.filter_map(|e| e.ok()).any(|entry| {
let path = entry.path();
path.extension().and_then(|e| e.to_str()) == Some("csv")
&& path
.file_stem()
.and_then(|s| s.to_str())
.map(|s| s.chars().all(|c| c.is_ascii_digit()))
.unwrap_or(false)
})
entries
.filter_map(|e| e.ok())
.any(|entry| gradescope::question_number(&entry.path()).is_some())
}
#[cfg(test)]
@@ -532,7 +526,7 @@ mod tests {
use super::*;
#[test]
fn a_labelled_source_parses() {
fn a_labeled_source_parses() {
let source = Source::parse("B=exports/e1-b", None).unwrap();
assert_eq!(source.form, "B");
assert_eq!(source.dir, PathBuf::from("exports/e1-b"));
+139 -7
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The canonical response row.
@@ -161,6 +161,23 @@ impl Response {
!self.bonus && (!self.dropped || self.dropped_full_credit)
}
/// Whether this row is a curve rather than an answer to a question.
///
/// A curve is a zero-point entry on the grading platform that was never
/// printed: bonus, in no placement of the record, and with no option marked.
/// Its points belong on the student's total, but it has nothing to report as
/// a question.
///
/// # Returns
///
/// `true` for a curve row.
pub fn is_curve(&self) -> bool {
self.bonus
&& self.item_ref.is_none()
&& self.selected.is_empty()
&& self.eliminated.is_empty()
}
/// The response coded for a dichotomous model.
///
/// Partial credit is rounded toward the majority: a half-credit response is
@@ -302,15 +319,43 @@ impl ResponseSet {
.sum()
}
/// Bonus points a student earned.
/// Bonus points a student earned on bonus questions.
pub fn bonus_total(&self, key: &str) -> f64 {
self.rows
.iter()
.filter(|r| r.student_key == key && r.bonus && !r.dropped)
.filter(|r| r.student_key == key && r.bonus && !r.dropped && !r.is_curve())
.map(|r| r.score)
.sum()
}
/// Points a curve added to a student's total.
pub fn curve_total(&self, key: &str) -> f64 {
self.rows
.iter()
.filter(|r| r.student_key == key && r.is_curve() && !r.dropped)
.map(|r| r.score)
.sum()
}
/// The total on the grading platform: every point a student earned, bonus
/// and curve included.
///
/// Bonus and curve points sit in the numerator with nothing added to the
/// denominator, which is how a question worth zero points adds up on
/// Gradescope. Divide by [`ResponseSet::points_available`] for the percentage
/// the student can see there.
///
/// # Arguments
///
/// * `key` - the student key.
///
/// # Returns
///
/// The student's total points.
pub fn points_of_record(&self, key: &str) -> f64 {
self.scored_total(key) + self.bonus_total(key) + self.curve_total(key)
}
/// Points available on scored items, taken from the most generous row seen
/// for each item so a student who skipped an item still has a denominator.
pub fn points_available(&self) -> f64 {
@@ -383,7 +428,7 @@ impl ResponseSet {
let items: Vec<u32> = if include_bonus {
self.all_items()
.into_iter()
.filter(|n| !self.item_dropped(*n))
.filter(|n| !self.item_dropped(*n) && !self.item_is_curve(*n))
.collect()
} else {
self.scored_items()
@@ -439,6 +484,14 @@ impl ResponseSet {
any
}
/// Whether every row for a number is a curve, so it is no item at all.
fn item_is_curve(&self, number: u32) -> bool {
self.rows
.iter()
.filter(|r| r.item_number == number)
.all(Response::is_curve)
}
/// Attaches item metadata from an assessment record and the catalog.
///
/// Ingest knows question numbers; only the record knows which item a number
@@ -464,7 +517,10 @@ impl ResponseSet {
for r in &mut self.rows {
let Some(p) = record.placement(r.item_number) else {
unmatched.insert(r.item_number);
// A curve is never in the record, so its absence says nothing.
if !r.is_curve() {
unmatched.insert(r.item_number);
}
continue;
};
r.item_ref = Some(p.item.clone());
@@ -475,7 +531,9 @@ impl ResponseSet {
r.bonus = r.bonus || p.bonus;
r.dropped = r.dropped || p.dropped;
r.dropped_full_credit = r.dropped_full_credit || p.dropped_with_credit();
if let Some(points) = p.points {
// A record that says zero is describing a bonus question the way the
// platform does, and must not erase the points the platform awarded.
if let Some(points) = p.points.filter(|points| *points > 0.0) {
// The record is authoritative for points as administered; the
// export sometimes carries a stale maximum.
if (points - r.points_possible).abs() > 1e-9 && r.points_possible > 0.0 {
@@ -541,6 +599,29 @@ impl ResponseSet {
list.join(", ")
));
}
// A removed drop leaves the report's denominator. If the platform gave
// everyone the points, the question is still in the grade the student
// can see, and the two percentages will disagree.
for p in record
.items
.iter()
.filter(|p| p.dropped && !p.dropped_with_credit())
{
let mut rows = self
.rows
.iter()
.filter(|r| r.item_number == p.number)
.peekable();
if rows.peek().is_some() && rows.all(|r| r.score > 0.0) {
warnings.push(format!(
"question {} is dropped as removed, but the export gives every student points \
for it, so reports will disagree with the platform's percentage; set \
`dropped_as: full_credit` on the placement if it stays in the platform's total",
p.number
));
}
}
self.warnings.extend(warnings.clone());
warnings
}
@@ -977,6 +1058,57 @@ mod tests {
assert_eq!(set.bonus_total("s1"), 2.0);
}
/// A curve row as the Gradescope import writes one.
fn curve(student: &str, number: u32, score: f64) -> Response {
let mut r = row(student, number, 0.0);
r.selected = vec![];
r.correct = None;
r.bonus = true;
r.points_possible = score;
r.score = score;
r
}
#[test]
fn the_total_of_record_adds_bonus_and_curve_without_raising_the_maximum() {
let mut set = ResponseSet::new();
// 1 of 2 and 2 of 2 on the scored questions.
set.rows.push(row("s1", 1, 0.5));
set.rows.push(row("s1", 2, 1.0));
// A bonus question worth 2, answered right.
let mut bonus = row("s1", 3, 1.0);
bonus.bonus = true;
set.rows.push(bonus);
set.rows.push(curve("s1", 4, 3.6));
assert_eq!(set.scored_total("s1"), 3.0);
assert_eq!(set.bonus_total("s1"), 2.0);
assert_eq!(set.curve_total("s1"), 3.6);
assert!((set.points_of_record("s1") - 8.6).abs() < 1e-9);
assert_eq!(set.points_available(), 4.0, "only scored questions");
}
#[test]
fn a_curve_is_not_an_item() {
let mut set = ResponseSet::new();
set.rows.push(row("s1", 1, 1.0));
set.rows.push(curve("s1", 2, 3.6));
assert_eq!(set.matrix(false).items, vec![1]);
assert_eq!(set.matrix(true).items, vec![1]);
}
#[test]
fn an_answered_bonus_question_is_not_a_curve() {
let mut bonus = row("s1", 1, 1.0);
bonus.bonus = true;
assert!(!bonus.is_curve());
// Nor is a blank answer to a bonus question the record knows about.
bonus.selected = vec![];
bonus.item_ref = Some("q-x".into());
assert!(!bonus.is_curve());
}
#[test]
fn dropped_items_leave_the_matrix() {
let mut set = ResponseSet::new();
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Where response data lives on disk.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Parquet reading and writing, isolated so the rest of the crate never touches
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! One error type for the whole crate.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Turning course data into documents.
+879 -35
View File
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Rendering an assessment as a Quarto worksheet a student can work through, and
+3 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Exporting an assessment as a Canvas-importable QTI 1.2 package.
@@ -1263,6 +1263,7 @@ mod tests {
explanation: Some("Only at constant volume.".into()),
hint: Some("Reconsider what stays constant in an open flask.".into()),
misconception: Some("Uses the constant-volume result.".into()),
misconception_id: None,
error_type: None,
defensible: false,
defense: None,
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The bibliography, in the formats other tools read.
+111 -8
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Writing reports for students and for yourself.
@@ -100,13 +100,16 @@ pub fn student(
// --- score
out.push_str(&format!(
"You scored **{:.1} of {:.1} points ({:.0}%)**",
summary.points, summary.points_possible, summary.percent
"You scored **{} of {} points ({:.1}%)**",
points(summary.total_points),
points(summary.points_possible),
summary.percent
));
let mut extras = Vec::new();
if summary.bonus_points > 0.0 {
out.push_str(&format!(
", plus {:.1} bonus point{}",
summary.bonus_points,
extras.push(format!(
"{} bonus point{}",
points(summary.bonus_points),
if (summary.bonus_points - 1.0).abs() < 1e-9 {
""
} else {
@@ -114,6 +117,20 @@ pub fn student(
}
));
}
if summary.curve_points != 0.0 {
extras.push(format!(
"{} curve point{}",
points(summary.curve_points),
if (summary.curve_points - 1.0).abs() < 1e-9 {
""
} else {
"s"
}
));
}
if !extras.is_empty() {
out.push_str(&format!(" including {}", extras.join(" and ")));
}
out.push_str(&format!(
", answering {} of {} questions correctly.\n\n",
summary.correct, summary.n_items
@@ -121,7 +138,7 @@ pub fn student(
if opts.comparison {
out.push_str(&format!(
"The class averaged {:.0}%. Your score is in the {}.\n\n",
"The class averaged {:.1}%. Your score is in the {}.\n\n",
cohort.mean_percent, summary.band
));
}
@@ -319,7 +336,26 @@ pub fn student(
course.text_for(objective)
));
}
// What meeting the target looks like, which is what to
// practice toward.
if let Some(explanation) = course
.learning_targets
.get(target)
.and_then(|t| t.explanation.as_deref())
{
out.push_str(&format!(
"What meeting it looks like: {}\n\n",
course.plain_refs(explanation)
));
}
}
// A named registry misconception carries a student-facing belief
// and correction. The option's own text, when it has one, wins for
// the belief, and keeps the sentence it has always had.
let registry = m
.misconception_id
.as_deref()
.and_then(|id| course.misconceptions.get(id));
if let Some(text) = &m.feedback {
out.push_str(&format!("{text}\n\n"));
} else if let Some(misconception) = &m.misconception {
@@ -327,10 +363,38 @@ pub fn student(
"The option you chose is the one students pick when {misconception}\n\n"
));
}
if let Some(entry) = registry {
if m.misconception.is_none() {
out.push_str(&format!(
"This answer reflects a common mistake: *{}*\n\n",
course.plain_refs(&entry.belief)
));
}
out.push_str(&format!("{}\n\n", course.plain_refs(&entry.correction)));
}
if !m.study.is_empty() {
out.push_str(&format!("Review: {}\n\n", m.study.join("; ")));
}
}
// The same mistake behind more than one answer is worth more than any
// one question: it is the thing to fix once.
let groups = shared_mistakes(&summary.missed);
if !groups.is_empty() {
out.push_str("## The same mistake, more than once\n\n");
for (id, numbers) in groups {
let Some(entry) = course.misconceptions.get(id) else {
continue;
};
let numbers: Vec<String> = numbers.iter().map(u32::to_string).collect();
out.push_str(&format!(
"**Questions {}**: *{}*\n\n{}\n\n",
numbers.join(", "),
course.plain_refs(&entry.belief),
course.plain_refs(&entry.correction)
));
}
}
}
if let Some(closing) = &opts.closing {
@@ -349,6 +413,31 @@ pub fn student(
out
}
/// Registry misconceptions that more than one missed question drew on.
///
/// # Arguments
///
/// * `missed` - a student's missed items.
///
/// # Returns
///
/// Each shared misconception id with the question numbers, in order of the
/// first question that drew it.
pub fn shared_mistakes(missed: &[crate::students::MissedItem]) -> Vec<(&str, Vec<u32>)> {
let mut groups: Vec<(&str, Vec<u32>)> = Vec::new();
for m in missed {
let Some(id) = m.misconception_id.as_deref() else {
continue;
};
match groups.iter_mut().find(|(g, _)| *g == id) {
Some((_, numbers)) => numbers.push(m.number),
None => groups.push((id, vec![m.number])),
}
}
groups.retain(|(_, numbers)| numbers.len() > 1);
groups
}
/// Writes the instructor's report on one administration.
///
/// # Arguments
@@ -684,6 +773,20 @@ pub fn roster(cohort: &Cohort) -> String {
out
}
/// Formats points the way a gradebook shows them, with no trailing zeros.
///
/// # Arguments
///
/// * `value` - the points.
///
/// # Returns
///
/// The points to at most two decimal places.
fn points(value: f64) -> String {
let text = format!("{value:.2}");
text.trim_end_matches('0').trim_end_matches('.').to_string()
}
/// A small unicode bar for a rate in `0.0..=1.0`.
///
/// # Arguments
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Rendering an assessment for a Quarto course website, with solutions gated
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Rendering a printed exam with Typst.
+4 -4
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! What gets emitted, and in what shape.
@@ -277,7 +277,7 @@ pub enum StimulusMode {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Fields {
/// The item's global id, `bank::item`. Useful printed in small grey type on a
/// The item's global id, `bank::item`. Useful printed in small gray type on a
/// review copy; never wanted on a student form.
#[serde(default = "yes")]
pub uid: bool,
@@ -438,7 +438,7 @@ pub struct RenderConfig {
/// Anything else you want the template to see, carried through untouched.
///
/// This is the escape hatch that keeps the crate out of your layout
/// decisions. Tier colours, a `show-solutions` flag, a font stack, a watermark
/// decisions. Tier colors, a `show-solutions` flag, a font stack, a watermark
/// string: put it here and read it from `extra` in the template.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub extra: BTreeMap<String, serde_yaml_ng::Value>,
+75 -4
View File
@@ -1,12 +1,12 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Handing a diagnostic to a Typst template.
//!
//! The same arrangement the exam export uses: this module decides what the
//! template is told, the template decides what it looks like, and the two meet at
//! a marker comment. Nothing here knows about page size or colour.
//! a marker comment. Nothing here knows about page size or color.
//!
//! Two slots are filled, both already defined for the exam path:
//!
@@ -149,7 +149,7 @@ impl Meta {
out.insert("generator", generator);
out.insert("policy", policy);
// `students-tested` is the honest name: it is how many people sat this
// assessment, which is not the enrolment. `class-size` stays as an alias
// assessment, which is not the enrollment. `class-size` stays as an alias
// so a template forked before this change keeps working.
out.insert("students-tested", Value::Int(self.n_students as i64));
out.insert("class-size", Value::Int(self.n_students as i64));
@@ -196,8 +196,26 @@ pub fn student_value(diagnostic: &StudentDiagnostic, config: &RenderConfig) -> V
let mut score = Value::dict();
score.insert("points", Value::Float(diagnostic.score.points));
score.insert("possible", Value::Float(diagnostic.score.points_possible));
score.insert("total", Value::Float(diagnostic.score.total_points));
score.insert("percent", Value::Float(diagnostic.score.percent));
score.insert("bonus", Value::Float(diagnostic.score.bonus_points));
score.insert("curve", Value::Float(diagnostic.score.curve_points));
score.insert(
"bonus-questions",
Value::Array(
diagnostic
.score
.bonus_questions
.iter()
.map(|question| {
let mut value = Value::dict();
value.insert("number", Value::Int(question.number as i64));
value.insert("points", Value::Float(question.points));
value
})
.collect(),
),
);
score.insert("correct", Value::Int(diagnostic.score.correct as i64));
score.insert("items", Value::Int(diagnostic.score.n_items as i64));
out.insert("score", score);
@@ -350,6 +368,12 @@ pub fn student_value(diagnostic: &StudentDiagnostic, config: &RenderConfig) -> V
.map(|text| markup_value(text, content)),
);
pair.insert("target", markup_value(&m.target, content));
pair.insert_some(
"explanation",
m.explanation
.as_ref()
.map(|text| markup_value(text, content)),
);
pair
})
.collect(),
@@ -363,6 +387,22 @@ pub fn student_value(diagnostic: &StudentDiagnostic, config: &RenderConfig) -> V
] {
value.insert_some(key, text.map(|t| markup_value(t, content)));
}
value.insert_some(
"mistake",
question.mistake.as_ref().map(|mistake| {
let mut entry = Value::dict();
entry.insert("id", Value::str(&mistake.id));
entry.insert_some(
"belief",
mistake
.belief
.as_ref()
.map(|text| markup_value(text, content)),
);
entry.insert("correction", markup_value(&mistake.correction, content));
entry
}),
);
value.insert(
"taught-in",
Value::Array(question.taught_in.iter().map(Value::str).collect()),
@@ -392,6 +432,33 @@ pub fn student_value(diagnostic: &StudentDiagnostic, config: &RenderConfig) -> V
),
);
out.insert(
"mistakes",
Value::Array(
diagnostic
.mistakes
.iter()
.map(|group| {
let mut entry = Value::dict();
entry.insert("id", Value::str(&group.id));
entry.insert("belief", markup_value(&group.belief, content));
entry.insert("correction", markup_value(&group.correction, content));
entry.insert(
"questions",
Value::Array(
group
.questions
.iter()
.map(|n| Value::Int(*n as i64))
.collect(),
),
);
entry
})
.collect(),
),
);
let dropped_questions = if config.student_sections.dropped_questions {
diagnostic.dropped_questions.as_slice()
} else {
@@ -546,6 +613,8 @@ pub fn cohort_value(diagnostic: &CohortDiagnostic, config: &RenderConfig) -> Val
Value::Array(diagnostic.distribution.bins.iter().map(bin_value).collect()),
);
out.insert("distribution", distribution);
out.insert("mean-bonus", Value::Float(diagnostic.mean_bonus));
out.insert("mean-curve", Value::Float(diagnostic.mean_curve));
let mut reliability = Value::dict();
reliability.insert_some("alpha", diagnostic.reliability.alpha.map(Value::Float));
@@ -1246,6 +1315,8 @@ mod tests {
fn empty_cohort() -> CohortDiagnostic {
CohortDiagnostic {
mean_bonus: 0.0,
mean_curve: 0.0,
n_students: 24,
n_items: 36,
distribution: Distribution {
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The data a template is given.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Loading a Typst file and splicing generated data into it.
+17 -4
View File
@@ -467,6 +467,19 @@
#pad(right: prose-pad)[#text(size: size-lead, fill: luma(90))[#rel.interpretation]]
]
#let mean-bonus = cb-data.at("mean-bonus", default: 0.0)
#let mean-curve = cb-data.at("mean-curve", default: 0.0)
#if mean-bonus != 0.0 or mean-curve != 0.0 [
#block(above: entry-gap)[
#pad(right: prose-pad)[#text(size: size-lead, fill: luma(90))[
These percentages match the gradebook, so they include an average of
#num(mean-bonus, digits: 2) bonus and #num(mean-curve, digits: 2) curve
points per student. Reliability and the item statistics do not.
]]
]
]
// ─────────────────────────────────────────────────────────────────────────────
// Where the class landed
// ─────────────────────────────────────────────────────────────────────────────
@@ -898,7 +911,7 @@
#explain[
Where the class sits on each kind of thinking. A drop from one level to the
next is expected; a cliff is worth reading as a gap in what was practised
next is expected; a cliff is worth reading as a gap in what was practiced
rather than in what was taught.
]
@@ -1039,7 +1052,7 @@
#explain[
One row per question, in the order they were numbered. Option letters are the
bank's, not any one form's. See the guide above for what p, r, and D mean; the
r column is colour-coded against the conventional cut points.
r column is color-coded against the conventional cut points.
#{
let dropped = cb-data.at("dropped-questions", default: ())
if dropped.len() > 0 [
@@ -1133,7 +1146,7 @@
// healthy item looks like on this exam, and the place to check a query about a
// question nothing was wrong with. The flagged ones are already called out by
// name in the triage lists above, so nothing is lost by putting them back among
// their neighbours here.
// their neighbors here.
// Dropped items come from their own list, since they carry no statistics and
// belong in none of the tables above. They belong here: the option spread is
// the evidence that justified dropping them, and re-running the report after a
@@ -1328,7 +1341,7 @@
None of these #n #plural(n, "expectation", "expectations") rests on prior
data, so they are predictions rather than calibrations. A prediction that
misses is a fact about the prediction: it does not flag the item, and it is
summarised here instead of appearing #n times in the tables above. Once
summarized here instead of appearing #n times in the tables above. Once
`coursebank calibrate` has written statistics back into the bank, a
subsequent miss means the cohort or the teaching moved, and it will be
flagged.
+2 -4
View File
@@ -261,10 +261,8 @@
#if show-name-block [
#v(1.5em)
I agree to follow the above instructions. I affirm that all work on this
assessment will be my own and that I will not give or receive any
unauthorized assistance. To have your assessment graded, you must write your
name, sign, and provide your student ID below.
I affirm that all work on this assessment will be my own and that I will not give or receive any unauthorized assistance.
To have your assessment graded, you must write your name, sign, and provide your student ID below.
#v(1em)
+176 -28
View File
@@ -57,7 +57,17 @@
name: "Sample Student",
email: "sample@example.edu",
form: "A",
score: (points: 27.0, possible: 36.0, percent: 75.0, bonus: 1.0, correct: 27, items: 36),
score: (
points: 27.0,
possible: 36.0,
total: 31.6,
percent: 87.8,
bonus: 1.0,
curve: 3.6,
bonus-questions: ((number: 37, points: 1.0), (number: 38, points: 0.0)),
correct: 27,
items: 36,
),
standing: (class-mean: 71.2, class-sd: 11.4, band: "upper half"),
levels: (
(
@@ -132,10 +142,13 @@
number: 2,
level: 3,
targets: ("t-sample-gap",),
measured: ((
objective: [A sample objective to work on.],
target: [A sample learning target under it.],
),),
measured: (
(
objective: [A sample objective to work on.],
target: [A sample learning target under it.],
explanation: [What a student who has met the target could say about it.],
),
),
correct: false,
credit: 0.0,
bonus: false,
@@ -143,9 +156,60 @@
class-rate: 0.58,
feedback: [This is the note written for the option that was chosen.],
hint: [This is the question you would ask someone reconsidering that option.],
mistake: (
id: "mis-sample",
belief: [A wrong idea, as a student would hold it.],
correction: [Why it fails, and what is true instead.],
),
taught-in: ("Enthalpy (L1.1), slides 12, 13",),
review: ((citation: "KKW §6.2", title: "Molecules and Medicine", url: "https://example.edu/6/2"),),
),
(
number: 14,
level: 2,
targets: ("t-sample-gap",),
measured: ((target: [A sample learning target under it.],),),
correct: false,
credit: 0.0,
bonus: false,
blank: false,
class-rate: 0.61,
mistake: (id: "mis-sample", correction: [Why it fails, and what is true instead.]),
taught-in: (),
review: (),
),
(
number: 37,
level: 1,
targets: (),
measured: (),
correct: true,
credit: 1.0,
bonus: true,
blank: false,
taught-in: (),
review: (),
),
(
number: 38,
level: 1,
targets: (),
measured: (),
correct: false,
credit: 0.0,
bonus: true,
blank: false,
taught-in: (),
review: (),
),
),
mistakes: (
(
id: "mis-sample",
belief: [A wrong idea, as a student would hold it.],
correction: [Why it fails, and what is true instead.],
questions: (2, 14),
),
),
dropped-questions: ((number: 35, full-credit: true),),
review-lectures: (
@@ -217,7 +281,7 @@
#let prose-pad = 1.5cm
// The same right edge for prose that already sits in a gutter, so a note body
// lines up with the explanation above it. One centimetre is the 2.8em of number
// lines up with the explanation above it. One centimeter is the 2.8em of number
// column plus gutter at the default body size.
#let prose-pad-inset = prose-pad - 1cm
@@ -275,7 +339,8 @@
// A rate is a fraction in 0..1; a percent is already out of 100. Keeping the two
// straight is the only arithmetic this template does.
#let pct(rate) = str(calc.round(rate * 100)) + "%"
#let pct1(value) = str(calc.round(value, digits: 0)) + "%"
#let pct1(value) = str(calc.round(value, digits: 1)) + "%"
#let pts(value) = str(calc.round(value, digits: 2))
#let plural(n, one, many) = if n == 1 { one } else { many }
@@ -410,9 +475,12 @@
)[
#pad(right: prose-pad - 1.1cm)[
#text(size: size-lead)[
This map explains what the exam covered and what your results mean, so you can use the information. Your score appears first because that's what you probably want to see. After that, you'll find more helpful details: which types of questions you did well on, which topics you might want to review, and what to read for each area. The last two sections are the most practical, so if you only read part of this, focus on those.
This map explains what the exam covered and what your results mean, so you can use the information.
Your score appears first because you probably want to see it.
After that, you'll find helpful details: which question types you did well on, which topics to review, and what to read for each area.
The last two sections are the most practical, so if you read only part of this, focus on those.
This report does not judge your abilities, and one question alone does not say much. If a score is based on just one or two questions, the report will point that out so you don't read too much into it.
This report doesn't judge your abilities, and one question alone doesn't say much.
]
]
]
@@ -431,11 +499,7 @@
stat-card(
"your score",
pct1(score.percent),
note: str(score.points)
+ " of "
+ str(score.possible)
+ " points"
+ (if score.at("bonus", default: 0.0) > 0.0 { " (+" + str(score.bonus) + " bonus)" } else { "" }),
note: pts(score.at("total", default: score.points)) + " of " + pts(score.possible) + " points",
),
stat-card(
"questions right",
@@ -457,6 +521,37 @@
},
)
// The arithmetic behind the score, so it can be checked against the gradebook.
// Bonus and curve points are worth zero on the platform: they raise the total and
// leave the denominator alone, which is why a score can pass 100%.
#let bonus-questions = score.at("bonus-questions", default: ())
#let curve-points = score.at("curve", default: 0.0)
#if bonus-questions.len() > 0 or curve-points != 0.0 [
#block(above: entry-gap)[
#table(
columns: (1fr, auto),
stroke: none,
inset: (x: 0pt, y: 3pt),
align: (left, right),
[Questions], [#pts(score.points) of #pts(score.possible)],
..if bonus-questions.len() > 0 {
(
[Bonus #plural(bonus-questions.len(), "question", "questions")
#bonus-questions.map(q => str(q.number)).join(", ")],
[+#pts(score.at("bonus", default: 0.0))],
)
} else { () },
..if curve-points != 0.0 {
([Curve], [#if curve-points > 0.0 [+]#pts(curve-points)])
} else { () },
table.hline(stroke: 0.5pt + luma(180)),
[*Final score*],
[*#pts(score.at("total", default: score.points)) of #pts(score.possible) (#pct1(score.percent))*],
)
]
]
#let dropped-questions = cb-data.at("dropped-questions", default: ())
#if dropped-questions.len() > 0 [
@@ -473,7 +568,7 @@
if credited.len() > 0 [
#plural(credited.len(), "Question", "Questions") #numbers(credited)
#plural(credited.len(), "was", "were") thrown out after the exam.
#plural(credited.len(), "was", "were") dropped after the exam.
Everyone received full credit for
#plural(credited.len(), "it", "them"), so
#plural(credited.len(), "it is", "they are") still counted in the
@@ -796,11 +891,19 @@
#let question-box(q) = {
let dropped = q.at("dropped", default: false)
let color = if dropped { luma(130) } else if q.at("blank", default: false) {
let bonus = q.at("bonus", default: false)
let credit = q.at("credit", default: 0.0)
// A bonus question can only add points, so one that earned nothing is not
// marked as an error, and its color follows the points it actually earned.
let color = if dropped { luma(130) } else if bonus and credit <= 0.0 {
luma(160)
} else if q.at("correct", default: false) == true {
} else if q.at("blank", default: false) {
luma(160)
} else if bonus and credit >= 0.999 {
ok-color
} else if q.at("credit", default: 0.0) > 0.0 { mid-color } else { bad-color }
} else if not bonus and q.at("correct", default: false) == true {
ok-color
} else if credit > 0.0 { mid-color } else { bad-color }
box(
width: 100%,
fill: color.lighten(85%),
@@ -814,6 +917,9 @@
if dropped {
linebreak()
text(size: size-tag, fill: luma(110))[out]
} else if bonus {
linebreak()
text(size: size-tag, fill: luma(110))[bonus]
} else {
let level = q.at("level", default: none)
if level != none {
@@ -848,6 +954,9 @@
#box(width: 0.7em, height: 0.7em, fill: bad-color.lighten(70%), radius: 2pt) not right ·
#box(width: 0.7em, height: 0.7em, fill: luma(210), radius: 2pt) left blank ·
#box(width: 0.7em, height: 0.7em, fill: luma(150), radius: 2pt) dropped, not scored
#if questions.any(q => q.at("bonus", default: false)) [
· a box marked *bonus* can only add points, so a gray one cost you nothing
]
]
]
]
@@ -864,7 +973,7 @@
let parts = ()
// Both tiers. The target is what this question actually asked, which is the
// thing to practise; the objective is the row it counted toward in the table
// thing to practice; the objective is the row it counted toward in the table
// above, which is how a student tells whether one slip cost them a claim. The
// objective is absent when the two would be the same sentence.
let measured = q.at("measured", default: ())
@@ -879,17 +988,24 @@
let target = [
#text(weight: "bold")[This question asked you to:] #markup(m.target)
]
if objective == none {
// What meeting the target looks like: the account to practice
// toward, from the target's own explanation.
let explanation = m.at("explanation", default: none)
let lines = (target,)
if explanation != none {
lines.push(text(size: size-small)[
#text(weight: "bold")[What meeting it looks like:] #markup(explanation)
])
}
if objective != none {
lines.push(text(size: size-small, fill: luma(110))[
Counts toward: #markup(objective)
])
}
if lines.len() == 1 {
target
} else {
stack(
dir: ttb,
spacing: step * 0.3,
target,
text(size: size-small, fill: luma(110))[
Counts toward: #markup(objective)
],
)
stack(dir: ttb, spacing: step * 0.3, ..lines)
}
}),
)
@@ -901,6 +1017,18 @@
let feedback = q.at("feedback", default: none)
if feedback != none { parts.push(markup(feedback)) }
// A named mistake from the course's registry, written for the student: the
// belief (absent when the option's own text already said it) and why it
// fails.
let mistake = q.at("mistake", default: none)
if mistake != none {
let belief = mistake.at("belief", default: none)
parts.push(callout("A common mistake", {
if belief != none [#emph(markup(belief)) ]
markup(mistake.correction)
}))
}
let hint = q.at("hint", default: none)
if hint != none { parts.push(callout("Try this", markup(hint))) }
@@ -964,7 +1092,9 @@
q.at("feedback", default: none) != none
or q.at("hint", default: none) != none
or q.at("worked", default: none) != none
or q.at("mistake", default: none) != none
or q.at("review", default: ()).len() > 0
or q.at("measured", default: ()).any(m => m.at("explanation", default: none) != none)
)
))
@@ -982,6 +1112,24 @@
]
]
// The same registry mistake behind more than one missed question: the thing
// to fix once, with the questions it cost.
#let mistakes = cb-data.at("mistakes", default: ())
#if show-feedback and mistakes.len() > 0 [
= The same mistake, more than once
#for group in mistakes [
#block(breakable: false, above: entry-gap, width: 100%)[
#pad(right: prose-pad-inset)[#stack(
dir: ttb,
spacing: step,
[#text(weight: "bold")[Questions #group.questions.map(str).join(", "):] #emph(markup(group.belief))],
markup(group.correction),
)]
]
]
]
// ─────────────────────────────────────────────────────────────────────────────
// Footer note
// ─────────────────────────────────────────────────────────────────────────────
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! A small writer for Typst literals.
+13 -7
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Long-form documentation: setup, authoring, and worked tutorials.
@@ -16,14 +16,16 @@
//!
//! 1. [`setup`] builds a course directory from nothing and explains what each file
//! is for.
//! 2. [`authoring`] writes items, with the design fields that make an item worth
//! 2. [`glossary`] adds the terms, equations, values, and misconceptions a
//! course's targets turn on, and the lecture pages built from them.
//! 3. [`authoring`] writes items, with the design fields that make an item worth
//! reusing.
//! 3. [`first_exam`] runs one exam end to end: assemble, export, administer,
//! 4. [`first_exam`] runs one exam end to end: assemble, export, administer,
//! ingest, analyze, report.
//! 4. [`typst_export`] covers printed output, template markers, and render config.
//! 5. [`assignment`] publishes a homework to the course website, with solutions
//! 5. [`typst_export`] covers printed output, template markers, and render config.
//! 6. [`assignment`] publishes a homework to the course website, with solutions
//! gated behind a password.
//! 6. [`recipes`] holds short answers to specific questions, for when you already
//! 7. [`recipes`] holds short answers to specific questions, for when you already
//! know the shape of the tool.
//!
//! ## Why the tutorials are in here rather than a wiki
@@ -45,6 +47,10 @@
#[doc = include_str!("../docs/guide/setup.md")]
pub mod setup {}
/// Terms, equations, values, and misconceptions, and the pages built from them.
#[doc = include_str!("../docs/guide/glossary.md")]
pub mod glossary {}
/// Writing items that are worth keeping.
#[doc = include_str!("../docs/guide/authoring.md")]
pub mod authoring {}
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Plumbing shared by more than one command handler.
+16 -11
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! # coursebank
@@ -20,13 +20,14 @@
//! |:--|:--|:--|
//! | `course.yaml` | identity, policy, units | you |
//! | `references.yaml` | the works the course cites | you |
//! | `lectures/*.yaml` | one lecture: readings, and what it teaches | you |
//! | `objectives/*.yaml` | one objective and its learning targets | you |
//! | `glossary.yaml` | terms, equations, values, and misconceptions no one objective owns | you, or `migrate glossary` |
//! | `lectures/*.yaml` | one lecture: readings, materials, and what it teaches | you |
//! | `objectives/*.yaml` | one objective, its learning targets, and the glossary entries they first need | you |
//! | `banks/*.yaml` | items, with design intent and pooled statistics | you, then `calibrate` |
//! | `assessments/*.yaml` | what was given, to whom, when | `assemble`, then you |
//! | `data/*.parquet` | one row per student per item | `ingest` |
//!
//! The first four are one course file split by subject; `coursebank course build`
//! The first five are one course file split by subject; `coursebank course build`
//! prints the merged result, and a course that keeps everything in `course.yaml`
//! still loads unchanged. See [`course::fragment`].
//!
@@ -69,18 +70,20 @@
//! what must be true; [`lint`] advises on what is usually a mistake, and every rule
//! has a code you can silence.
//!
//! Small samples are labelled as such. Every statistic computed from a class of
//! Small samples are labeled as such. Every statistic computed from a class of
//! twenty-five is reported with the caveat it deserves rather than three decimal
//! places of false precision.
//!
//! ## Dependency posture
//!
//! Deliberately small: serde, a YAML parser, clap, thiserror, and csv, plus arrow
//! and parquet behind a default-on feature that can be switched off. Dates, PRNG,
//! Deliberately small: serde, a YAML parser, clap, thiserror, csv, and regex
//! (built without its optional features, for the glossary's `conflicts` and
//! `target_text_forbid` patterns), plus arrow and parquet behind a default-on
//! feature that can be switched off. Dates, PRNG,
//! hashing, ZIP writing, and the psychometrics are implemented here rather than
//! pulled in — see [`date`], [`rng`], [`hash`], [`zipfile`], [`irt`]. For a tool
//! whose job is to still open a course repository in five years, that tradeoff
//! favours fewer moving parts.
//! favors fewer moving parts.
//!
//! ## Where to start
//!
@@ -106,9 +109,11 @@ pub mod migrate;
pub mod model;
pub mod util;
pub use util::{date, hash, markup, rng, yaml, zipfile};
pub use util::{citation, date, hash, markup, rng, yaml, zipfile};
pub use model::{assessment, bank, catalog, course, history, item, layout, seal, taxonomy};
pub use model::{
assessment, bank, calibration, catalog, course, history, item, layout, seal, taxonomy,
};
pub use course::fragment;
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The `coursebank` command-line interface.
+640 -7
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! One-time conversions from one layout to the next.
@@ -19,6 +19,10 @@
//! | [`options_plan`] and [`apply_options`] | option letters into names | a letter is a position, not an identity |
//! | [`stems`] | drops `version:` and `history:` | a stem's text is its identity |
//! | [`store_variants`] | fills the stored `variant` column | so other readers can group by option set |
//! | [`references`] | prose notes into citation fields | a DOI in a note is a link nobody can follow |
//! | [`order`] | removes the `order:` integers | objective order is derived, and targets have none |
//! | [`counters`] | drops the `-001` from item ids | a per-bank sequence number is not part of a name |
//! | [`glossary()`] | 1.0 `terms.yaml` and `values.yaml` into `glossary.yaml` | an entry is defined once, by id, where targets can point at it |
//!
//! # What the option migration does not touch
//!
@@ -71,6 +75,9 @@ use crate::layout::Layout;
use crate::store::{self, Store};
use crate::yaml;
mod glossary;
pub use glossary::{GlossaryMigration, glossary};
/// The width past which a flow sequence is written as a block list instead.
const FLOW_WIDTH: usize = 96;
@@ -96,7 +103,7 @@ pub struct Plan {
impl Plan {
/// The number of lines each planned file holds.
pub fn lines(&self) -> Vec<(PathBuf, usize)> {
pub fn lines(&self) -> Report {
self.files
.iter()
.map(|f| (f.path.clone(), f.text.lines().count()))
@@ -602,7 +609,7 @@ fn unqualify_line(line: &str) -> String {
///
/// Propagates read and write failures, including
/// [`Error::FeatureDisabled`] when the store is Parquet and the feature is off.
pub fn store_ids(root: &Path, write: bool) -> Result<Vec<(PathBuf, usize)>> {
pub fn store_ids(root: &Path, write: bool) -> Result<Report> {
let layout = Layout::new(root);
let store = Store::open(layout.data())?;
let mut out = Vec::new();
@@ -629,6 +636,15 @@ pub fn store_ids(root: &Path, write: bool) -> Result<Vec<(PathBuf, usize)>> {
Ok(out)
}
/// One file a migration changed, and how many things in it changed.
pub type Changed = (PathBuf, usize);
/// What a migration changed, file by file.
pub type Report = Vec<Changed>;
/// Old id to new id, for the migrations that rename things.
pub type Renames = BTreeMap<String, String>;
/// The per-item map from a pre-2.0 option letter to the name that replaces it.
pub type OptionMap = BTreeMap<String, BTreeMap<String, String>>;
@@ -844,7 +860,7 @@ pub fn options_plan(root: &Path) -> Result<(OptionMap, Vec<String>)> {
/// # Errors
///
/// Propagates read and write failures.
pub fn apply_options(root: &Path, map: &OptionMap, write: bool) -> Result<Vec<(PathBuf, usize)>> {
pub fn apply_options(root: &Path, map: &OptionMap, write: bool) -> Result<Report> {
let layout = Layout::new(root);
let mut out = Vec::new();
@@ -1132,7 +1148,7 @@ fn key_at_word(line: &str, key: &str) -> Option<usize> {
/// # Errors
///
/// Propagates read and write failures.
pub fn stems(root: &Path, write: bool) -> Result<Vec<(PathBuf, usize)>> {
pub fn stems(root: &Path, write: bool) -> Result<Report> {
let layout = Layout::new(root);
let mut out = Vec::new();
@@ -1221,7 +1237,7 @@ fn is_key(trimmed: &str, key: &str) -> bool {
/// # Errors
///
/// Propagates catalog, record, and store failures.
pub fn store_variants(root: &Path, write: bool) -> Result<Vec<(PathBuf, usize)>> {
pub fn store_variants(root: &Path, write: bool) -> Result<Report> {
let layout = Layout::new(root);
let catalog = crate::catalog::Catalog::load(root)?;
@@ -1265,6 +1281,475 @@ pub fn store_variants(root: &Path, write: bool) -> Result<Vec<(PathBuf, usize)>>
Ok(out)
}
/// Drops the trailing counter from every item id.
///
/// `q-fastq-quality-line-001` becomes `q-fastq-quality-line`. The counter was a
/// per-bank sequence number, which says when an item was written — something git
/// knows — and collides with another bank's numbering the moment an item moves.
///
/// This is a rename, not a normalization, and that distinction decides how it
/// has to be done. Stripping `bank::` from an id recovered a name that was
/// already inside it, so an old reference could simply be read as the new one.
/// A counter carries no such fallback: `q-x-001` and `q-x` are two unrelated
/// strings, and every file that names one has to be rewritten in the same pass
/// or the join to four terms of response data quietly breaks.
///
/// So: banks, assessment records, seals, and the response store, or nothing.
/// Two ids that would collide after stripping abort the whole migration rather
/// than merging two questions into one.
///
/// # Arguments
///
/// * `root` - the course directory.
/// * `write` - whether to write, or only to count.
///
/// # Returns
///
/// The renames, and one entry per file that changes with how many references it
/// holds.
///
/// # Errors
///
/// Returns [`Error::Invalid`] when two ids would collide, listing both, and
/// propagates load and write failures.
pub fn counters(root: &Path, write: bool) -> Result<(Renames, Report)> {
let layout = Layout::new(root);
let catalog = crate::catalog::Catalog::load(root)?;
let mut rename: Renames = BTreeMap::new();
let mut taken: Renames = BTreeMap::new();
let mut collisions = Vec::new();
for entry in &catalog.entries {
let Some(bare) = strip_counter(&entry.uid) else {
taken.insert(entry.uid.clone(), entry.uid.clone());
continue;
};
if let Some(first) = taken.get(&bare) {
collisions.push(format!(
"`{}` and `{first}` would both become `{bare}`. Give one of them a name that \
says what it asks rather than when it was written.",
entry.uid
));
continue;
}
taken.insert(bare.clone(), entry.uid.clone());
rename.insert(entry.uid.clone(), bare);
}
if !collisions.is_empty() {
return Err(Error::Invalid(collisions));
}
if rename.is_empty() {
return Ok((rename, Vec::new()));
}
let mut touched = Vec::new();
// Banks: the definition, and any `supersedes` pointing at one.
for path in yaml::list_yaml(&layout.banks())? {
let text = std::fs::read_to_string(&path).map_err(|e| Error::io(&path, e))?;
let (out, n) = rename_ids(&text, &rename, &["- id", "supersedes"]);
if n > 0 {
if write {
yaml::write_text(&path, &out)?;
}
touched.push((relative(root, &path), n));
}
}
// Records: the `item:` on every placement.
for path in yaml::list_yaml(&layout.assessments())? {
let text = std::fs::read_to_string(&path).map_err(|e| Error::io(&path, e))?;
let (out, n) = rename_ids(&text, &rename, &["item"]);
if n > 0 {
if write {
yaml::write_text(&path, &out)?;
}
touched.push((relative(root, &path), n));
}
}
// Seals go through the model, not the text, because the ids are inside the
// digest and a seal that has been rewritten has to say so.
for path in yaml::list_yaml(&layout.seals())? {
let mut seal = crate::seal::SealFile::load(&path)?;
let n = seal.rename_items(&rename);
if n > 0 {
if write {
seal.save(&path)?;
}
touched.push((relative(root, &path), n));
}
}
// The store, where the id is the join key.
let store = Store::open(layout.data())?;
for path in store.files()? {
let mut rows = store::read_flat(&path)?;
let mut n = 0;
for row in &mut rows {
if let Some(new) = rename.get(&row.item_ref) {
row.item_ref = new.clone();
n += 1;
}
}
if n > 0 {
if write {
store::write_flat(&path, &rows)?;
}
touched.push((relative(root, &path), n));
}
}
Ok((rename, touched))
}
/// An id with a trailing `-123` removed, or `None` when it has none.
fn strip_counter(id: &str) -> Option<String> {
let (head, tail) = id.rsplit_once('-')?;
if head.is_empty() || tail.is_empty() || !tail.chars().all(|c| c.is_ascii_digit()) {
return None;
}
Some(head.to_string())
}
/// Rewrites the value of the named keys wherever it is an id being renamed.
///
/// Keyed on the field name so that an id appearing in prose — a rationale that
/// mentions the item it replaced, a note — is left alone. A rename that edited
/// every matching string in the file would also edit the sentences about it.
fn rename_ids(text: &str, rename: &Renames, keys: &[&str]) -> (String, usize) {
let mut out: Vec<String> = Vec::new();
let mut count = 0;
for line in text.lines() {
let mut replaced = None;
for key in keys {
let Some(at) = field_value(line, key) else {
continue;
};
let value = &line[at.0..at.1];
// Canonicalized before the lookup, so a record that still carries a
// pre-2.0 `bank::` qualifier is renamed rather than skipped. Left
// alone it would keep pointing at an id the bank no longer has.
let Some(new) = rename.get(crate::item::canonical_id(value)) else {
continue;
};
replaced = Some(format!("{}{new}{}", &line[..at.0], &line[at.1..]));
count += 1;
break;
}
out.push(replaced.unwrap_or_else(|| line.to_string()));
}
let mut joined = out.join("\n");
if text.ends_with('\n') {
joined.push('\n');
}
(joined, count)
}
/// The byte range of a `key: value` scalar on one line, quotes excluded.
fn field_value(line: &str, key: &str) -> Option<(usize, usize)> {
let needle = format!("{key}:");
let at = line.find(&needle)?;
let before = &line[..at];
if before.chars().next_back().is_some_and(|c| {
c.is_ascii_alphanumeric() || c == '_' || (c == '-' && !key.starts_with('-'))
}) {
return None;
}
let rest = &line[at + needle.len()..];
let lead = rest.len() - rest.trim_start().len();
let value = rest.trim_start();
let quoted = value.starts_with(['"', '\'']);
let start = at + needle.len() + lead + usize::from(quoted);
let inner = &line[start..];
let end = inner
.find(|c: char| {
c == '"' || c == '\'' || c == ',' || c == '}' || c == ']' || c.is_whitespace()
})
.map(|n| start + n)
.unwrap_or(line.len());
(end > start).then_some((start, end))
}
/// Removes the hand-kept `order:` integers.
///
/// Nothing is written in their place. An objective's teaching order is its
/// position in a lecture's `teaches` list, which `migrate split` already
/// produced. A target has no order at all — an objective's targets are a set of
/// question templates, sampled from rather than worked through — so the integer
/// was asserting a sequence that was never taught. Where a list has to be
/// printed, [`crate::course::CourseFile::targets`] orders it by ceiling.
///
/// Two hundred and four integers in a real course, each of which had to be
/// bumped by hand when a target was inserted in the middle.
///
/// # Arguments
///
/// * `root` - the course directory.
/// * `write` - whether to write, or only to count.
///
/// # Returns
///
/// One entry per file that changes, with how many `order:` lines it loses.
///
/// # Errors
///
/// Propagates catalog and write failures, and refuses when the lectures do not
/// declare what they teach, since objective order would then have no source.
pub fn order(root: &Path, write: bool) -> Result<Report> {
let layout = Layout::new(root);
let course = CourseFile::load_dir(root)?;
if course.lectures.values().all(|l| l.teaches.is_empty())
&& !course.learning_objectives.is_empty()
{
return Err(Error::usage(
"no lecture declares what it teaches, so objective order would have nothing to come from. Run `coursebank migrate split` first."
.to_string(),
));
}
let mut out = Vec::new();
let mut files = vec![layout.course_file()];
files.extend(yaml::list_yaml(&layout.objectives())?);
for path in files {
if !path.is_file() {
continue;
}
let text = std::fs::read_to_string(&path).map_err(|e| Error::io(&path, e))?;
let (rewritten, dropped) = rewrite_order(&text);
if rewritten == text {
continue;
}
if write {
yaml::write_text(&path, &rewritten)?;
}
out.push((relative(root, &path), dropped));
}
Ok(out)
}
/// Removes every `order:` line.
///
/// Nothing replaces them. An objective's position comes from where its lecture
/// lists it in `teaches`, and a target has no position to derive: an
/// objective's targets are a set of question templates rather than steps in a
/// sequence, so the integer was asserting an order that was never taught.
fn rewrite_order(text: &str) -> (String, usize) {
let mut out: Vec<&str> = Vec::new();
let mut dropped = 0;
for line in text.lines() {
if key_at(line, 4).as_deref() == Some("order") {
dropped += 1;
continue;
}
out.push(line);
}
let mut joined = out.join("\n");
if text.ends_with('\n') {
joined.push('\n');
}
(joined, dropped)
}
/// Turns the citations written into `note:` fields into real fields.
///
/// A hand-built bibliography collects entries whose journal, volume, pages, and
/// DOI are all sitting in the one field that means "anything else worth saying".
/// Nothing can use them there: a reading list cannot link a DOI it cannot see,
/// and an export to Hayagriva or BibTeX has no journal to put in `parent` or
/// `journal`.
///
/// Textual, like the rest of this module, and for a reason specific to this
/// file: a bibliography is usually ordered the way its author thinks about it —
/// books, then the papers that matter — and the model holds it in a map, so a
/// serde round trip would alphabetize all of it to change twenty entries.
///
/// A field the entry already declares is never overwritten. Where the note
/// disagrees with it, the note's version is left in place for a person to look
/// at rather than silently replacing something that was typed deliberately.
///
/// # Arguments
///
/// * `root` - the course directory.
/// * `write` - whether to write, or only to count.
///
/// # Returns
///
/// One entry per file that changes, with how many notes were taken apart, and
/// one message per disagreement found.
///
/// # Errors
///
/// Propagates read and write failures.
pub fn references(root: &Path, write: bool) -> Result<(Report, Vec<String>)> {
let layout = Layout::new(root);
let mut touched = Vec::new();
let mut notes = Vec::new();
// Either file may hold the bibliography: its own, or the course file that
// has not been split yet.
for path in [layout.references_file(), layout.course_file()] {
if !path.is_file() {
continue;
}
let text = std::fs::read_to_string(&path).map_err(|e| Error::io(&path, e))?;
let (rewritten, count, said) = expand_notes(&text);
notes.extend(said);
if count > 0 {
if write {
yaml::write_text(&path, &rewritten)?;
}
touched.push((relative(root, &path), count));
}
}
Ok((touched, notes))
}
/// Replaces each reference's `note:` line with the fields it was carrying.
fn expand_notes(text: &str) -> (String, usize, Vec<String>) {
let lines: Vec<String> = text.lines().map(str::to_string).collect();
let Some(section) = blocks(&lines, 0)
.into_iter()
.find(|b| b.key == "references")
else {
return (text.to_string(), 0, Vec::new());
};
// Which lines belong to which entry, so a note is parsed with its own year
// and checked against its own fields.
let mut rewritten: BTreeMap<usize, Vec<String>> = BTreeMap::new();
let mut said = Vec::new();
let mut count = 0;
let start = lines.len() - section.lines.len();
for entry in blocks(&section.body(), 2) {
let offset = lines
.iter()
.enumerate()
.skip(start)
.find(|(_, l)| **l == entry.lines[0])
.map(|(i, _)| i);
let Some(offset) = offset else { continue };
let Some((note_at, note)) = entry
.lines
.iter()
.enumerate()
.find_map(|(i, l)| scalar(l, 4, "note").map(|v| (i, v)))
else {
continue;
};
let year = entry
.lines
.iter()
.find_map(|l| scalar(l, 4, "year"))
.and_then(|v| v.parse::<u32>().ok());
let parsed = crate::citation::parse(&note, year);
if parsed.container.is_none() && parsed.doi.is_none() {
continue;
}
let declared = |field: &str| entry.lines.iter().any(|l| scalar(l, 4, field).is_some());
let mut out: Vec<String> = Vec::new();
for (field, value, quote) in [
("container", parsed.container.clone(), false),
("volume", parsed.volume.clone(), true),
("pages", parsed.pages.clone(), true),
("doi", parsed.doi.clone(), true),
] {
let Some(value) = value else { continue };
if declared(field) {
said.push(format!(
"`{}` already declares {field}; the note's `{value}` was left in place",
entry.key
));
continue;
}
out.push(format!(" {field}: {}", render(&value, quote)));
}
if let Some(note) = &parsed.note {
out.push(format!(" note: {}", render(note, true)));
}
// A note whose every part was already declared is not a change.
if out.is_empty() {
continue;
}
if said
.iter()
.any(|m| m.starts_with(&format!("`{}`", entry.key)))
&& parsed.note.is_some()
&& out.len() == 1
{
continue;
}
rewritten.insert(offset + note_at, out);
count += 1;
}
if count == 0 {
return (text.to_string(), 0, said);
}
let mut out: Vec<String> = Vec::new();
for (i, line) in lines.iter().enumerate() {
match rewritten.get(&i) {
Some(replacement) => out.extend(replacement.clone()),
None => out.push(line.clone()),
}
}
let mut joined = out.join("\n");
if text.ends_with('\n') {
joined.push('\n');
}
(joined, count, said)
}
/// The value of a `key: value` line at an exact indentation, unquoted.
fn scalar(line: &str, indent: usize, key: &str) -> Option<String> {
if key_at(line, indent).as_deref() != Some(key) {
return None;
}
let value = line[indent + key.len() + 1..].trim();
if value.is_empty() {
return None;
}
for quote in ['\'', '"'] {
if let Some(inner) = value
.strip_prefix(quote)
.and_then(|v| v.strip_suffix(quote))
{
return Some(inner.replace("''", "'"));
}
}
Some(value.to_string())
}
/// A YAML scalar, quoted when it has to be.
///
/// Numbers are quoted whether they need it or not: `volume: 48` reads back as an
/// integer and the field is a string, so the file would stop loading.
fn render(value: &str, quote: bool) -> String {
let risky = quote
|| value.contains(": ")
|| value.ends_with(':')
|| value.contains(" #")
|| value.starts_with([
'[', '{', '&', '*', '!', '|', '>', '%', '@', '`', '\'', '"', '-',
]);
if risky {
format!("'{}'", value.replace('\'', "''"))
} else {
value.to_string()
}
}
/// One block of YAML: a key, the lines under it, and the comments above it.
#[derive(Debug, Clone, Default)]
struct Block {
@@ -1952,6 +2437,154 @@ items:
);
}
#[test]
fn the_order_integers_are_dropped_and_nothing_replaces_them() {
let text = r#"learning_objectives:
lo-x:
text: 'Read the formats.'
unit: u1
order: 2
level_ceiling: 2
learning_targets:
t-a:
text: 'First.'
objective: lo-x
order: 1
level_ceiling: 1
t-b:
text: 'Second.'
objective: lo-x
order: 2
level_ceiling: 2
"#;
let (out, dropped) = rewrite_order(text);
assert_eq!(dropped, 3, "one on the objective, one on each target");
assert!(!out.contains("order:"), "{out}");
// Nothing is written in their place: the objective's position comes
// from its lecture's `teaches`, and a target has no position. Matched
// on the inserted form rather than on `targets:`, which is a substring
// of the `learning_targets:` section header.
assert!(!out.contains("targets: ["), "{out}");
// Everything else is untouched, prose and ceilings included.
assert!(out.contains("text: 'Read the formats.'"), "{out}");
assert!(out.contains("level_ceiling: 2"), "{out}");
assert!(out.contains(" t-a:"), "{out}");
// Idempotent.
let (again, dropped) = rewrite_order(&out);
assert_eq!(dropped, 0);
assert_eq!(again, out);
}
#[test]
fn a_note_becomes_the_fields_it_was_carrying() {
let text = r#"references:
altschul1997gapped:
label: GBLAST97
kind: article
role: supplemental
title: 'Gapped BLAST and PSI-BLAST'
year: 1997
note: 'Nucleic Acids Res 25:3389-3402. doi:10.1093/nar/25.17.3389'
brown2013next:
label: BSM
kind: book
title: Next-generation DNA sequencing informatics
year: 2013
"#;
let (out, count, said) = expand_notes(text);
assert_eq!(count, 1);
assert!(said.is_empty(), "{said:?}");
assert!(out.contains(" container: Nucleic Acids Res"), "{out}");
assert!(out.contains(" volume: '25'"), "{out}");
assert!(out.contains(" pages: '3389-3402'"), "{out}");
assert!(out.contains(" doi: '10.1093/nar/25.17.3389'"), "{out}");
// Nothing left of the note, and nothing invented to replace it.
assert!(!out.contains("note:"), "{out}");
assert!(!out.contains("issue:"), "{out}");
// The book had no note and is untouched, label and all.
assert!(out.contains(" label: BSM"), "{out}");
assert!(out.contains(" brown2013next:"), "{out}");
}
#[test]
fn a_field_already_declared_is_never_overwritten() {
let text = r#"references:
steinegger2017mmseqs2:
kind: article
title: MMseqs2
year: 2017
note: 'Nat Biotechnol 35:1026-1028. doi:10.1038/nbt.3988'
doi: '10.1038/nbt.3988'
"#;
let (out, _, said) = expand_notes(text);
assert!(
said.iter().any(|m| m.contains("already declares doi")),
"{said:?}"
);
// One doi line, the one that was typed deliberately.
assert_eq!(out.matches("doi:").count(), 1, "{out}");
assert!(out.contains(" container: Nat Biotechnol"), "{out}");
}
#[test]
fn a_trailing_counter_is_recognized_and_nothing_else_is() {
assert_eq!(
strip_counter("q-fastq-quality-line-001").as_deref(),
Some("q-fastq-quality-line")
);
assert_eq!(
strip_counter("q-blast-seed-14").as_deref(),
Some("q-blast-seed")
);
// Already bare.
assert_eq!(strip_counter("q-fastq-quality-line"), None);
// A number that is part of the name, not a counter after it.
assert_eq!(strip_counter("q-fastqc-3prime-decay"), None);
assert_eq!(strip_counter("q-001"), Some("q".to_string()));
assert_eq!(strip_counter("q-x-"), None);
}
#[test]
fn a_rename_touches_the_named_fields_and_leaves_prose_alone() {
let mut rename = BTreeMap::new();
rename.insert("q-x-001".to_string(), "q-x".to_string());
let bank = r#"items:
- id: q-x-001
supersedes: q-x-001
stem: Which line?
design:
rationale: >-
Replaces q-x-001, which had two defensible answers.
"#;
let (out, n) = rename_ids(bank, &rename, &["- id", "supersedes"]);
assert_eq!(n, 2, "the id and the supersedes, not the sentence");
assert!(out.contains(" - id: q-x\n"), "{out}");
assert!(out.contains(" supersedes: q-x\n"), "{out}");
// The rationale is prose about the item; editing it would rewrite the
// sentence as well as the reference.
assert!(out.contains("Replaces q-x-001, which had"), "{out}");
}
#[test]
fn a_record_that_never_dropped_its_bank_prefix_is_still_renamed() {
let mut rename = BTreeMap::new();
rename.insert("q-x-001".to_string(), "q-x".to_string());
let flow = " - { number: 1, item: \"b-1-2::q-x-001\", key: [o-a] }\n";
let (out, n) = rename_ids(flow, &rename, &["item"]);
assert_eq!(n, 1);
// Left alone it would point at an id the bank no longer has.
assert!(out.contains("item: \"q-x\""), "{out}");
let block = " item: q-x-001\n";
let (out, n) = rename_ids(block, &rename, &["item"]);
assert_eq!(n, 1);
assert_eq!(out, " item: q-x\n");
}
#[test]
fn lecture_slugs_match_the_house_naming() {
assert_eq!(lecture_slug("L1.2"), "l-1-2");
+975
View File
@@ -0,0 +1,975 @@
//! `migrate glossary`: a 1.0 `terms.yaml` and `values.yaml` into the 2.1
//! registries.
//!
//! The same textual approach as the rest of [`crate::migrate`]. Each entry's
//! lines are copied and re-indented from a list item to a map entry, with three
//! edits: the `id` becomes the key and gains its prefix, `course` and `meaning`
//! are renamed, and `first` and `lectures`, which 2.1 derives from the targets
//! that list an entry, are dropped. Comments, folded scalars, and flow mappings
//! like `{ term: ten-fold }` survive.
//!
//! Every copied entry is parsed back and compared with the same conversion done
//! on the parsed source. An entry whose copied text does not read back the same
//! (a flow-mapping list item, a quoted key) is written by the serializer
//! instead, so the file is right even where it is not pretty. The whole file is
//! checked the same way before anything is written, and the course is loaded
//! again after.
//!
//! An entry the course already declares, in `glossary.yaml` or any other
//! fragment, is skipped, so a second run is a no-op.
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use serde::Serialize;
use serde::de::DeserializeOwned;
use serde_yaml_ng::{Mapping, Value as Yaml};
use super::{indent_of, key_at, relative, scalar};
use crate::course::CourseFile;
use crate::course::fragment::{Fragment, Section};
use crate::course::glossary::{self, RefKind, TERM_PREFIX, Term, VALUE_PREFIX, Value};
use crate::error::{Error, Result};
use crate::layout::Layout;
use crate::yaml;
/// What `migrate glossary` writes, and what it found on the way.
#[derive(Debug, Clone, Default)]
pub struct GlossaryMigration {
/// The file it writes, relative to the course root.
pub path: PathBuf,
/// The file's new contents; `None` when there is nothing to add.
pub text: Option<String>,
/// Ids added, terms first, each in source order.
pub added: Vec<String>,
/// Old id to new id, for every added entry whose id gained a prefix.
pub renamed: Vec<(String, String)>,
/// Entries the course already declares, with the file that declares them.
pub present: Vec<(String, PathBuf)>,
/// One message per dropped `first` or `lectures` that differs from what
/// the targets now say.
pub derived: Vec<String>,
/// Added entries that no target lists yet.
pub unlisted: Vec<String>,
/// One message per field dropped because 2.1 has no place for it.
pub dropped: Vec<String>,
/// One message per value whose `term:` was rewritten to the prefixed id.
pub relinked: Vec<String>,
}
/// Which 1.0 file an entry came from.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Kind {
Term,
Value,
}
impl Kind {
/// The top-level key, the same in 1.0 and 2.1.
fn section(self) -> &'static str {
match self {
Kind::Term => "terms",
Kind::Value => "values",
}
}
/// The prefix a 2.1 id carries.
fn prefix(self) -> &'static str {
match self {
Kind::Term => TERM_PREFIX,
Kind::Value => VALUE_PREFIX,
}
}
/// The registry an id of this kind lives in.
fn registry(self) -> Section {
match self {
Kind::Term => Section::Terms,
Kind::Value => Section::Values,
}
}
/// The 1.0 field that 2.1 derives from the targets instead.
fn derived(self) -> &'static str {
match self {
Kind::Term => "first",
Kind::Value => "lectures",
}
}
/// The 2.1 name of a renamed 1.0 field.
fn renamed(self, key: &str) -> Option<&'static str> {
match (self, key) {
(Kind::Term, "course") => Some("term"),
(Kind::Term, "meaning") => Some("definition"),
_ => None,
}
}
/// Every field the 2.1 entry has; anything else is dropped and reported.
fn fields(self) -> &'static [&'static str] {
match self {
Kind::Term => &[
"term",
"definition",
"kind",
"textbook",
"variants",
"symbol",
"units",
"exam_wording",
"status",
"tags",
"notes_private",
],
Kind::Value => &[
"name",
"value",
"range",
"units",
"say",
"exact",
"term",
"source",
"conflicts",
"recall",
"status",
"tags",
"notes_private",
],
}
}
}
/// One 1.0 entry, read leniently and converted to 2.1 fields.
#[derive(Debug, Clone)]
struct Entry {
/// The id as the 1.0 file wrote it.
old: String,
/// The 2.1 id.
id: String,
/// The 2.1 fields, renamed, unknown ones gone.
fields: Mapping,
/// What the dropped `first` or `lectures` said.
derived: Option<Yaml>,
/// One message per unknown field dropped.
dropped: Vec<String>,
/// The value's `term:` before it gained its prefix.
relinked: Option<String>,
}
/// One list item in a 1.0 file, as lines.
#[derive(Debug, Clone, Default)]
struct Item {
/// Comment lines directly above the item.
lead: Vec<String>,
/// The `- ` line and everything under it.
lines: Vec<String>,
/// The column of the item's dash.
dash: usize,
}
/// Writes a 1.0 `terms.yaml` and `values.yaml` into the course's
/// `glossary.yaml`.
///
/// Ids gain their `term-` and `val-` prefixes, `course` becomes `term` and
/// `meaning` becomes `definition`, a value's `term:` gains the prefix of the id
/// it names, and `first` and `lectures` are dropped: 2.1 derives them from the
/// targets that list an entry, and each one that disagrees with what the
/// targets now say is reported. A field 2.1 has no place for is dropped and
/// reported. The old files are left alone.
///
/// An existing `glossary.yaml` is added to, never rewritten: new entries go at
/// the end of its `terms:` and `values:` sections, and everything already in
/// it stays byte for byte.
///
/// # Arguments
///
/// * `root` - the course directory.
/// * `terms` - the 1.0 terms file, if migrating one.
/// * `values` - the 1.0 values file, if migrating one.
/// * `write` - whether to write, or only to report.
///
/// # Returns
///
/// What was (or would be) written, and the report.
///
/// # Errors
///
/// Returns [`Error::Usage`] when neither file is given, [`Error::Invalid`] for
/// entries that cannot be migrated (no id, an id that would not be valid, a
/// field 2.1 would reject), and [`Error::Other`] if the written file would not
/// read back as migrated. Nothing is written in any of those cases.
pub fn glossary(
root: &Path,
terms: Option<&Path>,
values: Option<&Path>,
write: bool,
) -> Result<GlossaryMigration> {
if terms.is_none() && values.is_none() {
return Err(Error::usage(
"nothing to migrate: give --terms, --values, or both",
));
}
let layout = Layout::new(root);
let course = CourseFile::load_dir(root)?;
let path = layout.glossary_file();
let mut report = GlossaryMigration {
path: relative(root, &path),
..GlossaryMigration::default()
};
let mut new_terms: BTreeMap<String, Term> = BTreeMap::new();
let mut new_values: BTreeMap<String, Value> = BTreeMap::new();
let mut sections: Vec<(Kind, Vec<Vec<String>>)> = Vec::new();
if let Some(file) = terms {
let chunks = convert(
root,
file,
Kind::Term,
&course,
&mut report,
&mut new_terms,
|f| &f.terms,
)?;
sections.push((Kind::Term, chunks));
}
if let Some(file) = values {
let chunks = convert(
root,
file,
Kind::Value,
&course,
&mut report,
&mut new_values,
|f| &f.values,
)?;
sections.push((Kind::Value, chunks));
}
if report.added.is_empty() {
return Ok(report);
}
let before = if path.is_file() {
Some(std::fs::read_to_string(&path).map_err(|e| Error::io(&path, e))?)
} else {
None
};
let names: Vec<String> = [terms, values]
.into_iter()
.flatten()
.map(file_name)
.collect();
let mut text = before.clone().unwrap_or_else(|| header(&names));
for (kind, chunks) in &sections {
if !chunks.is_empty() {
text = insert(&text, kind.section(), chunks)?;
}
}
verify(before.as_deref(), &text, &new_terms, &new_values)?;
if write {
yaml::write_text(&path, &text)?;
// The fragment reads back; this checks the course still assembles with
// it, and puts the old file back when it does not.
if let Err(e) = CourseFile::load_dir(root) {
match &before {
Some(old) => yaml::write_text(&path, old)?,
None => std::fs::remove_file(&path).map_err(|e| Error::io(&path, e))?,
}
return Err(e);
}
}
report.text = Some(text);
Ok(report)
}
/// Converts one 1.0 file into the entry chunks to write.
///
/// Fills in the report and `expected`, the typed entries the written file must
/// read back as.
fn convert<T>(
root: &Path,
file: &Path,
kind: Kind,
course: &CourseFile,
report: &mut GlossaryMigration,
expected: &mut BTreeMap<String, T>,
pick: fn(&Fragment) -> &BTreeMap<String, T>,
) -> Result<Vec<Vec<String>>>
where
T: DeserializeOwned + Serialize + PartialEq + Clone,
{
let text = std::fs::read_to_string(file).map_err(|e| Error::io(file, e))?;
let doc: Yaml = serde_yaml_ng::from_str(&text).map_err(|source| Error::Yaml {
path: file.to_path_buf(),
source,
})?;
let name = file_name(file);
let entries = entries(&doc, kind, &name)?;
// The text items line up with the parsed entries one for one, unless the
// list is written in some form the scanner does not follow; then every
// entry is serialized.
let items = items(&text, kind.section());
let items = (items.len() == entries.len()).then_some(items);
let first_use = course.glossary_first_use();
let mut problems = Vec::new();
let mut chunks = Vec::new();
for (n, entry) in entries.iter().enumerate() {
if let Some((section, at)) = course.origin(&entry.id) {
let at = relative(root, at);
if section == kind.registry() {
report.present.push((entry.id.clone(), at));
} else {
problems.push(format!(
"{name}: `{}` would become `{}`, which {} already uses under `{}`",
entry.old,
entry.id,
at.display(),
section.key()
));
}
continue;
}
let typed: T = match serde_yaml_ng::from_value(Yaml::Mapping(entry.fields.clone())) {
Ok(typed) => typed,
Err(e) => {
problems.push(format!("{name}: `{}`: {e}", entry.old));
continue;
}
};
let copied = items
.as_ref()
.and_then(|items| copied(&items[n], kind, entry))
.filter(|chunk| reads_back(chunk, kind, &entry.id, &typed, pick));
let chunk = match copied {
Some(chunk) => chunk,
None => serialized(&entry.id, &typed)?,
};
if entry.old != entry.id {
report.renamed.push((entry.old.clone(), entry.id.clone()));
}
report.dropped.extend(entry.dropped.iter().cloned());
if let Some(old) = &entry.relinked {
report.relinked.push(format!(
"`{}`: `term: {old}` now reads `term: {TERM_PREFIX}{old}`",
entry.id
));
}
if let Some(message) = derived_message(kind, entry, course) {
report.derived.push(message);
}
if !first_use.contains_key(entry.id.as_str()) {
report.unlisted.push(entry.id.clone());
}
report.added.push(entry.id.clone());
expected.insert(entry.id.clone(), typed);
chunks.push(chunk);
}
if !problems.is_empty() {
return Err(Error::Invalid(problems));
}
Ok(chunks)
}
/// Reads the entries of one 1.0 file.
///
/// Lenient about everything 2.1 does not need: a missing schema version, an id
/// written as a number, fields 2.1 has no place for. Strict about what would
/// make a wrong file: an entry with no id, an id that would not be valid with
/// its prefix, two entries that would share an id, and a field given twice
/// under its old and new names.
fn entries(doc: &Yaml, kind: Kind, file: &str) -> Result<Vec<Entry>> {
let section = kind.section();
let list = match doc.get(section) {
None | Some(Yaml::Null) => return Ok(Vec::new()),
Some(Yaml::Sequence(list)) => list,
Some(_) => {
return Err(Error::usage(format!(
"{file}: `{section}` is not a list; 1.0 writes one `- id:` item per entry"
)));
}
};
let mut out: Vec<Entry> = Vec::new();
let mut problems = Vec::new();
for (n, item) in list.iter().enumerate() {
let Yaml::Mapping(map) = item else {
problems.push(format!(
"{file}: {section} entry {} is not a mapping",
n + 1
));
continue;
};
let Some(old) = map.get("id").and_then(plain) else {
problems.push(format!("{file}: {section} entry {} has no `id`", n + 1));
continue;
};
let id = if old.starts_with(kind.prefix()) {
old.clone()
} else {
format!("{}{old}", kind.prefix())
};
if !glossary::is_prefixed_id(&id, kind.prefix()) {
problems.push(format!(
"{file}: `{old}` would become `{id}`, which is not a valid id; rename it in {file} first"
));
continue;
}
if out.iter().any(|e| e.id == id) {
problems.push(format!("{file}: two entries would both be `{id}`"));
continue;
}
let mut entry = Entry {
old: old.clone(),
id,
fields: Mapping::new(),
derived: None,
dropped: Vec::new(),
relinked: None,
};
for (key, value) in map {
let Some(key) = key.as_str() else {
entry
.dropped
.push(format!("`{}`: a key that is not a string", entry.id));
continue;
};
if key == "id" {
continue;
}
if key == kind.derived() {
entry.derived = Some(value.clone());
continue;
}
let name = kind.renamed(key).unwrap_or(key);
if !kind.fields().contains(&name) {
entry.dropped.push(format!(
"`{}`: `{key}`, which 2.1 has no field for",
entry.id
));
continue;
}
let mut value = value.clone();
if kind == Kind::Value
&& name == "term"
&& let Some(term) = value.as_str()
&& !term.starts_with(TERM_PREFIX)
{
entry.relinked = Some(term.to_string());
value = Yaml::String(format!("{TERM_PREFIX}{term}"));
}
if entry
.fields
.insert(Yaml::String(name.to_string()), value)
.is_some()
{
problems.push(format!(
"{file}: `{old}` gives `{name}` twice, once as `{key}`"
));
}
}
out.push(entry);
}
if !problems.is_empty() {
return Err(Error::Invalid(problems));
}
Ok(out)
}
/// A scalar as a string: strings as they are, numbers and booleans as written.
fn plain(value: &Yaml) -> Option<String> {
match value {
Yaml::String(s) => Some(s.clone()),
Yaml::Number(n) => Some(n.to_string()),
Yaml::Bool(b) => Some(b.to_string()),
_ => None,
}
}
/// The list items under a top-level key, as lines.
fn items(text: &str, section: &str) -> Vec<Item> {
let lines: Vec<&str> = text.lines().collect();
let Some(start) = lines
.iter()
.position(|l| key_at(l, 0).as_deref() == Some(section))
else {
return Vec::new();
};
let body: Vec<&str> = lines[start + 1..]
.iter()
.take_while(|l| key_at(l, 0).is_none())
.copied()
.collect();
let starts_item = |line: &str| {
let trimmed = line.trim_start();
trimmed.starts_with("- ") || trimmed == "-"
};
let Some(dash) = body
.iter()
.find(|l| starts_item(l))
.and_then(|l| indent_of(l))
else {
return Vec::new();
};
let mut out: Vec<Item> = Vec::new();
let mut lead: Vec<String> = Vec::new();
for line in body {
if indent_of(line) == Some(dash) && starts_item(line) {
if let Some(previous) = out.last_mut() {
lead = trailing_comments(&mut previous.lines);
}
out.push(Item {
lead: std::mem::take(&mut lead),
lines: vec![line.to_string()],
dash,
});
continue;
}
match out.last_mut() {
Some(item) => item.lines.push(line.to_string()),
None if line.trim_start().starts_with('#') => lead.push(line.to_string()),
None => {}
}
}
if let Some(last) = out.last_mut() {
trailing_comments(&mut last.lines);
}
out
}
/// Takes a run of trailing blank and comment lines off an item; returns the
/// comments, which belong to whatever follows.
fn trailing_comments(lines: &mut Vec<String>) -> Vec<String> {
let mut tail = Vec::new();
while lines.len() > 1 {
let last = lines.last().map(|l| l.trim()).unwrap_or_default();
if last.is_empty() || last.starts_with('#') {
tail.push(lines.pop().unwrap_or_default());
} else {
break;
}
}
tail.reverse();
tail.into_iter()
.filter(|l| l.trim().starts_with('#'))
.collect()
}
/// An item's lines as an entry: ` term-x:` and its fields four deep.
///
/// `None` when the item is not written in a form this follows line by line,
/// for the caller to serialize instead.
fn copied(item: &Item, kind: Kind, entry: &Entry) -> Option<Vec<String>> {
// The dash becomes a space, so the first field sits where the rest do.
let first = item.lines.first()?;
let after = first.get(item.dash + 1..)?;
let mut lines: Vec<String> = Vec::new();
let indent = if after.trim().is_empty() {
// A bare `-` with the fields on the lines below it.
lines.extend(item.lines[1..].iter().cloned());
lines.iter().find_map(|l| indent_of(l))?
} else {
lines.push(format!("{} {after}", &first[..item.dash]));
lines.extend(item.lines[1..].iter().cloned());
indent_of(&lines[0])?
};
// A flow mapping is one value, not lines of fields.
if lines
.iter()
.find(|l| indent_of(l).is_some())
.is_some_and(|l| l.trim_start().starts_with(['{', '[']))
{
return None;
}
// Split into fields: a key at the field indent starts one; comments and
// deeper lines continue it.
let mut fields: Vec<(String, Vec<String>)> = Vec::new();
let mut lead: Vec<String> = item.lead.clone();
for line in lines {
match key_at(&line, indent) {
Some(key) => fields.push((key, vec![line])),
None => match fields.last_mut() {
Some((_, field)) => field.push(line),
None if line.trim().is_empty() => {}
None if line.trim_start().starts_with('#') => lead.push(line),
None => return None,
},
}
}
if fields.is_empty() {
return None;
}
let mut out: Vec<String> = lead.iter().map(|l| format!(" {}", l.trim())).collect();
out.push(format!(" {}:", entry.id));
for (key, mut field) in fields {
if key == "id" || key == kind.derived() {
continue;
}
let name = kind.renamed(&key).unwrap_or(&key);
if !kind.fields().contains(&name) {
continue;
}
if name != key {
let line = &field[0];
if !line[indent..].starts_with(key.as_str()) {
return None;
}
field[0] = format!("{}{name}{}", &line[..indent], &line[indent + key.len()..]);
}
if kind == Kind::Value && key == "term" && entry.relinked.is_some() {
// Only a one-line plain scalar is rewritten in place.
let term = scalar(&field[0], indent, "term")?;
if field.len() > 1 || entry.relinked.as_deref() != Some(term.as_str()) {
return None;
}
field[0] = format!("{}term: {TERM_PREFIX}{term}", &field[0][..indent]);
}
out.extend(field.iter().map(|l| shift(l, indent, 4)));
}
while out.last().is_some_and(|l| l.trim().is_empty()) {
out.pop();
}
Some(out)
}
/// Moves a line from one indentation to another, keeping what is under it
/// relative.
fn shift(line: &str, from: usize, to: usize) -> String {
if line.trim().is_empty() {
return String::new();
}
if to >= from {
return format!("{}{line}", " ".repeat(to - from));
}
let leading = line.len() - line.trim_start().len();
line[leading.min(from - to)..].to_string()
}
/// Whether a chunk reads back as the entry it was copied from.
fn reads_back<T: DeserializeOwned + PartialEq>(
chunk: &[String],
kind: Kind,
id: &str,
expected: &T,
pick: fn(&Fragment) -> &BTreeMap<String, T>,
) -> bool {
let text = format!("{}:\n{}\n", kind.section(), chunk.join("\n"));
serde_yaml_ng::from_str::<Fragment>(&text)
.ok()
.is_some_and(|fragment| pick(&fragment).get(id) == Some(expected))
}
/// An entry written by the serializer, for when its lines could not be copied.
fn serialized<T: Serialize>(id: &str, entry: &T) -> Result<Vec<String>> {
let text = yaml::to_string(entry)?;
let mut out = vec![format!(" {id}:")];
out.extend(
text.lines()
.filter(|l| *l != "---")
.map(|l| format!(" {l}")),
);
Ok(out)
}
/// What a dropped `first` or `lectures` said, when the targets now say
/// something else.
fn derived_message(kind: Kind, entry: &Entry, course: &CourseFile) -> Option<String> {
let old = entry.derived.as_ref()?;
let said: Vec<String> = match old {
Yaml::Sequence(list) => list.iter().filter_map(plain).collect(),
other => plain(other).into_iter().collect(),
};
let now: Vec<&str> = match kind {
Kind::Term => course.first_lecture(&entry.id).into_iter().collect(),
Kind::Value => lectures_listing(course, &entry.id, RefKind::Value),
};
let same = said.iter().map(String::as_str).collect::<BTreeSet<_>>()
== now.iter().copied().collect::<BTreeSet<_>>();
if same {
return None;
}
let derived = if now.is_empty() {
"none".to_string()
} else if kind == Kind::Term {
now.join(", ")
} else {
format!("[{}]", now.join(", "))
};
let was = if kind == Kind::Term {
said.join(", ")
} else {
format!("[{}]", said.join(", "))
};
Some(format!(
"`{}`: `{}` was {was}; derived from the targets: {derived}",
entry.id,
kind.derived()
))
}
/// Every lecture, in order, with a target that lists an entry.
fn lectures_listing<'a>(course: &'a CourseFile, id: &str, kind: RefKind) -> Vec<&'a str> {
course
.lectures
.keys()
.map(String::as_str)
.filter(|lecture| {
course
.lecture_targets_in_order(lecture)
.iter()
.filter_map(|t| course.learning_targets.get(*t))
.any(|target| target.listed(kind).iter().any(|x| x == id))
})
.collect()
}
/// A path's file name, for messages.
fn file_name(path: &Path) -> String {
path.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_else(|| path.display().to_string())
}
/// The comment a new `glossary.yaml` starts with.
fn header(sources: &[String]) -> String {
format!(
"# glossary.yaml (coursebank 2.1)\n\
# Entries brought in from {} by `coursebank migrate glossary`. Move each one to the\n\
# objective file whose targets first need it; keep the cross-cutting ones here.\n",
sources.join(" and ")
)
}
/// Adds entry chunks at the end of a top-level section, or a new section at
/// the end of the file.
fn insert(text: &str, section: &str, chunks: &[Vec<String>]) -> Result<String> {
let mut lines: Vec<String> = text.lines().map(str::to_string).collect();
let new: Vec<String> = chunks.iter().flatten().cloned().collect();
match lines
.iter()
.position(|l| key_at(l, 0).as_deref() == Some(section))
{
Some(at) => {
let colon = lines[at].find(':').unwrap_or(lines[at].len());
let rest = lines[at][colon + 1..].trim().to_string();
if !(rest.is_empty() || rest.starts_with('#')) {
if rest == "{}" {
lines[at] = format!("{section}:");
} else {
return Err(Error::usage(format!(
"glossary.yaml writes `{section}:` on one line; write it as a block, one \
entry per key, and run this again"
)));
}
}
let mut end = at + 1;
while end < lines.len() && key_at(&lines[end], 0).is_none() {
end += 1;
}
// Blank lines and column-0 comments at the end of the section
// introduce whatever follows it.
while end > at + 1 {
let last = &lines[end - 1];
let comment = indent_of(last) == Some(0) && last.starts_with('#');
if last.trim().is_empty() || comment {
end -= 1;
} else {
break;
}
}
lines.splice(end..end, new);
}
None => {
while lines.last().is_some_and(|l| l.trim().is_empty()) {
lines.pop();
}
let has_content = lines.iter().any(|l| {
let t = l.trim();
!t.is_empty() && !t.starts_with('#')
});
if has_content {
lines.push(String::new());
}
lines.push(format!("{section}:"));
lines.extend(new);
}
}
Ok(format!("{}\n", lines.join("\n")))
}
/// Checks that the new file reads back as the old one plus the migrated
/// entries, and nothing else.
fn verify(
before: Option<&str>,
after: &str,
terms: &BTreeMap<String, Term>,
values: &BTreeMap<String, Value>,
) -> Result<()> {
let unreadable = |e: serde_yaml_ng::Error| {
Error::other(format!(
"the glossary.yaml this would write does not load ({e}); nothing was written"
))
};
let mut parsed: Fragment = serde_yaml_ng::from_str(after).map_err(unreadable)?;
for (id, term) in terms {
if parsed.terms.remove(id).as_ref() != Some(term) {
return Err(Error::other(format!(
"`{id}` does not read back as migrated; nothing was written"
)));
}
}
for (id, value) in values {
if parsed.values.remove(id).as_ref() != Some(value) {
return Err(Error::other(format!(
"`{id}` does not read back as migrated; nothing was written"
)));
}
}
let old: Fragment = match before {
Some(text) => serde_yaml_ng::from_str(text).map_err(unreadable)?,
None => Fragment::default(),
};
if yaml::to_string(&parsed)? != yaml::to_string(&old)? {
return Err(Error::other(
"migrating would change entries glossary.yaml already has; nothing was written",
));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
const TERMS: &str = "schema_version: \"1.0\"
terms:
# A coined one.
- id: decade
course: decade
variants: [decades]
kind: coined
meaning: >-
a factor of ten in a concentration
ratio
first: L2.1
color: blue
- id: term-gap
course: gap
meaning: a dash
";
fn parsed(text: &str, kind: Kind) -> Vec<Entry> {
let doc: Yaml = serde_yaml_ng::from_str(text).unwrap();
entries(&doc, kind, "terms.yaml").unwrap()
}
#[test]
fn entries_take_their_prefix_once_and_lose_derived_and_unknown_fields() {
let entries = parsed(TERMS, Kind::Term);
assert_eq!(entries[0].id, "term-decade");
assert_eq!(entries[1].id, "term-gap");
assert_eq!(entries[0].derived, Some(Yaml::String("L2.1".into())));
assert_eq!(entries[0].dropped.len(), 1);
assert!(entries[0].dropped[0].contains("`color`"));
assert!(entries[0].fields.contains_key("definition"));
assert!(!entries[0].fields.contains_key("meaning"));
}
#[test]
fn copied_lines_keep_comments_and_block_scalars() {
let entries = parsed(TERMS, Kind::Term);
let items = items(TERMS, "terms");
assert_eq!(items.len(), 2);
let chunk = copied(&items[0], Kind::Term, &entries[0]).unwrap();
assert_eq!(
chunk,
[
" # A coined one.",
" term-decade:",
" term: decade",
" variants: [decades]",
" kind: coined",
" definition: >-",
" a factor of ten in a concentration",
" ratio",
]
);
let term: Term =
serde_yaml_ng::from_value(Yaml::Mapping(entries[0].fields.clone())).unwrap();
assert!(reads_back(&chunk, Kind::Term, "term-decade", &term, |f| &f.terms));
}
#[test]
fn items_at_column_zero_are_reindented() {
let text = "terms:\n- id: gap\n course: gap\n meaning: a dash\n";
let entries = parsed(text, Kind::Term);
let items = items(text, "terms");
let chunk = copied(&items[0], Kind::Term, &entries[0]).unwrap();
assert_eq!(
chunk,
[" term-gap:", " term: gap", " definition: a dash"]
);
}
#[test]
fn a_value_names_its_term_by_the_new_id() {
let text = "values:\n - id: k\n name: k\n say: about 6\n term: decade\n";
let entries = parsed(text, Kind::Value);
assert_eq!(entries[0].relinked.as_deref(), Some("decade"));
let items = items(text, "values");
let chunk = copied(&items[0], Kind::Value, &entries[0]).unwrap();
assert_eq!(chunk.last().unwrap(), " term: term-decade");
}
#[test]
fn a_flow_item_falls_back_to_the_serializer() {
let text = "terms:\n - {id: gap, course: gap, meaning: a dash}\n";
let entries = parsed(text, Kind::Term);
let items = items(text, "terms");
assert!(copied(&items[0], Kind::Term, &entries[0]).is_none());
let term: Term =
serde_yaml_ng::from_value(Yaml::Mapping(entries[0].fields.clone())).unwrap();
let chunk = serialized("term-gap", &term).unwrap();
assert!(reads_back(&chunk, Kind::Term, "term-gap", &term, |f| &f.terms));
}
#[test]
fn bad_and_duplicate_ids_are_refused() {
let doc: Yaml = serde_yaml_ng::from_str(
"terms:\n - id: Two Words\n course: x\n meaning: y\n - id: a\n course: a\n meaning: b\n - id: term-a\n course: a\n meaning: b\n - course: z\n",
)
.unwrap();
let Err(Error::Invalid(problems)) = entries(&doc, Kind::Term, "terms.yaml") else {
panic!("expected problems");
};
assert_eq!(problems.len(), 3, "{problems:?}");
}
#[test]
fn insert_appends_to_a_section_or_adds_one() {
let text = "# keep\nterms:\n term-a:\n term: a\n definition: b\n\n# about values\nvalues: {}\n";
let chunk = vec![" term-c:".to_string(), " term: c".to_string()];
let out = insert(text, "terms", std::slice::from_ref(&chunk)).unwrap();
assert!(out.contains(" definition: b\n term-c:\n term: c\n\n# about values\n"));
let value = vec![" val-x:".to_string(), " name: x".to_string()];
let out = insert(&out, "values", &[value]).unwrap();
assert!(out.ends_with("values:\n val-x:\n name: x\n"));
let fresh = insert("# header\n", "terms", &[chunk]).unwrap();
assert_eq!(fresh, "# header\nterms:\n term-c:\n term: c\n");
}
#[test]
fn shift_moves_whole_blocks() {
assert_eq!(shift(" deeper", 2, 4), " deeper");
assert_eq!(shift(" deeper", 6, 4), " deeper");
assert_eq!(shift(" ", 2, 4), "");
}
}
+6 -3
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The data model: the four kinds of file, and the taxonomy they are written against.
@@ -13,8 +13,10 @@
//! taxonomy levels, cognitive processes, error types, status, flags
//! │
//! course identity, policy, objectives, lectures, stimuli
//! │ ├─ course::glossary terms, equations, values, misconceptions
//! │ └─ course::fragment merges course.yaml, references.yaml,
//! │ lectures/*.yaml, objectives/*.yaml
//! │ glossary.yaml, lectures/*.yaml,
//! │ objectives/*.yaml
//! │
//! item one question: stem, options, design intent, calibration
//! │
@@ -28,6 +30,7 @@
pub mod assessment;
pub mod bank;
pub mod calibration;
pub mod catalog;
pub mod course;
pub mod history;
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The assessment record: what you actually asked, in what order, on what date.
+25 -34
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The bank file: a collection of items scoped to a topic or a lecture.
@@ -440,6 +440,23 @@ fn validate_item(
o.id
));
}
// A registry misconception named by id has to exist, or the report
// that groups misses by it would group them under nothing.
if let Some(id) = &o.misconception_id {
let declared = match course {
Some(c) => c.misconceptions.contains_key(id),
None => crate::course::glossary::is_prefixed_id(
id,
crate::course::glossary::MISCONCEPTION_PREFIX,
),
};
if !declared {
issues.push(format!(
"option {}: misconception_id `{id}` is not a declared misconception",
o.id
));
}
}
}
if let Some(n) = expected_options {
if it.options.len() != n && !it.options.is_empty() {
@@ -530,38 +547,12 @@ fn validate_item(
}
// --- calibration plausibility ------
if let Some(c) = &it.calibration {
if let Some(p) = c.p_value {
if !(0.0..=1.0).contains(&p) {
issues.push(format!(
"calibration.p_value must be between 0 and 1, got {p}"
));
}
}
if let Some(r) = c.point_biserial {
if !(-1.0..=1.0).contains(&r) {
issues.push(format!(
"calibration.point_biserial must be between -1 and 1, got {r}"
));
}
}
for option in c.option_stats.keys() {
if it.option(option).is_none() {
issues.push(format!(
"calibration.option_stats has `{option}`, which is not an option of this item"
));
}
}
if let Some(irt) = &c.irt {
if irt.a <= 0.0 {
issues.push(format!("calibration.irt.a must be positive, got {}", irt.a));
}
if let Some(cp) = irt.c {
if !(0.0..1.0).contains(&cp) {
issues.push(format!("calibration.irt.c must be in [0, 1), got {cp}"));
}
}
}
if it.calibration.is_some() {
issues.push(
"has a `calibration:` block, but statistics live in analysis/calibration.yaml since \
2.0. A bank's diff should be a change of intent, not the output of a grading run."
.to_string(),
);
}
// --- retirement -----
+838
View File
@@ -0,0 +1,838 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Where the statistics live, which is not in the bank.
//!
//! A bank file is a reviewed artifact: someone wrote the question, someone
//! argued about the distractors, and the diff on it should be a change of
//! intent. Statistics are neither reviewed nor intended — they are what
//! happened — and writing them back into the bank means every grading run
//! produces a diff on a file whose history is supposed to be about wording.
//!
//! So the evidence lives in `analysis/`, in two kinds of file:
//!
//! | File | Format | Rewritten? | Holds |
//! |:--|:--|:--|:--|
//! | `analysis/administrations/<id>-items.csv` | CSV | never | one row per question |
//! | `analysis/administrations/<id>-options.csv` | CSV | never | one row per question and option |
//! | `analysis/calibration.yaml` | YAML | by `calibrate` | the pooled per-item view |
//!
//! The format follows the shape. An administration record is a table — fixed
//! columns, one row per question, machine-written, never hand-edited — so it is
//! CSV: one line per item rather than fifteen, which diffs better, and it loads
//! straight into pandas or DuckDB, which is much of the point of committing it.
//! The pooled view is not a table. It is three levels deep, item to variant to
//! option history, with fitted parameters and variable-length lists, and as CSV
//! that would be three files joined by keys — a relational schema for the one
//! file a person actually reads in a pull request. That stays YAML.
//!
//! Neither is Parquet, and the reason is the review workflow: a binary file
//! shows nothing in a diff and cannot be merged. Parquet is right for `data/`
//! precisely because that is bulk, ignored, and never reviewed.
//!
//! An administration file is written once and not touched again, for the same
//! reason a seal is not: it is a record of a thing that happened on a day. The
//! calibration file is the accepted rollup — what `lint` compares your
//! predictions against, and what a report reads — and `calibrate` proposes
//! changes to it as a diff you review before committing.
//!
//! Both are meant to be committed. Neither can carry student data, and that is
//! a property of the types rather than a promise: there is no field for a
//! student key, an identifier, a section, or an ability estimate, and the
//! structures reject unknown keys, so a file carrying one fails to load rather
//! than being quietly accepted. Everything per-person stays in `data/`, which
//! is what your `.gitignore` is for.
//!
//! # Linking back to the bank
//!
//! By item id, which since 2.0 names the item course-wide and has no file name
//! in it, and by variant digest, which says which option set the numbers
//! describe. A record also carries the stem digest it was measured against, so
//! [`CalibrationFile::validate`] can say that an item has been reworded since —
//! the statistics then describe a question that no longer exists under that id.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use crate::course::SCHEMA_VERSION;
use crate::date::Date;
use crate::error::{Error, Result};
use crate::item::{Calibration, IrtModel, IrtParams, OptionStat};
use crate::taxonomy::Flag;
use crate::yaml;
/// The file name of the pooled calibration store, under `analysis/`.
pub const CALIBRATION_FILE: &str = "calibration.yaml";
/// The pooled per-item statistics: `analysis/calibration.yaml`.
///
/// Keyed by item id. This is the file `calibrate` rewrites and the one
/// everything else reads; [`crate::catalog::Catalog::load`] fills each item's
/// in-memory calibration from it, so nothing downstream has to know where the
/// numbers came from.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct CalibrationFile {
/// Schema version.
#[serde(
default = "default_version",
deserialize_with = "yaml::flexible_string"
)]
pub schema_version: String,
/// One entry per calibrated item, by item id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub items: BTreeMap<String, Calibration>,
}
impl CalibrationFile {
/// Loads the store, or an empty one when the file does not exist yet.
///
/// Absence is not an error: a course that has not graded anything has no
/// statistics, and every command that reads them has to work anyway.
///
/// # Arguments
///
/// * `path` - the file, usually `analysis/calibration.yaml`.
///
/// # Returns
///
/// The store.
///
/// # Errors
///
/// Returns [`Error::Yaml`] when the file exists and does not parse.
pub fn load(path: &Path) -> Result<CalibrationFile> {
if !path.is_file() {
return Ok(CalibrationFile::default());
}
yaml::read(path)
}
/// Writes the store.
///
/// # Arguments
///
/// * `path` - the destination.
///
/// # Errors
///
/// Returns [`Error::Io`] on a write failure.
pub fn save(&self, path: &Path) -> Result<()> {
yaml::write(path, self)
}
/// The calibration recorded for one item.
///
/// # Arguments
///
/// * `item` - the item id.
///
/// # Returns
///
/// The record, or `None` when the item has never been calibrated.
pub fn get(&self, item: &str) -> Option<&Calibration> {
self.items.get(item)
}
/// Checks the store against the bank it describes.
///
/// The checks that matter for a file kept apart from what it refers to: a
/// record for an item that no longer exists, an option id the item does not
/// have, and — the one worth having — statistics measured against a stem
/// that has since been reworded, which since 2.0 means they describe a
/// different question wearing the same id.
///
/// # Arguments
///
/// * `catalog` - the loaded course.
///
/// # Returns
///
/// One message per problem, empty when the store agrees with the bank.
pub fn validate(&self, catalog: &crate::catalog::Catalog) -> Vec<String> {
let mut issues = Vec::new();
for (id, calibration) in &self.items {
let Some(entry) = catalog.get(id) else {
issues.push(format!(
"calibration for `{id}`: no such item. Statistics outlive an item only if \
it is retired, not deleted — a retired item keeps its id so its numbers \
still mean something."
));
continue;
};
let item = &entry.item;
for variant in &calibration.variants {
for option in variant.option_stats.keys() {
if item.option(option).is_none() {
issues.push(format!(
"calibration for `{id}`: variant `{}` has statistics for `{option}`, \
which is not an option of this item",
short(&variant.variant)
));
}
}
if !variant.key.is_empty()
&& variant.variant != item.variant_digest(&variant.key, &variant.distractors)
{
issues.push(format!(
"calibration for `{id}`: variant `{}` was measured against an option set \
that has since been reworded, so its numbers describe wording no \
student now sees",
short(&variant.variant)
));
}
}
for option in calibration.options.keys() {
if item.option(option).is_none() {
issues.push(format!(
"calibration for `{id}`: an option history names `{option}`, which is \
not an option of this item"
));
}
}
}
issues
}
}
/// What one administration measured: `analysis/administrations/<id>.yaml`.
///
/// Written once, when the exam is analyzed, and never rewritten. It is the
/// audit trail under [`CalibrationFile`]: the pooled numbers say an item sits
/// at 0.63, and these say which exams that came from and what each one saw.
/// Keeping them also means the history survives losing `data/`, which is
/// ignored by git and rotates.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MeasurementFile {
/// Schema version.
#[serde(
default = "default_version",
deserialize_with = "yaml::flexible_string"
)]
pub schema_version: String,
/// What was administered, and how it was analyzed.
pub administration: MeasurementMeta,
/// One entry per question, in the order it was printed.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub items: Vec<Measurement>,
}
/// What an administration was, for a reader two years later.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct MeasurementMeta {
/// The administration id these numbers came from.
pub id: String,
/// The assessment that was administered.
pub assessment: String,
/// The term.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub term: Option<String>,
/// The date it was given.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub date: Option<Date>,
/// The forms in play.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub forms: Vec<String>,
/// How many examinees the numbers pool over.
///
/// The one number to read before any of the others. A point-biserial on
/// twenty-seven students is a different kind of claim than one on three
/// hundred, and nothing below records how thin it is.
pub n_examinees: usize,
/// The item response model fitted, when one was.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model: Option<IrtModel>,
/// When the analysis was run.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generated: Option<Date>,
/// The version of the tool that ran it, since the numbers depend on it.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub coursebank: Option<String>,
}
/// One question's statistics from one administration.
///
/// Cohort aggregates only. There is deliberately no per-section or per-form
/// breakdown: those get small, and a small cell crossed with anything else is
/// how an aggregate stops being one.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Measurement {
/// The item id.
pub item: String,
/// The question number it was printed as.
pub number: u32,
/// The option set administered. See [`crate::item::Item::variant_digest`].
#[serde(default, skip_serializing_if = "Option::is_none")]
pub variant: Option<String>,
/// The stem as administered. See [`crate::item::Item::stem_digest`].
#[serde(default, skip_serializing_if = "Option::is_none")]
pub stem_digest: Option<String>,
/// Examinees who saw it.
pub n: usize,
/// Proportion correct.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub p_value: Option<f64>,
/// Corrected item-total point-biserial correlation.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub point_biserial: Option<f64>,
/// Upper-minus-lower-group discrimination index.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub discrimination_index: Option<f64>,
/// The option ids keyed correct, so a row in the options file says whether
/// it describes the answer or a distractor without a join.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub key: Vec<String>,
/// Per-option behavior, by option id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub option_stats: BTreeMap<String, OptionStat>,
/// Fitted parameters, when the sample supported a fit.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub irt: Option<IrtParams>,
/// Machine-detected problems with this question on this administration.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub flags: Vec<Flag>,
}
/// The columns of an items file, and the only ones accepted on read.
///
/// An allowlist rather than a type-level guarantee. In YAML the structures
/// reject unknown keys, so a file carrying a student column could not be
/// loaded; CSV readers are tolerant of extra columns, so the same assurance has
/// to be an explicit check. This is it, and [`MeasurementFile::read_csv`]
/// refuses any header not named here.
pub const ITEM_COLUMNS: [&str; 16] = [
"administration_id",
"assessment",
"term",
"date",
"forms",
"n_examinees",
"coursebank",
"generated",
"item",
"number",
"variant",
"stem_digest",
"n",
"p_value",
"point_biserial",
"discrimination_index",
];
/// The columns of an options file, and the only ones accepted on read.
pub const OPTION_COLUMNS: [&str; 9] = [
"administration_id",
"item",
"number",
"option",
"keyed",
"selection_rate",
"point_biserial",
"upper_group_rate",
"lower_group_rate",
];
/// One row of an items file.
#[derive(Debug, Clone, Serialize, Deserialize)]
struct ItemRow {
administration_id: String,
assessment: String,
term: String,
date: String,
forms: String,
n_examinees: usize,
coursebank: String,
generated: String,
item: String,
number: u32,
variant: String,
stem_digest: String,
n: usize,
p_value: String,
point_biserial: String,
discrimination_index: String,
}
/// One row of an options file.
#[derive(Debug, Clone, Serialize, Deserialize)]
struct OptionRow {
administration_id: String,
item: String,
number: u32,
option: String,
keyed: bool,
selection_rate: String,
point_biserial: String,
upper_group_rate: String,
lower_group_rate: String,
}
impl MeasurementFile {
/// The two file names this administration writes, items first.
///
/// # Arguments
///
/// * `dir` - usually `analysis/administrations`.
///
/// # Returns
///
/// The items path and the options path.
pub fn paths(&self, dir: &Path) -> (PathBuf, PathBuf) {
let stem = crate::course::slugify(&self.administration.id);
(
dir.join(format!("{stem}-items.csv")),
dir.join(format!("{stem}-options.csv")),
)
}
/// Writes the two files, refusing to overwrite either.
///
/// An administration is a thing that happened once, so replacing its record
/// is a deliberate act: delete the files first if you mean to re-analyze.
///
/// Two files rather than one because the data is two shapes — one row per
/// question, one row per question and option — and a single sparse table
/// serves neither. The administration's metadata repeats on every row,
/// which is what makes each file independently loadable and is the same
/// convention the response store already uses.
///
/// # Arguments
///
/// * `dir` - the directory to write into.
///
/// # Returns
///
/// The paths written.
///
/// # Errors
///
/// Returns [`Error::Usage`] when either file exists, and [`Error::Io`] or
/// [`Error::Csv`] on a write failure.
pub fn write_csv(&self, dir: &Path) -> Result<Vec<PathBuf>> {
let (items_path, options_path) = self.paths(dir);
for path in [&items_path, &options_path] {
if path.exists() {
return Err(Error::usage(format!(
"{} already records this administration. It happened once, so replacing it \
is a deliberate act: delete it first if you mean to re-analyze.",
path.display()
)));
}
}
std::fs::create_dir_all(dir).map_err(|e| Error::io(dir, e))?;
let meta = &self.administration;
let mut items = csv::Writer::from_path(&items_path).map_err(|e| Error::Csv {
path: items_path.clone(),
source: e,
})?;
let mut options = csv::Writer::from_path(&options_path).map_err(|e| Error::Csv {
path: options_path.clone(),
source: e,
})?;
for measurement in &self.items {
items
.serialize(ItemRow {
administration_id: meta.id.clone(),
assessment: meta.assessment.clone(),
term: meta.term.clone().unwrap_or_default(),
date: meta.date.map(|d| d.to_string()).unwrap_or_default(),
forms: meta.forms.join(";"),
n_examinees: meta.n_examinees,
coursebank: meta.coursebank.clone().unwrap_or_default(),
generated: meta.generated.map(|d| d.to_string()).unwrap_or_default(),
item: measurement.item.clone(),
number: measurement.number,
variant: measurement.variant.clone().unwrap_or_default(),
stem_digest: measurement.stem_digest.clone().unwrap_or_default(),
n: measurement.n,
p_value: number(measurement.p_value),
point_biserial: number(measurement.point_biserial),
discrimination_index: number(measurement.discrimination_index),
})
.map_err(|e| Error::Csv {
path: items_path.clone(),
source: e,
})?;
for (option, stat) in &measurement.option_stats {
options
.serialize(OptionRow {
administration_id: meta.id.clone(),
item: measurement.item.clone(),
number: measurement.number,
option: option.clone(),
keyed: measurement.key.iter().any(|k| k == option),
selection_rate: number(stat.selection_rate),
point_biserial: number(stat.point_biserial),
upper_group_rate: number(stat.upper_group_rate),
lower_group_rate: number(stat.lower_group_rate),
})
.map_err(|e| Error::Csv {
path: options_path.clone(),
source: e,
})?;
}
}
items.flush().map_err(|e| Error::io(&items_path, e))?;
options.flush().map_err(|e| Error::io(&options_path, e))?;
Ok(vec![items_path, options_path])
}
/// Reads one administration back from its two files.
///
/// # Arguments
///
/// * `items_path` - the items file. The options file is found beside it.
///
/// # Returns
///
/// The administration, with its per-option statistics reattached.
///
/// # Errors
///
/// Returns [`Error::Csv`] on a parse failure and [`Error::Invalid`] when a
/// file carries a column that is not in [`ITEM_COLUMNS`] or
/// [`OPTION_COLUMNS`] — which is how a student column is caught.
pub fn read_csv(items_path: &Path) -> Result<MeasurementFile> {
let options_path = PathBuf::from(
items_path
.to_string_lossy()
.replace("-items.csv", "-options.csv"),
);
let mut reader = open_csv(items_path, &ITEM_COLUMNS)?;
let mut meta = MeasurementMeta::default();
let mut items: Vec<Measurement> = Vec::new();
for row in reader.deserialize::<ItemRow>() {
let row = row.map_err(|e| Error::Csv {
path: items_path.to_path_buf(),
source: e,
})?;
meta = MeasurementMeta {
id: row.administration_id.clone(),
assessment: row.assessment.clone(),
term: some(&row.term),
date: parse_date(&row.date),
forms: row
.forms
.split(';')
.filter(|f| !f.is_empty())
.map(str::to_string)
.collect(),
n_examinees: row.n_examinees,
model: meta.model,
generated: parse_date(&row.generated),
coursebank: some(&row.coursebank),
};
items.push(Measurement {
item: row.item,
number: row.number,
variant: some(&row.variant),
stem_digest: some(&row.stem_digest),
n: row.n,
p_value: parse(&row.p_value),
point_biserial: parse(&row.point_biserial),
discrimination_index: parse(&row.discrimination_index),
..Measurement::default()
});
}
if options_path.is_file() {
let mut reader = open_csv(&options_path, &OPTION_COLUMNS)?;
for row in reader.deserialize::<OptionRow>() {
let row = row.map_err(|e| Error::Csv {
path: options_path.clone(),
source: e,
})?;
let Some(target) = items.iter_mut().find(|i| i.number == row.number) else {
continue;
};
if row.keyed && !target.key.contains(&row.option) {
target.key.push(row.option.clone());
}
target.option_stats.insert(
row.option,
OptionStat {
selection_rate: parse(&row.selection_rate),
point_biserial: parse(&row.point_biserial),
upper_group_rate: parse(&row.upper_group_rate),
lower_group_rate: parse(&row.lower_group_rate),
},
);
}
}
Ok(MeasurementFile {
schema_version: default_version(),
administration: meta,
items,
})
}
/// Reads every administration under a directory, oldest first.
///
/// # Arguments
///
/// * `dir` - usually `analysis/administrations`.
///
/// # Returns
///
/// The administrations, empty when the directory does not exist.
///
/// # Errors
///
/// Propagates read failures.
pub fn load_all(dir: &Path) -> Result<Vec<MeasurementFile>> {
if !dir.is_dir() {
return Ok(Vec::new());
}
let mut paths: Vec<PathBuf> = std::fs::read_dir(dir)
.map_err(|e| Error::io(dir, e))?
.filter_map(|e| e.ok().map(|e| e.path()))
.filter(|p| p.to_string_lossy().ends_with("-items.csv"))
.collect();
paths.sort();
let mut out = Vec::new();
for path in &paths {
out.push(MeasurementFile::read_csv(path)?);
}
out.sort_by(|a, b| {
a.administration
.date
.cmp(&b.administration.date)
.then(a.administration.id.cmp(&b.administration.id))
});
Ok(out)
}
}
/// Opens a CSV and refuses any column that is not on the allowlist.
fn open_csv(path: &Path, allowed: &[&str]) -> Result<csv::Reader<std::fs::File>> {
let mut reader = csv::Reader::from_path(path).map_err(|e| Error::Csv {
path: path.to_path_buf(),
source: e,
})?;
let headers = reader
.headers()
.map_err(|e| Error::Csv {
path: path.to_path_buf(),
source: e,
})?
.clone();
let unexpected: Vec<&str> = headers.iter().filter(|h| !allowed.contains(h)).collect();
if !unexpected.is_empty() {
return Err(Error::Invalid(vec![format!(
"{}: unexpected column(s) {}. These files are committed, so they hold cohort \
aggregates and nothing else — anything per-student belongs in data/, which is \
ignored.",
path.display(),
unexpected.join(", ")
)]));
}
Ok(reader)
}
/// A float as a CSV cell, empty when absent.
fn number(value: Option<f64>) -> String {
value.map(|v| format!("{v}")).unwrap_or_default()
}
/// A CSV cell as a float, absent when empty or unparseable.
fn parse(cell: &str) -> Option<f64> {
cell.trim().parse().ok()
}
/// A CSV cell as a string, absent when empty.
fn some(cell: &str) -> Option<String> {
(!cell.trim().is_empty()).then(|| cell.trim().to_string())
}
/// A CSV cell as a date, absent when empty or unparseable.
fn parse_date(cell: &str) -> Option<Date> {
some(cell).and_then(|c| c.parse().ok())
}
/// The schema version new files are written with.
fn default_version() -> String {
SCHEMA_VERSION.to_string()
}
/// A digest shortened for a message.
fn short(digest: &str) -> String {
digest.chars().take(8).collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::item::VariantCalibration;
fn tmp(tag: &str) -> std::path::PathBuf {
let p = std::env::temp_dir().join(format!("coursebank-cal-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&p);
std::fs::create_dir_all(&p).unwrap();
p
}
#[test]
fn an_absent_store_is_empty_rather_than_an_error() {
let dir = tmp("absent");
let store = CalibrationFile::load(&dir.join(CALIBRATION_FILE)).unwrap();
assert!(store.items.is_empty());
assert!(store.get("q-x").is_none());
}
#[test]
fn the_store_round_trips() {
let dir = tmp("round");
let path = dir.join(CALIBRATION_FILE);
let mut store = CalibrationFile::default();
store.items.insert(
"q-x".to_string(),
Calibration {
n_examinees: Some(27),
p_value: Some(0.63),
variants: vec![VariantCalibration {
variant: "4c81fa".into(),
n_examinees: Some(27),
..VariantCalibration::default()
}],
..Calibration::default()
},
);
store.save(&path).unwrap();
let back = CalibrationFile::load(&path).unwrap();
assert_eq!(back.get("q-x").unwrap().p_value, Some(0.63));
assert_eq!(back.get("q-x").unwrap().variants.len(), 1);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn an_administration_round_trips_through_two_csvs() {
let dir = tmp("csv");
let file = MeasurementFile {
schema_version: default_version(),
administration: MeasurementMeta {
id: "e1-2026f".into(),
assessment: "e1".into(),
term: Some("2026f".into()),
date: Some(Date::new(2026, 9, 15).unwrap()),
forms: vec!["A".into(), "B".into()],
n_examinees: 27,
model: None,
generated: Some(Date::new(2026, 9, 27).unwrap()),
coursebank: Some("0.0.0".into()),
},
items: vec![Measurement {
item: "q-fastq-quality-length-match".into(),
number: 1,
variant: Some("237d62f9af222f78".into()),
stem_digest: Some("8b22e0".into()),
n: 27,
p_value: Some(0.5926),
point_biserial: Some(0.31),
discrimination_index: None,
key: vec!["o-fourth-line".into()],
option_stats: [
(
"o-fourth-line".to_string(),
OptionStat {
selection_rate: Some(0.5926),
point_biserial: Some(0.31),
upper_group_rate: None,
lower_group_rate: None,
},
),
(
"o-third-line".to_string(),
OptionStat {
selection_rate: Some(0.1852),
point_biserial: Some(-0.18),
upper_group_rate: None,
lower_group_rate: None,
},
),
]
.into_iter()
.collect(),
irt: None,
flags: Vec::new(),
}],
};
let written = file.write_csv(&dir).unwrap();
assert_eq!(written.len(), 2, "one table per shape");
assert!(written[0].ends_with("e1-2026f-items.csv"));
assert!(written[1].ends_with("e1-2026f-options.csv"));
// One line per question, loadable on its own: the administration's
// metadata repeats on the row, as the response store already does.
let items = std::fs::read_to_string(&written[0]).unwrap();
assert!(
items.starts_with("administration_id,assessment,term,date,forms"),
"{items}"
);
assert!(
items.contains("e1-2026f,e1,2026f,2026-09-15,A;B,27"),
"{items}"
);
let options = std::fs::read_to_string(&written[1]).unwrap();
assert!(options.contains("o-fourth-line,true"), "{options}");
assert!(options.contains("o-third-line,false"), "{options}");
let back = MeasurementFile::read_csv(&written[0]).unwrap();
assert_eq!(back.administration.id, "e1-2026f");
assert_eq!(back.administration.n_examinees, 27);
assert_eq!(back.administration.forms, vec!["A", "B"]);
assert_eq!(back.items.len(), 1);
assert_eq!(back.items[0].p_value, Some(0.5926));
assert_eq!(back.items[0].key, vec!["o-fourth-line"]);
assert_eq!(back.items[0].option_stats.len(), 2);
assert_eq!(back.items[0].discrimination_index, None);
assert_eq!(MeasurementFile::load_all(&dir).unwrap().len(), 1);
// It happened once, so the record is not replaced by accident.
let err = file.write_csv(&dir).unwrap_err().to_string();
assert!(err.contains("deliberate"), "{err}");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_student_column_is_refused_on_read() {
let dir = tmp("identifiers");
let path = dir.join("e1-2026f-items.csv");
std::fs::write(
&path,
"administration_id,assessment,item,number,n,student_key\n e1-2026f,e1,q-x,1,27,abc123\n",
)
.unwrap();
// In YAML this fell out of the type, which rejects unknown keys. A CSV
// reader tolerates extra columns, so the same assurance has to be an
// explicit allowlist — and this is the test that it is one.
let err = MeasurementFile::read_csv(&path).unwrap_err().to_string();
assert!(err.contains("student_key"), "{err}");
assert!(err.contains("cohort aggregates"), "{err}");
let _ = std::fs::remove_dir_all(&dir);
}
}
+25 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Loading a whole course at once, and reporting on what it contains.
@@ -75,6 +75,12 @@ pub struct Catalog {
pub entries: Vec<Entry>,
/// Bank metadata by bank id.
pub banks: BTreeMap<String, crate::bank::BankMeta>,
/// The statistics, loaded from `analysis/` rather than from the banks.
///
/// Each entry's [`crate::item::Item::calibration`] is filled from this at
/// load, so everything downstream reads one item and does not have to know
/// that the numbers and the wording come from different files.
pub calibration: crate::calibration::CalibrationFile,
/// Map from global id to index into `entries`.
index: BTreeMap<String, usize>,
}
@@ -104,7 +110,10 @@ impl Catalog {
entries: Vec::new(),
banks: BTreeMap::new(),
index: BTreeMap::new(),
calibration: crate::calibration::CalibrationFile::default(),
};
catalog.calibration =
crate::calibration::CalibrationFile::load(&catalog.layout.calibration_file())?;
let mut problems = Vec::new();
let mut files = yaml::list_yaml(&catalog.layout.banks())?;
@@ -145,6 +154,15 @@ impl Catalog {
if !problems.is_empty() {
return Err(Error::Invalid(problems));
}
// Statistics are attached here rather than parsed from the bank, which
// is why an item can be read as one thing while its wording and its
// evidence live in files with different review cycles.
for entry in &mut catalog.entries {
if let Some(calibration) = catalog.calibration.items.get(&entry.uid) {
entry.item.calibration = Some(calibration.clone());
}
}
Ok(catalog)
}
@@ -791,6 +809,10 @@ impl Gap {
/// How much attention a finding deserves.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum Severity {
/// Information, not a problem: a count of entries waiting for approval, or
/// a convention that is reported but not enforced. `coursebank lint`
/// prints notes and does not fail on them.
Note,
/// Worth knowing.
Low,
/// Worth scheduling.
@@ -803,6 +825,7 @@ impl Severity {
/// A short label for report output.
pub fn label(self) -> &'static str {
match self {
Severity::Note => "note",
Severity::Low => "low",
Severity::Medium => "medium",
Severity::High => "high",
+387 -30
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The course file: identity plus the registries every bank references.
@@ -23,6 +23,7 @@
//! belongs to the course and the administration, never to the item.
pub mod fragment;
pub mod glossary;
use std::collections::BTreeMap;
use std::fmt;
@@ -34,13 +35,20 @@ use serde::{Deserialize, Deserializer, Serialize, Serializer};
use crate::date::Date;
use crate::error::{Error, Result};
use crate::taxonomy::Level;
use crate::taxonomy::{Format, Level};
use crate::yaml;
use fragment::Section;
use glossary::{Authoring, Equation, Material, Misconception, Term, Value};
/// The schema version this build of the tool writes.
pub const SCHEMA_VERSION: &str = "2.0";
///
/// 2.1 is additive: it adds the term, equation, value, and misconception
/// registries ([`glossary`]), short student-facing text on objectives, targets,
/// and lectures, and per-course exam conditions to [`Policy`]. Nothing is renamed
/// or removed and nothing already present becomes required, so a 2.0 course
/// loads and validates unchanged.
pub const SCHEMA_VERSION: &str = "2.1";
/// The schema major versions this build can read.
///
@@ -112,6 +120,22 @@ pub struct CourseFile {
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub stimuli: BTreeMap<String, Stimulus>,
/// Terms, keyed by id such as `term-gap-penalty`. See [`glossary`].
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub terms: BTreeMap<String, Term>,
/// Equations, keyed by id such as `eq-global-recurrence`.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub equations: BTreeMap<String, Equation>,
/// Taught numbers, keyed by id such as `val-per-decade`.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub values: BTreeMap<String, Value>,
/// Named mistakes, keyed by id such as `mis-only-matches-count`.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub misconceptions: BTreeMap<String, Misconception>,
/// Which file defined each id, relative to the course root.
///
/// Populated by [`fragment::assemble`] and empty for a course parsed
@@ -193,6 +217,169 @@ pub struct Policy {
/// student or an instructor actually acts on.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub grade_scale: Vec<GradeBand>,
/// How exams are sat, as students were told.
///
/// Pipelines and the linter read these instead of assuming one course's
/// rules: the objectives prompt used to hard-code closed book, no
/// calculator, no printed equations, and two minutes an item.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub exam_conditions: Option<ExamConditions>,
/// Named default blueprints, such as `module` and `final`, as published to
/// students. An assessment's own blueprint wins.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub blueprints: BTreeMap<String, PolicyBlueprint>,
/// The course's own names and one-line descriptions for levels 1 to 5, as
/// students were told them. A lecture page labels ceilings with these, and
/// falls back to the generic names when one is absent.
#[serde(
default,
deserialize_with = "level_keyed",
skip_serializing_if = "BTreeMap::is_empty"
)]
pub levels: BTreeMap<String, LevelName>,
/// Regular expressions that must not match objective or target text.
///
/// The objectives prompt's ban on "compute, calculate, estimate" was one
/// course's rule written into a shared tool; this makes it one course's
/// setting. The linter reports every match, case-insensitively.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub target_text_forbid: Vec<String>,
}
/// How exams are sat.
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct ExamConditions {
/// Whether exams are closed book.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub closed_book: Option<bool>,
/// Whether a calculator is allowed. When it is not, no item may need
/// arithmetic beyond a few seconds.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub calculator: Option<bool>,
/// Whether a relation an item turns on may appear.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub equations: Option<EquationPolicy>,
/// Item formats exams use now.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub formats: Vec<Format>,
/// The time budget an item is written to, in seconds.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub seconds_per_item: Option<f64>,
}
/// Whether an exam shows students the relations its items turn on.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum EquationPolicy {
/// Never: students bring the relation.
NeverPrinted,
/// On a supplied equation sheet.
Sheet,
/// In the stem of the item that needs it.
InStem,
}
/// A named default blueprint.
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct PolicyBlueprint {
/// Scored items at each level, keyed by level number.
#[serde(deserialize_with = "level_keyed")]
pub level_counts: BTreeMap<String, u32>,
/// Bonus items at each level, keyed by level number.
#[serde(
default,
deserialize_with = "level_keyed",
skip_serializing_if = "BTreeMap::is_empty"
)]
pub bonus_counts: BTreeMap<String, u32>,
/// The time allowed, in minutes.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub minutes: Option<f64>,
}
/// The course's name for one level.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct LevelName {
/// The name, as published: `Definitions`, `Application`.
pub name: String,
/// One line, as published.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
}
impl Policy {
/// The name a lecture page gives a level: the course's own, else the
/// generic one.
///
/// # Arguments
///
/// * `level` - the level.
///
/// # Returns
///
/// The display name.
pub fn level_name(&self, level: Level) -> String {
self.levels
.get(&level.code().to_string())
.map(|l| l.name.clone())
.unwrap_or_else(|| level.name().to_string())
}
}
/// Reads a map keyed by level number, whether YAML wrote the keys as `'1'` or
/// as `1`.
///
/// JSON Schema can only describe string keys, so the schema and the examples
/// quote them; an unquoted key is the natural thing to type, and loads too.
fn level_keyed<'de, D, T>(d: D) -> std::result::Result<BTreeMap<String, T>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de>,
{
/// One key, as a string or an integer.
struct Key(String);
impl<'de> Deserialize<'de> for Key {
fn deserialize<D: Deserializer<'de>>(d: D) -> std::result::Result<Key, D::Error> {
struct V;
impl Visitor<'_> for V {
type Value = Key;
fn expecting(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("a level number")
}
fn visit_str<E: de::Error>(self, v: &str) -> std::result::Result<Key, E> {
Ok(Key(v.to_string()))
}
fn visit_u64<E: de::Error>(self, v: u64) -> std::result::Result<Key, E> {
Ok(Key(v.to_string()))
}
fn visit_i64<E: de::Error>(self, v: i64) -> std::result::Result<Key, E> {
Ok(Key(v.to_string()))
}
}
d.deserialize_any(V)
}
}
impl PartialEq for Key {
fn eq(&self, other: &Key) -> bool {
self.0 == other.0
}
}
impl Eq for Key {}
impl PartialOrd for Key {
fn partial_cmp(&self, other: &Key) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}
impl Ord for Key {
fn cmp(&self, other: &Key) -> std::cmp::Ordering {
self.0.cmp(&other.0)
}
}
let raw: BTreeMap<Key, T> = BTreeMap::deserialize(d)?;
Ok(raw.into_iter().map(|(k, v)| (k.0, v)).collect())
}
/// One letter-grade band.
@@ -214,7 +401,7 @@ pub struct GradeBand {
/// The attainment word attached to the band, such as `Meritorious`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub attainment: Option<String>,
/// A colour group, so a report can tint A bands alike without parsing
/// A color group, so a report can tint A bands alike without parsing
/// letters. Defaults to the letter's first character.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub group: Option<String>,
@@ -250,6 +437,10 @@ impl Default for Policy {
mastery_threshold: mastery_default(),
min_items_for_mastery: 2,
grade_scale: Vec::new(),
exam_conditions: None,
blueprints: BTreeMap::new(),
levels: BTreeMap::new(),
target_text_forbid: Vec::new(),
}
}
}
@@ -301,11 +492,16 @@ pub struct Unit {
}
/// A lecture or class session.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Lecture {
/// Human title.
pub title: String,
/// The lecture in two to four sentences: what it covers and the arc its
/// objectives follow. Student-facing; the lecture page opens with it, and it
/// may hold brace references.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
/// The date it ran, when known.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub date: Option<Date>,
@@ -327,6 +523,11 @@ pub struct Lecture {
/// Assigned readings for the session, in the order you assign them.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub readings: Vec<Reading>,
/// Links the lecture page lists beside the readings: slides, the brief,
/// notebooks, the recording. [`Lecture::slides_url`] is the older
/// single-link form and still loads.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub materials: Vec<Material>,
}
/// A work the course cites: a textbook, an article, a dataset, a recording.
@@ -591,6 +792,8 @@ pub struct Reading {
pub focus: Option<String>,
/// What to gloss, and why it is out of scope.
pub skip: Option<String>,
/// Estimated reading time in minutes, for the lecture page's workload line.
pub minutes: Option<u32>,
/// A reading written as a bare string before this schema existed, held
/// unparsed.
pub text: Option<String>,
@@ -684,6 +887,9 @@ impl Serialize for Reading {
if let Some(v) = &self.skip {
map.serialize_entry("skip", v)?;
}
if let Some(v) = &self.minutes {
map.serialize_entry("minutes", v)?;
}
if let Some(v) = &self.text {
map.serialize_entry("text", v)?;
}
@@ -721,6 +927,8 @@ impl<'de> Deserialize<'de> for Reading {
#[serde(default)]
skip: Option<String>,
#[serde(default)]
minutes: Option<u32>,
#[serde(default)]
text: Option<String>,
}
@@ -751,6 +959,7 @@ impl<'de> Deserialize<'de> for Reading {
summary: m.summary,
focus: m.focus,
skip: m.skip,
minutes: m.minutes,
text: m.text,
})
}
@@ -794,21 +1003,39 @@ pub struct Objective {
/// The objective as you would state it to students. Reports quote this
/// verbatim, so write it in the second person and start with a verb.
pub text: String,
/// Two to four sentences under the objective on the lecture page: what it
/// is about, why it matters, and what it builds on or leads to.
/// Student-facing; may hold brace references.
///
/// The course guide asks students to read targets as a map. This is the
/// legend.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
/// The unit it belongs to.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub unit: Option<String>,
/// The lectures that develop it.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub lectures: Vec<String>,
/// Position in teaching order, low first.
/// Position in teaching order, low first. Derived; authoring it is
/// deprecated.
///
/// The registry is a map, so declaration order is lost on load, and sorting
/// The registry is a map, so declaration order is lost on load and sorting
/// by id would put `lo-enthalpy` before `lo-first-law` when the second is a
/// prerequisite of the first. Anything that prints objectives in the order
/// you teach them, a lecture page above all, needs this. Objectives without
/// it sort last, by id.
/// prerequisite of the first. Something has to supply the order.
///
/// Since 2.0 that something is [`Lecture::teaches`], which is a sequence:
/// the position of an objective in the list of what a lecture covers, and
/// the position of the lecture in the course, together say when it is
/// taught. [`fragment::assemble`] fills this in from those two, so a course
/// that used to carry thirty-nine hand-kept integers now carries none, and
/// inserting an objective is a one-line edit rather than a renumber.
///
/// An authored value still wins, so a 1.0 course loads unchanged.
/// `coursebank migrate order` removes them.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub order: Option<u32>,
/// The highest level you intend to assess this objective at. Assembling an
/// item above the ceiling is a warning: either the item overreaches or the
/// ceiling needs raising.
@@ -865,8 +1092,20 @@ pub struct Target {
///
/// Ordered within its objective rather than across the course, so two
/// targets under different objectives never compete for a position and
/// inserting one renumbers nothing outside its own group.
#[serde(default, skip_serializing_if = "Option::is_none")]
/// Retained only so a pre-2.0 course still loads. Ignored.
///
/// Targets do not have an order. An objective's targets are a set of
/// question templates, not steps in a sequence: they are not taught in
/// order, an exam samples from them rather than working through them, and
/// the study workflow reads the list as a checklist and counts what it can
/// do cold. A position would assert a sequence that does not exist.
///
/// Where a list has to be printed, [`CourseFile::targets`] orders it by
/// ceiling and then by id. Where one target genuinely depends on another,
/// that is [`Target::prerequisites`], which says so directly.
///
/// `coursebank migrate order` removes it.
#[serde(default, skip_serializing)]
pub order: Option<u32>,
/// The highest level you intend to assess this target at. Omit it to inherit
/// the objective's.
@@ -882,6 +1121,92 @@ pub struct Target {
/// unassessed objective exempts its targets regardless of this.
#[serde(default = "yes")]
pub assessed: bool,
/// The canonical account a student who has met this target could give, in
/// two to four sentences: the rule or mechanism, what changes the outcome,
/// and when it holds. Student-facing; may hold brace references.
///
/// Not a worked example. The lecture brief expands it, and slides and
/// items are checked against it. A student report quotes it under a missed
/// question, as what meeting the target looks like.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub explanation: Option<String>,
/// Term ids a student needs to meet this target.
///
/// A term is new on the first lecture, in teaching order, whose targets
/// list it, and listed as earlier vocabulary on later ones.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub terms: Vec<String>,
/// Equation ids this target turns on.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub equations: Vec<String>,
/// Value ids this target uses.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub values: Vec<String>,
/// Misconception ids an item on this target is most likely to draw on.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub misconceptions: Vec<String>,
/// Notes for whoever writes items, slides, or the brief from this target.
/// Instructor-facing: never rendered for students.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub authoring: Option<Authoring>,
}
impl Default for Objective {
fn default() -> Objective {
Objective {
text: String::new(),
description: None,
unit: None,
lectures: Vec::new(),
order: None,
level_ceiling: None,
prerequisites: Vec::new(),
tags: Vec::new(),
assessed: true,
}
}
}
impl Default for Target {
fn default() -> Target {
Target {
text: String::new(),
objective: String::new(),
lectures: Vec::new(),
order: None,
level_ceiling: None,
prerequisites: Vec::new(),
tags: Vec::new(),
assessed: true,
explanation: None,
terms: Vec::new(),
equations: Vec::new(),
values: Vec::new(),
misconceptions: Vec::new(),
authoring: None,
}
}
}
impl Target {
/// The ids this target lists in one registry.
///
/// # Arguments
///
/// * `kind` - which registry.
///
/// # Returns
///
/// The listed ids, empty for the objective and target kinds.
pub fn listed(&self, kind: glossary::RefKind) -> &[String] {
match kind {
glossary::RefKind::Term => &self.terms,
glossary::RefKind::Equation => &self.equations,
glossary::RefKind::Value => &self.values,
glossary::RefKind::Misconception => &self.misconceptions,
glossary::RefKind::Objective | glossary::RefKind::Target => &[],
}
}
}
/// The prefix an objective id is expected to carry.
@@ -1183,6 +1508,33 @@ impl CourseFile {
));
}
}
// A manuscript with no journal is a citation nobody can print. The
// fields exist; a note that carries them instead is data the reading
// list cannot link and the bibliography exporters cannot use.
if matches!(
reference.kind,
ReferenceKind::Article | ReferenceKind::Preprint
) && reference.container.is_none()
{
issues.push(format!(
"reference `{key}`: an {} needs a `container` — the journal, preprint \
server, or proceedings it appeared in. Run `coursebank migrate references` \
if it is sitting in the `note`.",
match reference.kind {
ReferenceKind::Preprint => "preprint",
_ => "article",
}
));
}
if let Some(note) = &reference.note {
if crate::citation::looks_like_a_citation(note) {
issues.push(format!(
"reference `{key}`: the note still carries a citation. Volume, pages, \
and DOI have their own fields, and a DOI in a note is a link nobody \
can follow. `coursebank migrate references` takes it apart."
));
}
}
if let Some(pmid) = &reference.pmid {
if !pmid.trim().chars().all(|c| c.is_ascii_digit()) {
issues.push(format!(
@@ -1276,6 +1628,7 @@ impl CourseFile {
}
issues.extend(self.prerequisite_cycles());
issues.extend(self.glossary_problems());
issues
}
@@ -1646,9 +1999,20 @@ impl CourseFile {
.filter(|(_, target)| target.objective == objective)
.map(|(id, _)| id)
.collect();
// By ceiling, then by id. A target is a question template rather than a
// step in a sequence — an objective's targets are not taught in an
// order, and an exam samples from them — so there is no teaching order
// to print. What there is is depth, and grouping by it puts the
// checklist in the order the study methods apply: recall for a Level 1
// target, explanation for Level 2, variations and written solutions
// above that. Ties break by id so the list is stable.
ids.sort_by_key(|id| {
let target = &self.learning_targets[*id];
(target.order.unwrap_or(u32::MAX), (*id).clone())
(
self.effective_level_ceiling(id)
.map(|l| l.code())
.unwrap_or(0),
(*id).clone(),
)
});
ids.into_iter().map(String::as_str).collect()
}
@@ -2098,11 +2462,8 @@ impl CourseFile {
"L01".to_string(),
Lecture {
title: "Course introduction".to_string(),
date: None,
unit: Some("u-intro".to_string()),
slides_url: None,
teaches: Vec::new(),
readings: Vec::new(),
..Lecture::default()
},
);
let mut los = BTreeMap::new();
@@ -2114,9 +2475,7 @@ impl CourseFile {
lectures: vec!["L01".to_string()],
order: Some(1),
level_ceiling: Some(Level::Understand),
prerequisites: Vec::new(),
tags: Vec::new(),
assessed: true,
..Objective::default()
},
);
// One target as well, because the two tiers are easier to understand
@@ -2130,10 +2489,7 @@ impl CourseFile {
objective: "lo-example".to_string(),
lectures: vec!["L01".to_string()],
order: Some(1),
level_ceiling: None,
prerequisites: Vec::new(),
tags: Vec::new(),
assessed: true,
..Target::default()
},
);
CourseFile {
@@ -2157,6 +2513,10 @@ impl CourseFile {
learning_targets: targets,
references: BTreeMap::new(),
stimuli: BTreeMap::new(),
terms: BTreeMap::new(),
equations: BTreeMap::new(),
values: BTreeMap::new(),
misconceptions: BTreeMap::new(),
origins: BTreeMap::new(),
}
}
@@ -2520,10 +2880,7 @@ lectures:
unit: None,
lectures: vec!["L1.1".to_string()],
order: Some(3),
level_ceiling: None,
prerequisites: Vec::new(),
tags: Vec::new(),
assessed: true,
..Objective::default()
},
);
assert_eq!(c.targets_without_readings(), vec!["lo-orphan"]);
+243 -8
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! One course, several files.
@@ -12,7 +12,10 @@
//! course.yaml course, policy, units
//! references.yaml references
//! lectures/l-1-2.yaml one lecture, its readings, and what it teaches
//! objectives/lo-x.yaml one objective and its targets
//! objectives/lo-x.yaml one objective, its targets, and the glossary entries
//! its targets are the first to need
//! glossary.yaml terms, equations, values, and misconceptions that
//! belong to no single objective
//! ```
//!
//! The merge happens in memory on every command. Nothing is generated on disk
@@ -58,6 +61,7 @@ use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use super::glossary::{Equation, Misconception, Term, Value};
use super::{
COURSE_FILE, Course, CourseFile, Lecture, Objective, Policy, Reference, SCHEMA_VERSION,
SUPPORTED_MAJORS, Stimulus, Target, Unit,
@@ -69,6 +73,11 @@ use crate::yaml;
/// The file holding the bibliography when it is kept out of `course.yaml`.
pub const REFERENCES_FILE: &str = "references.yaml";
/// The file holding glossary entries that belong to no single objective: the
/// cross-cutting ones, and everything `coursebank migrate glossary` brings in
/// until each entry is moved to the objective file whose targets first need it.
pub const GLOSSARY_FILE: &str = "glossary.yaml";
/// One registry section of a course.
///
/// Used to say which file a fact came from, and to keep a fragment from
@@ -91,11 +100,19 @@ pub enum Section {
References,
/// Shared stimuli.
Stimuli,
/// Terms.
Terms,
/// Equations.
Equations,
/// Taught values.
Values,
/// Named misconceptions.
Misconceptions,
}
impl Section {
/// Every section, in the order a merged course lists them.
pub const ALL: [Section; 8] = [
pub const ALL: [Section; 12] = [
Section::Course,
Section::Policy,
Section::Units,
@@ -104,6 +121,19 @@ impl Section {
Section::Targets,
Section::References,
Section::Stimuli,
Section::Terms,
Section::Equations,
Section::Values,
Section::Misconceptions,
];
/// The four registries 2.1 adds, which any objective file, `glossary.yaml`,
/// or `course.yaml` may define.
pub const GLOSSARY: [Section; 4] = [
Section::Terms,
Section::Equations,
Section::Values,
Section::Misconceptions,
];
/// The YAML key this section is written under.
@@ -117,6 +147,10 @@ impl Section {
Section::Targets => "learning_targets",
Section::References => "references",
Section::Stimuli => "stimuli",
Section::Terms => "terms",
Section::Equations => "equations",
Section::Values => "values",
Section::Misconceptions => "misconceptions",
}
}
@@ -131,6 +165,10 @@ impl Section {
Section::Targets => "target",
Section::References => "reference",
Section::Stimuli => "stimulus",
Section::Terms => "term",
Section::Equations => "equation",
Section::Values => "value",
Section::Misconceptions => "misconception",
}
}
@@ -141,6 +179,9 @@ impl Section {
Section::Lectures => "lectures/*.yaml",
Section::Objectives | Section::Targets | Section::Stimuli => "objectives/*.yaml",
Section::References => REFERENCES_FILE,
Section::Terms | Section::Equations | Section::Values | Section::Misconceptions => {
"objectives/*.yaml or glossary.yaml"
}
}
}
}
@@ -162,6 +203,8 @@ pub enum Role {
Lecture,
/// A file under `objectives/`.
Objective,
/// `glossary.yaml`.
Glossary,
}
impl Role {
@@ -171,10 +214,13 @@ impl Role {
Role::Root => true,
Role::References => section == Section::References,
Role::Lecture => section == Section::Lectures,
Role::Objective => matches!(
section,
Section::Objectives | Section::Targets | Section::Stimuli
),
Role::Objective => {
matches!(
section,
Section::Objectives | Section::Targets | Section::Stimuli
) || Section::GLOSSARY.contains(&section)
}
Role::Glossary => Section::GLOSSARY.contains(&section),
}
}
@@ -185,6 +231,7 @@ impl Role {
Role::References => "references",
Role::Lecture => "lecture",
Role::Objective => "objective",
Role::Glossary => "glossary",
}
}
}
@@ -237,6 +284,22 @@ pub struct Fragment {
/// Shared stimuli by id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub stimuli: BTreeMap<String, Stimulus>,
/// Terms by id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub terms: BTreeMap<String, Term>,
/// Equations by id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub equations: BTreeMap<String, Equation>,
/// Taught values by id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub values: BTreeMap<String, Value>,
/// Named misconceptions by id.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub misconceptions: BTreeMap<String, Misconception>,
}
impl Fragment {
@@ -286,6 +349,18 @@ impl Fragment {
if !self.stimuli.is_empty() {
out.push(Section::Stimuli);
}
if !self.terms.is_empty() {
out.push(Section::Terms);
}
if !self.equations.is_empty() {
out.push(Section::Equations);
}
if !self.values.is_empty() {
out.push(Section::Values);
}
if !self.misconceptions.is_empty() {
out.push(Section::Misconceptions);
}
out
}
}
@@ -315,6 +390,10 @@ pub fn files(layout: &Layout) -> Result<Vec<(PathBuf, Role)>> {
if references.is_file() {
out.push((references, Role::References));
}
let glossary = layout.glossary_file();
if glossary.is_file() {
out.push((glossary, Role::Glossary));
}
for path in yaml::list_yaml(&layout.lectures())? {
out.push((path, Role::Lecture));
}
@@ -371,6 +450,10 @@ struct Merge {
targets: BTreeMap<String, Target>,
references: BTreeMap<String, Reference>,
stimuli: BTreeMap<String, Stimulus>,
terms: BTreeMap<String, Term>,
equations: BTreeMap<String, Equation>,
values: BTreeMap<String, Value>,
misconceptions: BTreeMap<String, Misconception>,
origins: BTreeMap<(Section, String), PathBuf>,
issues: Vec<String>,
}
@@ -492,6 +575,46 @@ impl Merge {
&mut self.issues,
);
}
if role.allows(Section::Terms) {
absorb(
&mut self.terms,
fragment.terms,
Section::Terms,
path,
&mut self.origins,
&mut self.issues,
);
}
if role.allows(Section::Equations) {
absorb(
&mut self.equations,
fragment.equations,
Section::Equations,
path,
&mut self.origins,
&mut self.issues,
);
}
if role.allows(Section::Values) {
absorb(
&mut self.values,
fragment.values,
Section::Values,
path,
&mut self.origins,
&mut self.issues,
);
}
if role.allows(Section::Misconceptions) {
absorb(
&mut self.misconceptions,
fragment.misconceptions,
Section::Misconceptions,
path,
&mut self.origins,
&mut self.issues,
);
}
}
/// Records a section that may only be declared once.
@@ -535,6 +658,27 @@ impl Merge {
}
}
// Teaching order, from the two sequences that already declare it: the
// lectures in course order, and each lecture's `teaches` list. This is
// what lets an objective stop carrying a hand-kept integer.
let mut position = 0u32;
let ordered: Vec<String> = self.lectures.keys().cloned().collect();
for lecture_id in ordered {
let teaches = self
.lectures
.get(&lecture_id)
.map(|l| l.teaches.clone())
.unwrap_or_default();
for objective_id in teaches {
position += 1;
if let Some(objective) = self.objectives.get_mut(&objective_id) {
if objective.order.is_none() {
objective.order = Some(position);
}
}
}
}
// A target with no lecture of its own is taught wherever its objective
// is. Collected first: the read of `objectives` and the write to
// `targets` cannot overlap in one pass.
@@ -585,6 +729,10 @@ impl Merge {
learning_targets: self.targets,
references: self.references,
stimuli: self.stimuli,
terms: self.terms,
equations: self.equations,
values: self.values,
misconceptions: self.misconceptions,
origins: self.origins,
})
}
@@ -800,6 +948,93 @@ units:
assert!(course.origin("nothing-like-this").is_none());
}
#[test]
fn teaching_order_is_derived_from_the_two_sequences_that_declare_it() {
let root = tmp("order");
write(&root, "course.yaml", ROOT);
write(
&root,
"lectures/l-1-2.yaml",
"lectures:\n L1.2:\n title: One\n teaches: [lo-second, lo-first]\n",
);
write(
&root,
"lectures/l-1-3.yaml",
"lectures:\n L1.3:\n title: Two\n teaches: [lo-third]\n",
);
write(
&root,
"objectives/lo-first.yaml",
r#"learning_objectives:
lo-first:
text: A.
lo-second:
text: B.
lo-third:
text: C.
"#,
);
let course = assemble(&root).unwrap();
// Position in `teaches`, not id order: the lecture lists `lo-second`
// first and that is what teaching it first means.
assert_eq!(course.learning_objectives["lo-second"].order, Some(1));
assert_eq!(course.learning_objectives["lo-first"].order, Some(2));
assert_eq!(course.learning_objectives["lo-third"].order, Some(3));
assert_eq!(
course.lecture_objectives("L1.2"),
vec!["lo-second", "lo-first"]
);
}
#[test]
fn targets_print_by_ceiling_rather_than_in_a_sequence() {
let root = tmp("target-order");
write(&root, "course.yaml", ROOT);
write(
&root,
"lectures/l-1-2.yaml",
"lectures:\n L1.2:\n title: One\n teaches: [lo-x]\n",
);
write(
&root,
"objectives/lo-x.yaml",
r#"learning_objectives:
lo-x:
text: A.
level_ceiling: 3
learning_targets:
t-predict:
text: Predict the effect.
objective: lo-x
level_ceiling: 3
t-define:
text: Define the term.
objective: lo-x
level_ceiling: 1
t-explain:
text: Explain the mechanism.
objective: lo-x
level_ceiling: 2
"#,
);
let course = assemble(&root).unwrap();
assert!(course.validate().is_empty(), "{:?}", course.validate());
// Shallowest first, which is the order the study methods apply in:
// recall, then explanation, then variations. Not id order, which would
// put `t-define` after `t-predict` for no reason at all.
assert_eq!(
course.targets("lo-x"),
vec!["t-define", "t-explain", "t-predict"]
);
// A target with no ceiling of its own inherits the objective's, so it
// sorts where that puts it.
assert_eq!(course.learning_targets["t-define"].order, None);
}
#[test]
fn teaching_the_same_objective_from_two_lectures_unions() {
let root = tmp("union");
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The usage history of a course, derived by scanning assessment records.
+20 -5
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The item: one assessable question, with both the intent it was written with
@@ -157,7 +157,14 @@ pub struct Item {
pub design: Option<Design>,
/// What the evidence says, accumulated across administrations.
#[serde(default, skip_serializing_if = "Option::is_none")]
/// What the statistics say, filled in from `analysis/` at load.
///
/// Read from the store and never written back: `skip_serializing` means a
/// bank file cannot acquire a `calibration:` block by being round-tripped
/// through this type. A bank is a reviewed artifact whose diff should be a
/// change of intent, and every grading run would otherwise produce a diff
/// on it. See [`crate::calibration`].
#[serde(default, skip_serializing)]
pub calibration: Option<Calibration>,
/// The last review decision recorded for this item.
@@ -235,6 +242,14 @@ pub struct Choice {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub misconception: Option<String>,
/// The registry misconception this distractor encodes, `mis-...`.
///
/// Reports group misses by it across items. The option's own
/// [`Choice::misconception`] text, when present, overrides the registry's
/// `belief` for this item; when absent, reports use the registry's.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub misconception_id: Option<String>,
/// The category of that error.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub error_type: Option<ErrorType>,
@@ -497,7 +512,7 @@ pub struct Solution {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model_answer: Option<String>,
/// The worked reasoning a student can learn from: the derivation, the estimate,
/// the argument for the key over its neighbours. This is the body of the
/// the argument for the key over its neighbors. This is the body of the
/// solutions document.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub explanation: Option<String>,
@@ -667,7 +682,7 @@ pub struct VariantCalibration {
/// Upper-minus-lower-group discrimination index.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub discrimination_index: Option<f64>,
/// Per-option behaviour within this set, keyed by option id.
/// Per-option behavior within this set, keyed by option id.
#[serde(default, skip_serializing_if = "std::collections::BTreeMap::is_empty")]
pub option_stats: std::collections::BTreeMap<String, OptionStat>,
/// Fitted item response theory parameters for this set.
+28 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The on-disk layout of a course directory.
@@ -49,6 +49,12 @@ impl Layout {
self.root.join(crate::course::fragment::REFERENCES_FILE)
}
/// `glossary.yaml`, the terms, equations, values, and misconceptions that
/// belong to no single objective.
pub fn glossary_file(&self) -> PathBuf {
self.root.join(crate::course::fragment::GLOSSARY_FILE)
}
/// Directory holding one file per lecture.
pub fn lectures(&self) -> PathBuf {
self.root.join("lectures")
@@ -64,6 +70,24 @@ impl Layout {
self.root.join("banks")
}
/// Directory holding the statistics, which are kept out of the banks.
///
/// Committed, unlike [`Layout::data`]: everything under here is a cohort
/// aggregate with no student in it. See [`crate::calibration`].
pub fn analysis(&self) -> PathBuf {
self.root.join("analysis")
}
/// The pooled per-item calibration store.
pub fn calibration_file(&self) -> PathBuf {
self.analysis().join(crate::calibration::CALIBRATION_FILE)
}
/// Directory holding one immutable record per administration.
pub fn measurements(&self) -> PathBuf {
self.analysis().join("administrations")
}
/// Directory holding assessment records.
pub fn assessments(&self) -> PathBuf {
self.root.join("assessments")
@@ -117,6 +141,8 @@ impl Layout {
pub fn create_all(&self) -> Result<()> {
for dir in [
self.root.clone(),
self.analysis(),
self.measurements(),
self.lectures(),
self.objectives(),
self.banks(),
+82 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The sealed administration: what the students actually held in their hands.
@@ -124,6 +124,26 @@ pub struct SealMeta {
/// from the file's own contents; [`SealFile::verify`] rebuilds a seal from the
/// live course and compares this against it.
pub digest: String,
/// The digests this file had before it was rewritten, oldest first.
///
/// A seal is meant to be tamper-evident, which puts a migration that
/// renames an id inside it in an awkward position: the id is part of
/// [`SealFile::digest_input`], so rewriting the file invalidates the digest
/// that vouched for it, and recomputing it quietly would produce a record
/// that looks untouched and is not.
///
/// So a migration does both — recomputes the digest and leaves the old one
/// here. What the seal then says is the honest thing: these are the
/// contents, this is what they hash to, and here is what the file hashed to
/// before each rewrite. Anyone holding an earlier copy, a backup or a git
/// revision, can check it against the right entry.
///
/// Deliberately outside [`SealFile::digest_input`]: a field that recorded
/// past digests and was itself covered by the current one could not be
/// appended to without invalidating what it describes.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub superseded_digests: Vec<String>,
}
/// What wrote a seal.
@@ -458,6 +478,7 @@ pub fn build(catalog: &Catalog, record: &AssessmentFile, opts: &Options) -> Resu
content: opts.content,
dropped,
digest: String::new(),
superseded_digests: Vec::new(),
},
items,
forms: sealed_forms,
@@ -787,6 +808,39 @@ impl SealFile {
buf
}
/// Renames the items this seal froze, keeping the digest honest.
///
/// The only sanctioned way to rewrite a seal. It recomputes the digest,
/// because the ids are part of what the digest covers, and records the
/// previous one in [`SealMeta::superseded_digests`], because a recomputed
/// digest with no trace of the recompute is a record that claims never to
/// have been touched.
///
/// # Arguments
///
/// * `rename` - a map from old item id to new.
///
/// # Returns
///
/// How many placements were renamed. Zero leaves the file alone, digest
/// included.
pub fn rename_items(&mut self, rename: &BTreeMap<String, String>) -> usize {
let mut renamed = 0;
for item in &mut self.items {
if let Some(new) = rename.get(&item.item) {
item.item = new.clone();
renamed += 1;
}
}
if renamed == 0 {
return 0;
}
let previous = std::mem::take(&mut self.seal.digest);
self.seal.superseded_digests.push(previous);
self.seal.digest = self.recompute_digest();
renamed
}
/// Recomputes this file's digest from its own contents.
///
/// # Returns
@@ -1195,6 +1249,7 @@ mod tests {
content: true,
dropped: Vec::new(),
digest: String::new(),
superseded_digests: Vec::new(),
},
items: vec![SealedItem {
number: 1,
@@ -1343,6 +1398,31 @@ mod tests {
assert_eq!(short("sha256:0123456789abcdef"), "sha256:01234567");
}
#[test]
fn renaming_an_item_recomputes_the_digest_and_says_it_did() {
let mut seal = sample();
seal.seal.digest = seal.recompute_digest();
let original = seal.seal.digest.clone();
assert!(seal.check_self().is_empty());
let mut rename = BTreeMap::new();
rename.insert(seal.items[0].item.clone(), "q-renamed".to_string());
assert_eq!(seal.rename_items(&rename), 1);
// The digest covers the ids, so it has to move — and the file has to
// admit that it moved rather than looking untouched.
assert_eq!(seal.items[0].item, "q-renamed");
assert_ne!(seal.seal.digest, original);
assert_eq!(seal.seal.superseded_digests, vec![original]);
assert!(seal.check_self().is_empty(), "{:?}", seal.check_self());
// A rename that matches nothing leaves the file entirely alone.
let steady = seal.seal.digest.clone();
assert_eq!(seal.rename_items(&BTreeMap::new()), 0);
assert_eq!(seal.seal.digest, steady);
assert_eq!(seal.seal.superseded_digests.len(), 1);
}
#[test]
fn round_trips_through_yaml() {
let file = sample();
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The pedagogical vocabulary: levels, cognitive processes, error types, and
+3 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Small self-contained utilities with no knowledge of courses or assessments.
@@ -9,6 +9,7 @@
//! replaces a dependency that would otherwise need to keep working for as long as
//! a course repository needs to stay readable.
pub mod citation;
pub mod date;
pub mod hash;
pub mod markup;
+330
View File
@@ -0,0 +1,330 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Pulling a citation apart when it was written as prose.
//!
//! A bibliography assembled by hand tends to collect entries like
//!
//! ```text
//! note: 'Nucleic Acids Res 25:3389-3402. doi:10.1093/nar/25.17.3389'
//! ```
//!
//! which is a complete citation in a field that means "anything else worth
//! saying". Nothing can use it: a reading list cannot link the DOI, an export to
//! Hayagriva or BibTeX has no journal to put in `parent` or `journal`, and the
//! `container`, `volume`, `pages`, and `doi` fields sit empty beside it.
//!
//! [`parse`] takes such a note apart. What it cannot account for it leaves in
//! the note, which is the important half of the contract: a note reading
//! `'Bioinformatics 18:440-445. Origin of spaced seeds.'` yields the journal,
//! the volume, the pages, and a note that still says where spaced seeds came
//! from. Nothing is discarded and nothing is invented — an issue number that was
//! never written down stays absent, even when a publisher's DOI happens to
//! encode one.
/// The parts of a citation recovered from a note.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Parsed {
/// The journal, proceedings, or book the work appeared in.
pub container: Option<String>,
/// The volume.
pub volume: Option<String>,
/// The page range, as `first-last`.
pub pages: Option<String>,
/// The DOI, bare.
pub doi: Option<String>,
/// What the note still says after the citation is removed.
pub note: Option<String>,
}
/// Takes a citation apart, leaving the rest of the note alone.
///
/// # Arguments
///
/// * `note` - the note as written.
/// * `year` - the record's year, which is how a trailing year is recognized as
/// part of a conference name rather than part of the title of the venue.
///
/// # Returns
///
/// The parts found. Every field is independently optional: a note that carries
/// only a DOI yields only a DOI.
pub fn parse(note: &str, year: Option<u32>) -> Parsed {
let mut out = Parsed::default();
let mut rest = note.trim().to_string();
if let Some((container, volume, pages, tail)) = citation(&rest) {
out.container = Some(container);
out.volume = Some(volume);
out.pages = Some(pages);
rest = tail;
}
if let Some((doi, tail)) = doi(&rest) {
out.doi = Some(doi);
rest = tail;
}
// A venue with no volume or pages — a conference, usually — is named by the
// clause that ends in the year the work was published.
if out.container.is_none() {
if let Some(y) = year {
if let Some((container, tail)) = venue(&rest, y) {
out.container = Some(container);
rest = tail;
}
}
}
// The year belongs to the record, not to the name of the venue.
if let (Some(container), Some(y)) = (&out.container, year) {
let suffix = format!(" {y}");
if let Some(trimmed) = container.strip_suffix(&suffix) {
out.container = Some(trimmed.trim_end().to_string());
}
}
let rest = rest.trim().trim_start_matches('.').trim().to_string();
out.note = (!rest.is_empty()).then_some(rest);
out
}
/// Finds `Journal 25:3389-3402` or `Journal 48, 443-453` at the start.
///
/// The volume is the first digit run that follows a space and is followed by a
/// separator and a page range. Requiring the whole shape is what keeps a year in
/// a conference name (`Proc. FOCS 2000.`) from being read as a volume.
///
/// # Returns
///
/// The container, volume, page range, and whatever followed.
fn citation(text: &str) -> Option<(String, String, String, String)> {
let bytes = text.as_bytes();
let mut at = 0;
while at < bytes.len() {
// A volume follows a space, so that a digit inside a name is not one.
if !(bytes[at].is_ascii_digit() && at > 0 && bytes[at - 1] == b' ') {
at += 1;
continue;
}
let volume_start = at;
let volume_end = digits(bytes, volume_start);
let mut cursor = spaces(bytes, volume_end);
// The separator between volume and pages is a colon or a comma.
if cursor < bytes.len() && (bytes[cursor] == b':' || bytes[cursor] == b',') {
cursor = spaces(bytes, cursor + 1);
let first_start = cursor;
let first_end = digits(bytes, first_start);
if first_end > first_start {
let dash = text[first_end..]
.strip_prefix('-')
.or_else(|| text[first_end..].strip_prefix('\u{2013}'));
if let Some(after_dash) = dash {
let last_offset = text.len() - after_dash.len();
let last_end = digits(bytes, last_offset);
if last_end > last_offset {
let container = text[..volume_start].trim_end_matches([' ', ',']);
if !container.is_empty() {
return Some((
container.to_string(),
text[volume_start..volume_end].to_string(),
format!(
"{}-{}",
&text[first_start..first_end],
&text[last_offset..last_end]
),
text[last_end..].to_string(),
));
}
}
}
}
}
at = volume_end;
}
None
}
/// Finds a `doi:10.…` anywhere in the text.
///
/// # Returns
///
/// The DOI and the text with it removed.
fn doi(text: &str) -> Option<(String, String)> {
let lower = text.to_ascii_lowercase();
let at = lower.find("doi:")?;
let after = text[at + 4..].trim_start();
let offset = text.len() - after.len();
let end = after
.find(char::is_whitespace)
.map(|n| offset + n)
.unwrap_or(text.len());
let doi = text[offset..end].trim_end_matches('.');
if !doi.starts_with("10.") {
return None;
}
let mut remainder = String::from(text[..at].trim_end());
let tail = text[end..].trim();
if !tail.is_empty() {
if !remainder.is_empty() {
remainder.push(' ');
}
remainder.push_str(tail);
}
Some((doi.to_string(), remainder))
}
/// Finds a leading clause ending in the publication year: `Proc. FOCS 2000.`
///
/// # Returns
///
/// The clause without its trailing period, and whatever followed.
fn venue(text: &str, year: u32) -> Option<(String, String)> {
let needle = format!("{year}.");
let at = text.find(&needle)?;
let clause = text[..at + needle.len() - 1].trim();
if clause.is_empty() {
return None;
}
Some((
clause.to_string(),
text[at + needle.len()..].trim().to_string(),
))
}
/// The end of a run of ASCII digits starting at `from`.
fn digits(bytes: &[u8], from: usize) -> usize {
let mut at = from;
while at < bytes.len() && bytes[at].is_ascii_digit() {
at += 1;
}
at
}
/// The end of a run of spaces starting at `from`.
fn spaces(bytes: &[u8], from: usize) -> usize {
let mut at = from;
while at < bytes.len() && bytes[at] == b' ' {
at += 1;
}
at
}
/// Whether a note still looks like it is carrying a citation.
///
/// Used by validation to say so, rather than leaving a note that a reading list
/// cannot link and an export cannot use.
///
/// # Arguments
///
/// * `note` - the note as written.
///
/// # Returns
///
/// `true` when a volume and page range, or a DOI, can be found in it.
pub fn looks_like_a_citation(note: &str) -> bool {
citation(note.trim()).is_some() || doi(note.trim()).is_some()
}
#[cfg(test)]
mod tests {
use super::*;
/// Every article note in a real course bibliography, which is where the
/// shapes below come from. Two separator styles, DOIs in three positions,
/// a conference with no volume, and prose that has to survive.
#[test]
fn a_journal_citation_comes_apart() {
let p = parse(
"Nucleic Acids Res 25:3389-3402. doi:10.1093/nar/25.17.3389",
Some(1997),
);
assert_eq!(p.container.as_deref(), Some("Nucleic Acids Res"));
assert_eq!(p.volume.as_deref(), Some("25"));
assert_eq!(p.pages.as_deref(), Some("3389-3402"));
assert_eq!(p.doi.as_deref(), Some("10.1093/nar/25.17.3389"));
assert_eq!(p.note, None);
// The DOI encodes volume 25, issue 17. Nothing infers the issue from
// it: a field nobody wrote down stays empty.
}
#[test]
fn an_abbreviation_keeps_its_final_period() {
let p = parse("J. Mol. Biol. 48, 443-453.", Some(1970));
assert_eq!(p.container.as_deref(), Some("J. Mol. Biol."));
assert_eq!(p.volume.as_deref(), Some("48"));
assert_eq!(p.pages.as_deref(), Some("443-453"));
assert_eq!(p.note, None);
}
#[test]
fn prose_after_a_citation_stays_in_the_note() {
let p = parse(
"Bioinformatics 18:440-445. Origin of spaced seeds.",
Some(2002),
);
assert_eq!(p.container.as_deref(), Some("Bioinformatics"));
assert_eq!(p.note.as_deref(), Some("Origin of spaced seeds."));
// A caveat the author wrote is the last thing to throw away.
let p = parse("J Mol Biol 215:403-410. Verify before use.", Some(1990));
assert_eq!(p.note.as_deref(), Some("Verify before use."));
let p = parse(
"Bioinformatics 25:2078-2079. doi:10.1093/bioinformatics/btp352. Author list is the \
core set plus the 1000 Genomes Data Processing Subgroup; verify.",
Some(2009),
);
assert_eq!(p.doi.as_deref(), Some("10.1093/bioinformatics/btp352"));
assert_eq!(
p.note.as_deref(),
Some(
"Author list is the core set plus the 1000 Genomes Data Processing Subgroup; \
verify."
)
);
}
#[test]
fn a_conference_has_a_year_where_a_volume_would_be() {
let p = parse(
"Proc. FOCS 2000. doi:10.1109/SFCS.2000.892127. The FM-index. Theory background.",
Some(2000),
);
assert_eq!(p.container.as_deref(), Some("Proc. FOCS"));
// No volume and no pages were written, so none are invented — and the
// 2000 in the DOI is not mistaken for either.
assert_eq!(p.volume, None);
assert_eq!(p.pages, None);
assert_eq!(p.doi.as_deref(), Some("10.1109/SFCS.2000.892127"));
assert_eq!(p.note.as_deref(), Some("The FM-index. Theory background."));
}
#[test]
fn a_note_with_nothing_to_find_is_left_whole() {
let p = parse("Origin of the MAPQ score.", Some(2008));
assert_eq!(p.container, None);
assert_eq!(p.note.as_deref(), Some("Origin of the MAPQ score."));
}
#[test]
fn a_multi_word_journal_is_not_cut_at_a_number() {
let p = parse("Advances in Mathematics 20, 367-387.", Some(1976));
assert_eq!(p.container.as_deref(), Some("Advances in Mathematics"));
assert_eq!(p.volume.as_deref(), Some("20"));
}
#[test]
fn what_validation_looks_for() {
assert!(looks_like_a_citation("Nat Methods 12:59-60."));
assert!(looks_like_a_citation("doi:10.1038/nmeth.3176"));
assert!(!looks_like_a_citation("Origin of minimizers."));
assert!(!looks_like_a_citation("Verify before use."));
// A page range with no volume is not a citation shape.
assert!(!looks_like_a_citation("see pages 12-14"));
}
}
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! A minimal calendar date, serialized as `YYYY-MM-DD`.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Content fingerprints and student pseudonyms.
+3 -3
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Converting the authoring markup into HTML, plain text, and Typst.
@@ -113,7 +113,7 @@ pub fn to_plain(src: &str) -> String {
out.trim().to_string()
}
/// Converts authoring markup to Pandoc-flavoured Markdown, for a Quarto document.
/// Converts authoring markup to Pandoc-flavored Markdown, for a Quarto document.
///
/// Subscripts and superscripts become Pandoc's `~x~` and `^x^`, the symbol table
/// renders as Unicode, and the bold, italic, and inline-code spans are already
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! A small deterministic random number generator.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Reading and writing the YAML and JSON files the tool owns.
+2 -2
View File
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: Prosperity-3.0.0
// Copyright Scientific Computing Studio
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! A minimal ZIP writer, stored (uncompressed) entries only.
+54
View File
@@ -0,0 +1,54 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! Fixtures shared by the 2.1 integration tests: the examples in
//! `docs/design/2.1/examples/`, copied into a scratch directory so a test may
//! change them.
#![allow(dead_code)]
use std::path::{Path, PathBuf};
/// `docs/design/2.1/`.
pub fn design() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("docs/design/2.1")
}
/// A fresh, empty directory under the system temp dir.
pub fn scratch(tag: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("coursebank-2-1-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
dir
}
/// Copies a directory tree.
pub fn copy_tree(from: &Path, to: &Path) {
std::fs::create_dir_all(to).unwrap();
for entry in std::fs::read_dir(from).unwrap() {
let path = entry.unwrap().path();
let target = to.join(path.file_name().unwrap());
if path.is_dir() {
copy_tree(&path, &target);
} else {
std::fs::copy(&path, &target).unwrap();
}
}
}
/// The 2.1 example course, copied so a test may change it.
pub fn v21_course(tag: &str) -> PathBuf {
let root = scratch(tag);
copy_tree(&design().join("examples/v21"), &root);
let _ = std::fs::remove_file(root.join("lecture-page-preview.md"));
root
}
/// Replaces the first occurrence of `from` with `to` in a file of a course.
pub fn edit(root: &Path, relative: &str, from: &str, to: &str) {
let path = root.join(relative);
let text = std::fs::read_to_string(&path).unwrap();
assert!(text.contains(from), "{relative} does not contain {from:?}");
std::fs::write(&path, text.replacen(from, to, 1)).unwrap();
}
+7
View File
@@ -38,6 +38,10 @@ fn root() -> PathBuf {
/// Every `.md` file under `docs/`, sorted so failures are reported in a stable
/// order.
///
/// `docs/design/` is skipped. It holds design records (a specification, its
/// examples, an implementation plan), which are read in the repository rather
/// than compiled into the guide, and whose example output is quoted as written.
fn markdown_files(dir: &Path, out: &mut Vec<PathBuf>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
@@ -45,6 +49,9 @@ fn markdown_files(dir: &Path, out: &mut Vec<PathBuf>) {
let mut paths: Vec<PathBuf> = entries.filter_map(|e| e.ok()).map(|e| e.path()).collect();
paths.sort();
for path in paths {
if path.ends_with("docs/design") {
continue;
}
if path.is_dir() {
markdown_files(&path, out);
} else if path.extension().is_some_and(|e| e == "md") {
+338
View File
@@ -0,0 +1,338 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The lint rules 2.1 adds, one test each.
//!
//! Each test breaks the example course the way the prototype's self-test does
//! (`docs/design/2.1/prototype/check21.py --selftest`) and checks that the rule
//! fires. `docs/design/2.1/CODES.md` maps the specification's codes to these
//! rule codes.
mod common;
use std::path::PathBuf;
use std::process::Command;
use common::v21_course;
use coursebank::catalog::{Catalog, Severity};
use coursebank::course::fragment::Section;
use coursebank::course::glossary::{Term, TermKind};
use coursebank::lint::{self, Finding, Rule, Thresholds};
/// The example course, loaded.
fn catalog(tag: &str) -> Catalog {
Catalog::load(&v21_course(tag)).expect("the example loads")
}
/// The glossary findings for a rule.
fn fired(catalog: &Catalog, rule: Rule) -> Vec<Finding> {
lint::lint_glossary(catalog)
.into_iter()
.filter(|f| f.rule == rule)
.collect()
}
/// Asserts a rule fires once, on `subject`, with `needle` in its message.
fn fires(catalog: &Catalog, rule: Rule, subject: &str, needle: &str) {
let found = fired(catalog, rule);
assert!(
found
.iter()
.any(|f| f.subject == subject && f.message.contains(needle)),
"{} did not fire on {subject} with {needle:?}: {found:#?}",
rule.code()
);
}
/// The example item, for the rules that read items.
fn item(catalog: &mut Catalog) -> &mut coursebank::item::Item {
&mut catalog.entries[0].item
}
#[test]
fn the_example_draws_exactly_the_two_expected_glossary_findings() {
let findings = lint::lint_glossary(&catalog("baseline"));
let summary: Vec<(String, &str)> = findings
.iter()
.map(|f| (f.subject.clone(), f.rule.code()))
.collect();
assert_eq!(
summary,
vec![
("val-per-decade".to_string(), "glossary-unused"),
("glossary".to_string(), "glossary-proposed"),
],
"{findings:#?}"
);
let proposed = &findings[1];
assert_eq!(proposed.severity, Severity::Note);
assert!(proposed.message.starts_with("2 proposed entries"));
}
#[test]
fn w2101_an_entry_nothing_needs() {
let mut c = catalog("w2101");
c.course.terms.insert(
"term-unused".into(),
Term {
term: "unused".into(),
definition: "Not needed.".into(),
kind: TermKind::Standard,
textbook: Vec::new(),
variants: Vec::new(),
symbol: None,
units: None,
exam_wording: None,
status: None,
tags: Vec::new(),
notes_private: None,
},
);
fires(
&c,
Rule::GlossaryUnused,
"term-unused",
"no target lists this term",
);
}
#[test]
fn w2102_a_renamed_term_with_no_textbook_name() {
let mut c = catalog("w2102");
c.course.terms.get_mut("term-gap").unwrap().kind = TermKind::Renamed;
fires(&c, Rule::NoTextbookName, "term-gap", "a renamed term");
}
#[test]
fn w2103_a_coined_term_with_no_exam_wording() {
let mut c = catalog("w2103");
c.course.terms.get_mut("term-decade").unwrap().exam_wording = None;
fires(&c, Rule::NoExamWording, "term-decade", "exam_wording");
}
#[test]
fn w2104_a_definition_that_uses_its_term() {
let mut c = catalog("w2104");
c.course.terms.get_mut("term-gap").unwrap().definition = "A gap is a dash.".into();
fires(&c, Rule::CircularDefinition, "term-gap", "uses 'gap'");
}
#[test]
fn w2105_a_coined_term_in_an_item() {
let mut c = catalog("w2105");
item(&mut c).stem.push_str(" Think in decades.");
fires(
&c,
Rule::CoinedTermInItem,
"q-align-which-scores-higher",
"stem uses the coined term 'decades' (term-decade); write 'ten-fold'",
);
}
#[test]
fn w2106_a_value_with_no_source() {
let mut c = catalog("w2106");
c.course
.values
.get_mut("val-alignment-count-100")
.unwrap()
.source = None;
fires(
&c,
Rule::UnsourcedValue,
"val-alignment-count-100",
"no `source`",
);
}
#[test]
fn w2107_a_value_variant_in_student_prose_and_in_an_item() {
let mut c = catalog("w2107");
c.course
.learning_targets
.get_mut("t-align-fill-matrix")
.unwrap()
.explanation = Some("It costs 5.7 kJ.".into());
item(&mut c).options[0].text = "About 5.8 kJ per decade.".into();
fires(
&c,
Rule::ValueConflict,
"t-align-fill-matrix",
"explanation writes '5.7 kJ', a variant of val-per-decade",
);
fires(
&c,
Rule::ValueConflict,
"q-align-which-scores-higher",
"state it as 'about 6 kJ/mol'",
);
}
#[test]
fn w2108_target_text_the_policy_forbids() {
let mut c = catalog("w2108");
c.course
.learning_targets
.get_mut("t-align-fill-matrix")
.unwrap()
.text = "Calculate the score.".into();
fires(
&c,
Rule::ForbiddenTargetWording,
"t-align-fill-matrix",
"the text says 'Calculat'",
);
}
#[test]
fn w2109_an_explanation_using_a_term_its_target_does_not_list() {
let mut c = catalog("w2109");
c.course
.learning_targets
.get_mut("t-align-gap-penalty-effect")
.unwrap()
.explanation = Some("Unlike a {term-local-alignment}, the path changes.".into());
fires(
&c,
Rule::UnlistedTerm,
"t-align-gap-penalty-effect",
"{term-local-alignment}",
);
}
#[test]
fn w2109_a_term_inherited_from_a_prerequisite_target_is_listed() {
// t-align-gap-penalty-effect has t-align-fill-matrix as a prerequisite,
// which lists the traceback.
let mut c = catalog("w2109-inherited");
c.course
.learning_targets
.get_mut("t-align-gap-penalty-effect")
.unwrap()
.explanation = Some("Follow the {term-traceback}.".into());
assert!(fired(&c, Rule::UnlistedTerm).is_empty());
}
#[test]
fn w2110_an_item_prints_a_recall_value() {
let mut c = catalog("w2110");
item(&mut c).stem = "Given about 6 kJ/mol per ten-fold ratio, which is larger?".into();
fires(
&c,
Rule::RecallValuePrinted,
"q-align-which-scores-higher",
"stem prints 'about 6 kJ/mol' (val-per-decade)",
);
}
#[test]
fn w2111_a_deep_target_on_an_equation_with_no_implications() {
let mut c = catalog("w2111");
c.course
.equations
.get_mut("eq-global-recurrence")
.unwrap()
.implications
.clear();
fires(
&c,
Rule::EquationWithoutImplications,
"t-align-fill-matrix",
"turns on eq-global-recurrence",
);
}
#[test]
fn i2112_an_entry_defined_away_from_the_objective_that_first_needs_it() {
let mut c = catalog("i2112");
c.course.origins.insert(
(Section::Terms, "term-gap".into()),
PathBuf::from("objectives/lo-alignment-recurrence.yaml"),
);
let found = fired(&c, Rule::DefinedElsewhere);
assert_eq!(found.len(), 1, "{found:#?}");
assert_eq!(found[0].subject, "term-gap");
assert_eq!(found[0].severity, Severity::Note);
assert!(
found[0]
.message
.contains("first needed by t-align-matrix-terms in l09"),
"{found:#?}"
);
}
#[test]
fn i2113_counts_proposed_entries() {
let mut c = catalog("i2113");
c.course.terms.get_mut("term-decade").unwrap().status = None;
fires(
&c,
Rule::ProposedEntries,
"glossary",
"1 proposed entry waiting for the instructor: val-per-decade",
);
}
#[test]
fn an_unresolved_target_reference_in_an_old_reading_field_is_a_warning() {
let mut c = catalog("legacy-ref");
c.course.lectures.get_mut("l09").unwrap().readings[0].focus = Some("See {t-typo}.".into());
fires(
&c,
Rule::UnresolvedReference,
"l09",
"reading 1 focus refers to {t-typo}",
);
}
#[test]
fn a_registry_misconception_designs_a_distractor_and_gives_it_feedback() {
// Approved items are held to the completeness rules. The option that names
// a registry misconception and nothing else passes both of them.
let mut c = catalog("misconception-id");
item(&mut c).status = coursebank::taxonomy::Status::InReview;
let findings = lint::lint_catalog(&c, &Thresholds::default());
let about_option = |rule: Rule| {
findings
.iter()
.any(|f| f.rule == rule && f.message.contains("o-matches-only"))
};
assert!(!about_option(Rule::UndesignedDistractor), "{findings:#?}");
assert!(!about_option(Rule::NoStudentFeedback), "{findings:#?}");
}
#[test]
fn every_glossary_rule_has_a_glossary_code() {
let glossary: Vec<Rule> = Rule::ALL
.into_iter()
.filter(|r| r.family() == "glossary")
.collect();
assert_eq!(glossary.len(), 14);
assert!(glossary.iter().all(|r| r.code().starts_with("glossary-")));
}
/// Runs the binary on a course.
fn coursebank(root: &std::path::Path, args: &[&str]) -> std::process::Output {
Command::new(env!("CARGO_BIN_EXE_coursebank"))
.arg("-C")
.arg(root)
.args(args)
.output()
.expect("the binary runs")
}
#[test]
fn notes_are_printed_but_do_not_fail_lint() {
let root = v21_course("cli-notes");
let only_notes = coursebank(&root, &["lint", "--only", "glossary-proposed"]);
let stdout = String::from_utf8_lossy(&only_notes.stdout);
assert!(stdout.contains("[note] glossary-proposed"), "{stdout}");
assert!(only_notes.status.success(), "{stdout}");
let with_a_warning = coursebank(
&root,
&["lint", "--only", "glossary-proposed,glossary-unused"],
);
assert!(!with_a_warning.status.success());
}
+158
View File
@@ -0,0 +1,158 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! `coursebank migrate glossary`, against `docs/design/2.1/examples/migration/`.
//!
//! The example course's own `glossary.yaml` is that migration's output after a
//! person linked the value to its term, so each test starts from the example
//! with it removed.
mod common;
use std::path::{Path, PathBuf};
use std::process::Command;
use common::{design, edit, v21_course};
use coursebank::course::fragment::Fragment;
use coursebank::migrate;
/// The example course without its glossary.
fn course(tag: &str) -> PathBuf {
let root = v21_course(tag);
std::fs::remove_file(root.join("glossary.yaml")).unwrap();
root
}
fn source(name: &str) -> PathBuf {
design().join("examples/migration").join(name)
}
fn fragment(path: &Path) -> Fragment {
serde_yaml_ng::from_str(&std::fs::read_to_string(path).unwrap()).unwrap()
}
fn run(root: &Path, write: bool) -> migrate::GlossaryMigration {
migrate::glossary(
root,
Some(&source("terms.yaml")),
Some(&source("values.yaml")),
write,
)
.unwrap()
}
#[test]
fn migration_reproduces_the_expected_glossary() {
let root = course("mig-expected");
let report = run(&root, true);
let written = fragment(&root.join("glossary.yaml"));
let expected = fragment(&source("glossary.expected.yaml"));
assert_eq!(written.terms, expected.terms);
assert_eq!(written.values, expected.values);
// The report the expected file's header comment describes.
assert_eq!(
report.renamed,
[
("decade".to_string(), "term-decade".to_string()),
("per-decade".to_string(), "val-per-decade".to_string()),
]
);
assert_eq!(report.derived.len(), 2, "{:?}", report.derived);
assert!(report.derived[0].contains("`first` was L2.1"));
assert!(report.derived[1].contains("`lectures` was [L2.1, L2.4, L2.5]"));
assert_eq!(report.unlisted, ["term-decade", "val-per-decade"]);
assert!(report.dropped.is_empty() && report.relinked.is_empty());
// The course still validates with the written file.
let issues = coursebank::Catalog::load(&root)
.unwrap()
.validate()
.unwrap();
assert!(issues.is_empty(), "{issues:#?}");
}
#[test]
fn migrating_twice_is_a_no_op() {
let root = course("mig-twice");
run(&root, true);
let before = std::fs::read(root.join("glossary.yaml")).unwrap();
let again = run(&root, true);
assert!(again.text.is_none());
assert!(again.added.is_empty() && again.renamed.is_empty());
assert_eq!(again.present.len(), 2);
assert_eq!(std::fs::read(root.join("glossary.yaml")).unwrap(), before);
// The example as shipped already holds both, linked: also a no-op.
let shipped = v21_course("mig-shipped");
assert!(run(&shipped, true).text.is_none());
}
#[test]
fn a_dropped_first_is_compared_with_the_targets() {
let root = course("mig-derived");
edit(
&root,
"objectives/lo-alignment-scoring.yaml",
"terms: [term-alignment-matrix, term-gap,",
"terms: [term-decade, term-alignment-matrix, term-gap,",
);
let report = run(&root, false);
assert!(
report.derived[0].contains("`first` was L2.1; derived from the targets: l09"),
"{:?}",
report.derived
);
assert_eq!(report.unlisted, ["val-per-decade"]);
assert!(!root.join("glossary.yaml").exists(), "a dry run wrote");
}
#[test]
fn an_existing_glossary_is_added_to_not_rewritten() {
let root = course("mig-existing");
let before = "# Cross-cutting terms.\nterms:\n term-other:\n term: other\n \
definition: Something else.\n";
std::fs::write(root.join("glossary.yaml"), before).unwrap();
let report = run(&root, true);
let after = std::fs::read_to_string(root.join("glossary.yaml")).unwrap();
assert!(after.starts_with(before), "{after}");
assert!(after.contains("\nvalues:\n val-per-decade:\n"));
assert_eq!(report.added, ["term-decade", "val-per-decade"]);
let written = fragment(&root.join("glossary.yaml"));
assert_eq!(written.terms.len(), 2);
assert_eq!(written.values.len(), 1);
}
#[test]
fn the_cli_reports_and_a_dry_run_writes_nothing() {
let root = course("mig-cli");
let terms = source("terms.yaml");
let values = source("values.yaml");
let out = Command::new(env!("CARGO_BIN_EXE_coursebank"))
.arg("-C")
.arg(&root)
.args(["migrate", "glossary", "--dry-run", "--terms"])
.arg(&terms)
.arg("--values")
.arg(&values)
.output()
.unwrap();
let stdout = String::from_utf8_lossy(&out.stdout);
assert!(out.status.success(), "{stdout}");
assert!(stdout.contains("decade -> term-decade"));
assert!(stdout.contains("nothing written"));
assert!(!root.join("glossary.yaml").exists());
let none = Command::new(env!("CARGO_BIN_EXE_coursebank"))
.arg("-C")
.arg(&root)
.args(["migrate", "glossary"])
.output()
.unwrap();
assert!(!none.status.success());
}
+516
View File
@@ -0,0 +1,516 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! What 2.1 renders: the lecture page, the glossary page, and the student
//! reports, and what it must never render for a student.
mod common;
use std::path::Path;
use common::{design, v21_course};
use coursebank::assessment::{AssessmentFile, Form};
use coursebank::catalog::Catalog;
use coursebank::lecture::{PageOptions, Style, glossary_markdown, page_markdown};
use coursebank::responses::{Response, ResponseSet};
use coursebank::typst::config::{RenderConfig, Variant};
/// The example lecture page in the Plain style, which is the style the
/// preview was written in.
fn plain_page(catalog: &Catalog) -> String {
let options = PageOptions {
style: Style::Plain,
..PageOptions::default()
};
page_markdown(&catalog.course, "l09", &options).expect("renders")
}
/// A page line with its link targets removed: `[gap](glossary.html#x)`
/// becomes `[gap]`, which is how the prototype's preview writes a link.
fn unlinked(line: &str) -> String {
let mut out = String::new();
let mut rest = line;
while let Some(at) = rest.find("](") {
out.push_str(&rest[..=at]);
let after = &rest[at + 2..];
match after.find(')') {
Some(close) => rest = &after[close + 1..],
None => {
rest = after;
break;
}
}
}
out.push_str(rest);
out
}
#[test]
fn the_lecture_page_has_the_content_and_order_of_the_preview() {
let catalog = Catalog::load(&v21_course("page-snapshot")).unwrap();
let rendered: Vec<String> = plain_page(&catalog)
.lines()
.map(unlinked)
.filter(|l| !l.trim().is_empty())
.collect();
let preview_text =
std::fs::read_to_string(design().join("examples/v21/lecture-page-preview.md")).unwrap();
// The prototype prints a target reference as the raw id; coursebank prints
// its number on the page, as the 2.0 readings page always has.
let preview_text = preview_text.replace("(T t-align-fill-matrix)", "(T 3)");
let preview: Vec<&str> = preview_text
.lines()
.filter(|l| !l.trim().is_empty() && !l.starts_with("<!--"))
.collect();
// Every preview line appears, in the preview's order.
let mut at = 0;
for line in &preview {
match rendered[at..].iter().position(|r| r == line) {
Some(offset) => at += offset + 1,
None => panic!(
"missing or out of order: {line:?}\n\nrendered:\n{}",
rendered.join("\n")
),
}
}
// And the page adds only what the specification asks for beyond the
// prototype: the workload line, and the reading's summary and skip notes.
let extra: Vec<&String> = rendered
.iter()
.filter(|r| !preview.contains(&r.as_str()))
.collect();
assert_eq!(
extra,
vec![
"About 40 minutes of assigned reading.",
" Introduces the alignment matrix and [the global alignment recurrence], then \
contrasts the global and local fills.",
" Gloss the affine-gap extension. It comes in a later lecture.",
],
);
}
#[test]
fn the_quarto_page_anchors_new_terms_and_links_to_them() {
let catalog = Catalog::load(&v21_course("page-quarto")).unwrap();
let page = page_markdown(&catalog.course, "l09", &PageOptions::default()).unwrap();
assert!(page.contains("[**gap penalty**]{#glossary-term-gap-penalty} $d$:"));
assert!(page.contains("[gap penalty](#glossary-term-gap-penalty)"));
assert!(page.contains("### the global alignment recurrence {#glossary-eq-global-recurrence}"));
assert!(page.contains("(@) **Define the alignment matrix"));
assert!(page.contains("- $F(i,j)$: the best score"));
assert!(page.contains("- [Lecture brief](https://example.edu/briefs/l09)"));
}
#[test]
fn a_later_lecture_lists_earlier_terms_with_the_lecture_that_introduced_them() {
let root = v21_course("page-later");
std::fs::write(
root.join("lectures/l10.yaml"),
"lectures:\n l10:\n title: Database search\n teaches: [lo-search]\n",
)
.unwrap();
std::fs::write(
root.join("objectives/lo-search.yaml"),
"learning_objectives:\n lo-search: { text: Search a database., level_ceiling: 2 }\n\
learning_targets:\n t-search-seeds:\n text: Explain why a search starts from \
short exact matches.\n objective: lo-search\n level_ceiling: 2\n terms: \
[term-gap-penalty, term-seed]\nterms:\n term-seed: { term: seed, definition: A short \
exact match a search extends. }\n",
)
.unwrap();
let catalog = Catalog::load(&root).unwrap();
let course = &catalog.course;
assert_eq!(course.first_lecture("term-gap-penalty"), Some("l09"));
assert_eq!(course.first_lecture("term-seed"), Some("l10"));
assert_eq!(course.first_lecture("val-per-decade"), None);
let page = page_markdown(course, "l10", &PageOptions::default()).unwrap();
assert!(page.contains("[**seed**]{#glossary-term-seed}"), "{page}");
assert!(
page.contains(
"Terms from earlier lectures: [gap penalty](glossary.html#glossary-term-gap-penalty) \
(l09)"
),
"{page}"
);
}
#[test]
fn the_glossary_page_lists_terms_alphabetically_with_their_first_lecture() {
let catalog = Catalog::load(&v21_course("glossary-page")).unwrap();
let page = glossary_markdown(&catalog.course, &PageOptions::default());
let terms: Vec<&str> = page
.lines()
.filter_map(|l| l.strip_prefix("- [**"))
.map(|l| l.split("**]").next().unwrap())
.collect();
assert_eq!(
terms,
vec![
"alignment matrix",
"decade",
"gap",
"gap penalty",
"global alignment",
"local alignment",
"substitution score",
"traceback",
]
);
assert!(page.contains("*First in l09: Pairwise alignment.*"));
// A link between entries stays on this page.
assert!(page.contains("aligned against a [gap](#glossary-term-gap)"));
assert!(page.contains("## Equations"));
assert!(
page.contains("- free energy per ten-fold concentration ratio at 37 °C: about 6 kJ/mol")
);
}
/// A second item that encodes the same registry misconception, so a student
/// who misses both has one mistake behind two answers.
const SECOND_ITEM: &str = "
- id: q-align-equal-matches
status: draft
level: 2
stem: 'Two alignments of one pair have equal matches. Which scores higher?'
learning_targets: [t-align-score-columns]
options:
- { id: o-fewer-gaps, text: 'The one with fewer gaps.', correct: true }
- { id: o-tie, text: 'Neither; equal matches mean equal scores.', misconception_id: mis-only-matches-count }
";
/// The record both items were given on.
const RECORD: &str = "
assessment: { id: alignment-quiz, title: Alignment quiz }
items:
- { number: 1, item: q-align-which-scores-higher, learning_targets: [t-align-score-columns] }
- { number: 2, item: q-align-equal-matches, learning_targets: [t-align-score-columns] }
";
/// One response row.
fn response(student: &str, number: u32, item: &str, chosen: &str, credit: f64) -> Response {
Response {
administration_id: "BIOSC 1540/2027s/alignment-quiz".into(),
course: "BIOSC 1540".into(),
term: "2027s".into(),
assessment_id: "alignment-quiz".into(),
date: None,
form: None,
form_position: None,
student_key: student.into(),
sid: None,
name: None,
email: None,
section: None,
item_number: number,
item_ref: Some(item.into()),
item_version: None,
variant: None,
selected: vec![chosen.into()],
selected_source: vec![chosen.into()],
eliminated: vec![],
eliminated_source: vec![],
correct: Some(credit >= 0.999),
credit,
points_possible: 1.0,
score: credit,
response_time_seconds: None,
level: None,
learning_targets: vec!["t-align-score-columns".into()],
topics: vec![],
bonus: false,
dropped: false,
dropped_full_credit: false,
}
}
/// The example with a second item, and the record and responses for a student
/// who missed both by the same mistake.
fn administered(tag: &str) -> (Catalog, AssessmentFile, ResponseSet) {
let root = v21_course(tag);
let bank = root.join("banks/alignment.yaml");
let mut text = std::fs::read_to_string(&bank).unwrap();
text.push_str(SECOND_ITEM);
std::fs::write(&bank, text).unwrap();
let catalog = Catalog::load(&root).unwrap();
let record: AssessmentFile = serde_yaml_ng::from_str(RECORD).unwrap();
let mut set = ResponseSet::new();
set.rows = vec![
response(
"s1",
1,
"q-align-which-scores-higher",
"o-matches-only",
0.0,
),
response("s1", 2, "q-align-equal-matches", "o-tie", 0.0),
response(
"s2",
1,
"q-align-which-scores-higher",
"o-columns-all-count",
1.0,
),
response("s2", 2, "q-align-equal-matches", "o-fewer-gaps", 1.0),
];
(catalog, record, set)
}
/// The Markdown report for the student who missed both.
fn markdown_report(catalog: &Catalog, record: &AssessmentFile, set: &ResponseSet) -> String {
let cohort = coursebank::students::summarize(set, &catalog.course, Some(catalog), None);
let summary = cohort
.students
.iter()
.find(|s| s.student_key == "s1")
.unwrap();
coursebank::report::student(
summary,
&cohort,
&catalog.course,
record,
&coursebank::report::StudentOptions::default(),
)
}
/// The PDF report's payload for the student who missed both, as Typst source.
fn typst_report(catalog: &Catalog, set: &ResponseSet) -> String {
let cohort = coursebank::students::summarize(set, &catalog.course, Some(catalog), None);
let summary = cohort
.students
.iter()
.find(|s| s.student_key == "s1")
.unwrap();
let diagnostic = coursebank::diagnostic::student(
summary,
&cohort,
catalog,
set,
None,
&coursebank::diagnostic::Options::default(),
);
let config = RenderConfig::for_variant(Variant::StudentReport);
coursebank::typst::diagnostic::student_value(&diagnostic, &config).to_typst(0)
}
#[test]
fn a_student_report_explains_the_target_and_groups_misses_by_misconception() {
let (catalog, record, set) = administered("report-markdown");
let report = markdown_report(&catalog, &record, &set);
assert!(
report.contains(
"What meeting it looks like: An alignment's score is the sum over its columns."
),
"{report}"
);
// The registry's belief and correction, for an option that names the
// misconception and says nothing itself.
assert!(
report.contains(
"This answer reflects a common mistake: *An alignment score counts the matching \
positions"
),
"{report}"
);
assert!(
report.contains("Every column counts: matches add"),
"{report}"
);
// Both misses share the misconception, so they are grouped under it.
assert!(
report.contains("## The same mistake, more than once\n\n**Questions 1, 2**:"),
"{report}"
);
}
#[test]
fn the_pdf_report_carries_the_same_explanation_mistake_and_grouping() {
let (catalog, _, set) = administered("report-typst");
let payload = typst_report(&catalog, &set);
assert!(payload.contains("explanation:"), "{payload}");
assert!(payload.contains("mistake: ("), "{payload}");
assert!(payload.contains("mis-only-matches-count"), "{payload}");
let compact: String = payload.split_whitespace().collect();
assert!(
compact.contains("mistakes:((id:\"mis-only-matches-count\""),
"{payload}"
);
assert!(compact.contains("questions:(1,2,)"), "{payload}");
}
/// Instructor-facing text that must never reach a student: `authoring`,
/// `notes_private`, `evidence`, `conflicts`, and `status`.
fn private_text(catalog: &Catalog) -> Vec<String> {
let course = &catalog.course;
let mut out = Vec::new();
for target in course.learning_targets.values() {
if let Some(a) = &target.authoring {
out.extend(a.ceiling_item.clone());
out.extend(a.number_work.clone());
out.extend(a.supplied.iter().cloned());
out.extend(a.constraints.iter().cloned());
out.extend(a.examples.iter().cloned());
}
}
for m in course.misconceptions.values() {
out.extend(m.evidence.iter().filter_map(|e| e.detail.clone()));
}
for v in course.values.values() {
out.extend(v.conflicts.iter().cloned());
}
out.extend(
course
.terms
.values()
.filter_map(|t| t.notes_private.clone()),
);
out.extend(
course
.equations
.values()
.filter_map(|e| e.notes_private.clone()),
);
out.extend(
course
.values
.values()
.filter_map(|v| v.notes_private.clone()),
);
out.extend(
course
.misconceptions
.values()
.filter_map(|m| m.notes_private.clone()),
);
// A short note can repeat words a student-facing field also uses (`supplied:
// the scoring scheme` beside a target that says "the scoring scheme"), and
// finding those on a page proves nothing. Only text that appears nowhere a
// student is meant to read is a test of the rule.
let public: Vec<String> = course
.student_prose()
.iter()
.map(|p| p.text.to_string())
.chain(catalog.entries.iter().flat_map(|e| {
std::iter::once(e.item.stem.clone())
.chain(e.item.options.iter().map(|o| o.text.clone()))
}))
.collect();
out.retain(|secret| !public.iter().any(|p| p.contains(secret.as_str())));
out
}
#[test]
fn instructor_fields_never_reach_a_student_page_report_or_export() {
let (mut catalog, record, set) = administered("rule-5");
// The example has no private notes, so give every registry entry one.
let course = &mut catalog.course;
for (id, t) in course.terms.iter_mut() {
t.notes_private = Some(format!("PRIVATE NOTE on {id}"));
}
for (id, e) in course.equations.iter_mut() {
e.notes_private = Some(format!("PRIVATE NOTE on {id}"));
}
for (id, v) in course.values.iter_mut() {
v.notes_private = Some(format!("PRIVATE NOTE on {id}"));
}
for (id, m) in course.misconceptions.iter_mut() {
m.notes_private = Some(format!("PRIVATE NOTE on {id}"));
}
// Every course in the wild that has a proposed entry would print the word
// somewhere if status leaked; the example has two.
let secrets = private_text(&catalog);
assert!(secrets.len() > 10, "{secrets:#?}");
let form = Form {
id: "A".into(),
seed: 0,
shuffle_items: false,
shuffle_options: false,
};
let mut outputs: Vec<(String, String)> = Vec::new();
for style in [Style::Quarto, Style::Plain] {
let options = PageOptions {
style,
..PageOptions::default()
};
outputs.push((
format!("lecture page ({style:?})"),
page_markdown(&catalog.course, "l09", &options).unwrap(),
));
outputs.push((
format!("glossary page ({style:?})"),
glossary_markdown(&catalog.course, &options),
));
}
outputs.push((
"readings page".into(),
coursebank::lecture::readings_markdown(&catalog.course, "l09", Style::Quarto).unwrap(),
));
outputs.push((
"student report".into(),
markdown_report(&catalog, &record, &set),
));
outputs.push(("student PDF report".into(), typst_report(&catalog, &set)));
let qti =
coursebank::qti::build(&catalog, &record, &coursebank::qti::QtiOptions::default()).unwrap();
outputs.push(("Canvas quiz".into(), qti.quiz_xml));
for variant in [
coursebank::practice::Variant::Worksheet,
coursebank::practice::Variant::Solutions,
] {
let options = coursebank::practice::Options {
form: form.clone(),
variant,
answer_space: false,
};
outputs.push((
format!("practice {variant:?}"),
coursebank::practice::render(&catalog, &record, &options).unwrap(),
));
}
let site =
coursebank::site::render(&catalog, &record, coursebank::site::Options::new(form)).unwrap();
outputs.push(("site page".into(), site.page));
outputs.push(("site questions".into(), site.questions_qmd));
for (name, text) in &outputs {
for secret in &secrets {
assert!(
!text.contains(secret.as_str()),
"{name} prints instructor-facing text {secret:?}"
);
}
assert!(
!text.contains("proposed"),
"{name} prints an entry's status"
);
}
}
#[test]
fn the_cli_writes_the_page_and_the_glossary() {
let root = v21_course("cli-page");
let run = |args: &[&str]| {
std::process::Command::new(env!("CARGO_BIN_EXE_coursebank"))
.arg("-C")
.arg(&root)
.args(args)
.output()
.unwrap()
};
let page = run(&["lecture", "page", "l09", "--style", "plain"]);
assert!(page.status.success());
assert!(String::from_utf8_lossy(&page.stdout).starts_with("# l09: Pairwise alignment"));
let out = root.join("glossary.qmd");
let glossary = run(&["lecture", "glossary", "--out", out.to_str().unwrap()]);
assert!(glossary.status.success());
assert!(Path::new(&out).is_file());
}
+243
View File
@@ -0,0 +1,243 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The 2.1 schema against its specification in `docs/design/2.1/`.
//!
//! The examples there are the fixtures, read in place, so the specification and
//! these tests cannot drift apart: `examples/v20/` must load unchanged,
//! `examples/v21/` uses every new field, and `schemas/` is the JSON Schema each
//! generated file must equal.
mod common;
use std::path::PathBuf;
use common::{design, scratch, v21_course};
use coursebank::catalog::Catalog;
use coursebank::course::CourseFile;
use coursebank::jsonschema::{self, Kind};
use serde_json::Value;
/// The JSON pointers at which two values differ.
fn differences(path: &str, generated: &Value, target: &Value, out: &mut Vec<String>) {
match (generated, target) {
(Value::Object(a), Value::Object(b)) => {
for key in a.keys().chain(b.keys().filter(|k| !a.contains_key(*k))) {
let here = format!("{path}/{}", key.replace('~', "~0").replace('/', "~1"));
match (a.get(key), b.get(key)) {
(Some(x), Some(y)) => differences(&here, x, y, out),
(Some(_), None) => out.push(format!("{here} (only generated)")),
(None, Some(_)) => out.push(format!("{here} (only in the target)")),
(None, None) => {}
}
}
}
(Value::Array(a), Value::Array(b)) if a.len() == b.len() => {
for (i, (x, y)) in a.iter().zip(b).enumerate() {
differences(&format!("{path}/{i}"), x, y, out);
}
}
_ if generated == target => {}
_ => out.push(path.to_string()),
}
}
/// Differences that are known and kept, with the reason.
///
/// Both are spellings in 2.0 descriptions that the code already had before 2.1:
/// `schemas-original/` was generated by a build that spelled them the American
/// way, and the 2.1 targets were built from it.
const KNOWN: &[(&str, &str)] = &[
(
"calibration.schema.json",
"/properties/items/additionalProperties/properties/variants/items/properties/option_stats\
/description",
),
(
"course.schema.json",
"/properties/policy/properties/grade_scale/items/properties/group/description",
),
];
#[test]
fn every_generated_schema_matches_its_target() {
let mut unexpected = Vec::new();
for kind in Kind::ALL {
let name = kind.filename();
let text = std::fs::read_to_string(design().join("schemas").join(name))
.unwrap_or_else(|e| panic!("{name}: {e}"));
let target: Value = serde_json::from_str(&text).unwrap();
let mut found = Vec::new();
differences("", &jsonschema::schema(kind), &target, &mut found);
for pointer in found {
if !KNOWN.iter().any(|(file, p)| *file == name && *p == pointer) {
unexpected.push(format!("{name}: {pointer}"));
}
}
}
assert!(
unexpected.is_empty(),
"generated schemas differ from docs/design/2.1/schemas:\n {}",
unexpected.join("\n ")
);
}
#[test]
fn the_known_differences_are_still_there() {
// If one of these is fixed, the allow-list should shrink with it.
for (file, pointer) in KNOWN {
let kind = Kind::ALL
.into_iter()
.find(|k| k.filename() == *file)
.unwrap();
let generated = jsonschema::schema(kind);
let text = std::fs::read_to_string(design().join("schemas").join(file)).unwrap();
let target: Value = serde_json::from_str(&text).unwrap();
assert_ne!(
generated.pointer(pointer),
target.pointer(pointer),
"{file}{pointer}"
);
}
}
/// The 2.0 fragments, completed into a course with the ids they cite.
fn v20_course(tag: &str) -> PathBuf {
let root = scratch(tag);
std::fs::write(
root.join("course.yaml"),
"schema_version: '2.0'\ncourse: { code: BIOSC 1540, title: Computational Biology, term: \
2026f }\nunits: [{ id: u1, title: Sequences and alignment }]\nreferences:\n \
durbin1998biological: { title: Biological Sequence Analysis }\n",
)
.unwrap();
std::fs::create_dir_all(root.join("lectures")).unwrap();
std::fs::create_dir_all(root.join("objectives")).unwrap();
let v20 = design().join("examples/v20");
std::fs::copy(v20.join("l09.yaml"), root.join("lectures/l09.yaml")).unwrap();
std::fs::copy(
v20.join("lo-alignment-recurrence.yaml"),
root.join("objectives/lo-alignment-recurrence.yaml"),
)
.unwrap();
std::fs::write(
root.join("objectives/lo-alignment-scoring.yaml"),
"learning_objectives:\n lo-alignment-scoring: { text: Score an alignment., unit: u1, \
level_ceiling: 3 }\nlearning_targets:\n t-align-matrix-terms: { text: Define the \
matrix., objective: lo-alignment-scoring, level_ceiling: 1 }\n \
t-align-local-vs-global: { text: Compare local and global., objective: \
lo-alignment-scoring, level_ceiling: 2 }\n",
)
.unwrap();
root
}
#[test]
fn the_2_0_fragments_load_and_validate_unchanged() {
let root = v20_course("v20-load");
let course = CourseFile::load_dir(&root).expect("2.0 fragments load");
let issues = course.validate();
assert!(issues.is_empty(), "{issues:#?}");
assert!(course.terms.is_empty() && course.equations.is_empty());
}
#[test]
fn the_2_1_example_loads_every_new_field() {
let root = v21_course("v21-load");
let catalog = Catalog::load(&root).expect("the 2.1 example loads");
let course = &catalog.course;
assert_eq!(course.terms.len(), 8);
assert_eq!(course.equations.len(), 1);
assert_eq!(course.values.len(), 2);
assert_eq!(course.misconceptions.len(), 2);
// Where each entry came from: the objective file, or glossary.yaml.
assert_eq!(
course.origin("term-gap").map(|(_, p)| p.to_path_buf()),
Some(PathBuf::from("objectives/lo-alignment-scoring.yaml"))
);
assert_eq!(
course
.origin("val-per-decade")
.map(|(_, p)| p.to_path_buf()),
Some(PathBuf::from("glossary.yaml"))
);
let target = &course.learning_targets["t-align-fill-matrix"];
assert!(target.explanation.is_some());
assert_eq!(target.equations, vec!["eq-global-recurrence".to_string()]);
assert!(target.authoring.as_ref().unwrap().ceiling_item.is_some());
let lecture = &course.lectures["l09"];
assert!(lecture.description.is_some());
assert_eq!(lecture.materials.len(), 3);
assert_eq!(lecture.readings[0].minutes, Some(40));
let policy = &course.policy;
assert_eq!(policy.levels["3"].name, "Application");
assert_eq!(policy.blueprints["module"].level_counts["2"], 18);
assert_eq!(policy.target_text_forbid.len(), 3);
assert_eq!(
policy.exam_conditions.as_ref().unwrap().seconds_per_item,
Some(120.0)
);
let item = catalog.get("q-align-which-scores-higher").expect("item");
assert_eq!(
item.item.options[1].misconception_id.as_deref(),
Some("mis-only-matches-count")
);
}
#[test]
fn the_migration_output_loads_as_a_glossary_fragment() {
let root = scratch("migration-output");
std::fs::write(
root.join("course.yaml"),
"course: { code: X, title: Y, term: Z }\nreferences:\n KKW: { title: The Molecules of \
Life }\n",
)
.unwrap();
std::fs::copy(
design().join("examples/migration/glossary.expected.yaml"),
root.join("glossary.yaml"),
)
.unwrap();
let course = CourseFile::load_dir(&root).expect("loads");
assert!(course.terms.contains_key("term-decade"));
assert!(course.values.contains_key("val-per-decade"));
}
#[test]
fn a_lecture_file_may_not_define_glossary_entries() {
let root = v20_course("v20-lecture-glossary");
std::fs::write(
root.join("lectures/l10.yaml"),
"lectures:\n l10: { title: Ten }\nterms:\n term-x: { term: x, definition: An x. }\n",
)
.unwrap();
let message = CourseFile::load_dir(&root).unwrap_err().to_string();
assert!(message.contains("may not define `terms`"), "{message}");
}
#[test]
fn a_glossary_id_defined_twice_names_both_files() {
let root = v21_course("v21-dup");
std::fs::write(
root.join("glossary.yaml"),
"terms:\n term-gap: { term: gap, definition: A dash. }\n",
)
.unwrap();
let message = CourseFile::load_dir(&root).unwrap_err().to_string();
assert!(
message.contains("term `term-gap` is defined in two places"),
"{message}"
);
assert!(message.contains("glossary.yaml"), "{message}");
assert!(
message.contains("objectives/lo-alignment-scoring.yaml"),
"{message}"
);
}
+309
View File
@@ -0,0 +1,309 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright Scienting
// Source: https://git.scient.ing/education/coursebank
//! The errors 2.1 adds, one test each.
//!
//! Each test breaks the example course in `docs/design/2.1/examples/v21/` the way
//! the prototype's self-test does (`prototype/check21.py --selftest`) and checks
//! that validation reports it. `docs/design/2.1/CODES.md` maps the
//! specification's codes to the messages asserted here.
mod common;
use common::{edit, v21_course};
use coursebank::catalog::Catalog;
use coursebank::course::CourseFile;
use coursebank::course::glossary::Symbol;
/// The example course, loaded.
fn course(tag: &str) -> CourseFile {
CourseFile::load_dir(&v21_course(tag)).expect("the example loads")
}
/// Asserts exactly one issue contains `needle`.
fn reports(issues: &[String], needle: &str) {
let hits = issues.iter().filter(|i| i.contains(needle)).count();
assert_eq!(
hits, 1,
"expected one issue containing {needle:?}, got {issues:#?}"
);
}
#[test]
fn the_example_validates_with_no_errors() {
let catalog = Catalog::load(&v21_course("valid")).unwrap();
let issues = catalog.validate().unwrap();
assert!(issues.is_empty(), "{issues:#?}");
}
#[test]
fn e2101_an_id_in_two_registries() {
let mut c = course("e2101");
let target = c.learning_targets["t-align-fill-matrix"].clone();
c.learning_targets.insert("term-gap".into(), target);
reports(
&c.validate(),
"`term-gap` is declared as both a learning target and a term",
);
}
#[test]
fn e2101_a_registry_id_without_its_prefix() {
let mut c = course("e2101-prefix");
let term = c.terms.remove("term-gap").unwrap();
c.terms.insert("gap".into(), term);
let issues = c.validate();
reports(&issues, "term `gap`: a term id is `term-` followed by");
}
#[test]
fn e2102_a_target_lists_an_undeclared_entry() {
let mut c = course("e2102");
c.learning_targets
.get_mut("t-align-fill-matrix")
.unwrap()
.terms
.push("term-nope".into());
reports(
&c.validate(),
"target `t-align-fill-matrix`: `terms` names `term-nope`, which is not a declared term",
);
}
#[test]
fn e2102_an_id_listed_under_the_wrong_registry() {
let mut c = course("e2102-wrong");
c.learning_targets
.get_mut("t-align-fill-matrix")
.unwrap()
.equations
.push("term-gap".into());
reports(
&c.validate(),
"`equations` names `term-gap`, which is not a declared equation",
);
}
#[test]
fn e2103_a_symbol_with_neither_term_nor_meaning() {
let mut c = course("e2103");
c.equations
.get_mut("eq-global-recurrence")
.unwrap()
.symbols
.push(Symbol {
symbol: "q".into(),
term: None,
meaning: None,
units: None,
});
reports(
&c.validate(),
"equation `eq-global-recurrence`: symbol `q` needs a `term` or a `meaning`",
);
}
#[test]
fn e2103_a_symbol_naming_an_undeclared_term() {
let mut c = course("e2103-term");
c.equations.get_mut("eq-global-recurrence").unwrap().symbols[2].term = Some("term-nope".into());
reports(
&c.validate(),
"symbol `d` names the term `term-nope`, which is not declared",
);
}
#[test]
fn e2104_a_value_with_neither_value_nor_range() {
let mut c = course("e2104");
c.values.get_mut("val-alignment-count-100").unwrap().value = None;
reports(
&c.validate(),
"value `val-alignment-count-100`: give a `value` or a `range`",
);
}
#[test]
fn e2104_a_range_that_runs_high_to_low_and_an_undeclared_term() {
let mut c = course("e2104-range");
let v = c.values.get_mut("val-per-decade").unwrap();
v.range = Some([7.0, 5.0]);
v.term = Some("term-nope".into());
let issues = c.validate();
reports(
&issues,
"value `val-per-decade`: range runs from 7 down to 5",
);
reports(
&issues,
"value `val-per-decade`: term `term-nope` is not declared",
);
}
#[test]
fn e2105_a_citation_key_not_in_references() {
let mut c = course("e2105");
c.values
.get_mut("val-alignment-count-100")
.unwrap()
.source
.as_mut()
.unwrap()
.reference = "nobody2020".into();
c.terms.get_mut("term-alignment-matrix").unwrap().textbook[0].reference =
Some("nobody2021".into());
c.misconceptions
.get_mut("mis-only-matches-count")
.unwrap()
.evidence[0]
.reference = Some("nobody2022".into());
let issues = c.validate();
reports(
&issues,
"source cites `nobody2020`, which is not in references",
);
reports(&issues, "textbook cites `nobody2021`");
reports(&issues, "evidence cites `nobody2022`");
}
#[test]
fn e2106_a_brace_reference_that_does_not_resolve() {
let mut c = course("e2106");
c.terms.get_mut("term-gap").unwrap().definition = "See {term-missing}.".into();
reports(
&c.validate(),
"term `term-gap`: definition refers to `{term-missing}`, which is not a declared term",
);
}
#[test]
fn e2106_an_old_reading_keeps_its_2_0_behavior_for_targets() {
// A 2.0 reading with a mistyped `{t-...}` validated and rendered the braces
// literally. It still validates (the linter reports it); a new kind of
// reference in the same field is checked.
let mut c = course("e2106-legacy");
let reading = &mut c.lectures.get_mut("l09").unwrap().readings[0];
reading.focus = Some("See {t-typo} and {val-typo}.".into());
let issues = c.validate();
assert!(!issues.iter().any(|i| i.contains("t-typo")), "{issues:#?}");
reports(&issues, "reading 1 focus refers to `{val-typo}`");
}
#[test]
fn e2106_a_new_field_checks_every_kind() {
let mut c = course("e2106-new");
c.learning_objectives
.get_mut("lo-alignment-scoring")
.unwrap()
.description = Some("Builds on {t-typo}.".into());
reports(
&c.validate(),
"objective `lo-alignment-scoring`: description refers to `{t-typo}`",
);
}
#[test]
fn e2107_a_pattern_that_does_not_compile() {
let mut c = course("e2107");
c.values
.get_mut("val-per-decade")
.unwrap()
.conflicts
.push("5.7(".into());
c.policy.target_text_forbid.push("[unclosed".into());
let issues = c.validate();
reports(
&issues,
"value `val-per-decade`: conflicts pattern `5.7(` does not compile",
);
reports(
&issues,
"policy.target_text_forbid: pattern `[unclosed` does not compile",
);
}
#[test]
fn e2108_an_option_names_an_undeclared_misconception() {
let root = v21_course("e2108");
edit(
&root,
"banks/alignment.yaml",
"misconception_id: mis-only-matches-count",
"misconception_id: mis-nope",
);
let issues = Catalog::load(&root).unwrap().validate().unwrap();
reports(
&issues,
"option o-matches-only: misconception_id `mis-nope` is not a declared misconception",
);
}
#[test]
fn messages_name_the_file_that_defined_the_entry() {
let mut c = course("attribution");
c.terms.get_mut("term-gap").unwrap().definition = "See {term-missing}.".into();
let issues = c.validate();
assert!(
issues
.iter()
.any(|i| i.starts_with("objectives/lo-alignment-scoring.yaml: term `term-gap`")),
"{issues:#?}"
);
}
#[test]
fn policy_levels_and_blueprints_are_keyed_by_level() {
let mut c = course("policy");
let level = c.policy.levels["1"].clone();
c.policy.levels.insert("6".into(), level);
c.policy
.blueprints
.get_mut("module")
.unwrap()
.level_counts
.insert("7".into(), 1);
let issues = c.validate();
reports(&issues, "policy.levels: `6` is not a level");
reports(&issues, "policy.blueprints: `module` counts items at `7`");
}
/// The checks the specification does not number: required text that is
/// present but empty, and numbers outside the range the schema gives.
#[test]
fn empty_required_text_and_out_of_range_numbers_are_errors() {
let mut c = course("required");
c.terms.get_mut("term-gap").unwrap().definition = " ".into();
c.equations.get_mut("eq-global-recurrence").unwrap().says = String::new();
c.values.get_mut("val-per-decade").unwrap().say = String::new();
c.misconceptions
.get_mut("mis-only-matches-count")
.unwrap()
.correction = String::new();
let lecture = c.lectures.get_mut("l09").unwrap();
lecture.materials[0].url = String::new();
lecture.readings[0].minutes = Some(0);
c.policy.levels.get_mut("3").unwrap().name = String::new();
c.policy.blueprints.get_mut("module").unwrap().minutes = Some(0.0);
c.policy.exam_conditions.as_mut().unwrap().seconds_per_item = Some(0.0);
let issues = c.validate();
reports(&issues, "term `term-gap`: empty definition");
reports(&issues, "equation `eq-global-recurrence`: empty says");
reports(&issues, "value `val-per-decade`: empty say");
reports(
&issues,
"misconception `mis-only-matches-count`: empty correction",
);
reports(&issues, "lecture `l09`: material 1 has no url");
reports(
&issues,
"lecture `l09` reading 1: minutes must be at least 1",
);
reports(&issues, "policy.levels: level 3 has an empty name");
reports(&issues, "policy.blueprints: `module` gives no time");
reports(
&issues,
"policy.exam_conditions: seconds_per_item must be above 0",
);
}
+2
View File
@@ -0,0 +1,2 @@
*
!.gitignore