feat: move corpus to library
Test / test (push) Failing after 1m38s

This commit is contained in:
alexm committed 2026-10-06 10:27:46 -04:00
1 parent 5b8b079165
commit cae5cff55a
1350 files changed
+197 -941995

No files matched your search

+21 -19
View File
@@ -4,21 +4,23 @@ Programming+ teaches what Mojo does when it runs, so everything it says about Mo
Every command below is a pixi task; the [README](../README.md#commands) lists them all.
The corpus itself (the pin, the harvested material, our programs and godbolt's answers) lives in pplus-library, the submodule `library/`, under `library/mojo/`; the pipeline that builds and checks it (`scripts/mojo/`) and what it generates for the app (`src/mojo/generated/`) live here. A task that changes the corpus leaves changes in both repositories: commit and push `library/` first, then commit core with the new pointer (`git add library`) and the regenerated files.
## Mojo of our own
There are three kinds of Mojo program we write, and each has a folder, a task to start one, and a job:
| Kind | Folder | Start one | Its job | Shipped in the app |
|---|---|---|---|---|
| probe | `mojo/probes/` | `pixi run new-probe <name> "<question>"` | Ask the pinned compiler one question (what does `//` do on a negative Int?). Godbolt's answer is the point; it is recorded, not judged. | no, but the checker and interpreter are held to the answers |
| exercise program | `mojo/local/` | `pixi run new-exercise <id>` | The Mojo of a catalog problem. It must pass its own `# expect-stdout:` lines. | yes, in `src/mojo/generated/programs.ts`, badged verified when it passed |
| stress program | `mojo/stress/` | `pixi run new-stress <name> "<what it exercises>"` | Realistic Mojo that stresses the interpreter. Its `# expect-stdout:` lines are the interpreter's prediction; godbolt confirms or refutes it. | no |
| probe | `library/mojo/probes/` | `pixi run new-probe <name> "<question>"` | Ask the pinned compiler one question (what does `//` do on a negative Int?). Godbolt's answer is the point; it is recorded, not judged. | no, but the checker and interpreter are held to the answers |
| exercise program | `library/mojo/local/` | `pixi run new-exercise <id>` | The Mojo of a catalog problem. It must pass its own `# expect-stdout:` lines. | yes, in `src/mojo/generated/programs.ts`, badged verified when it passed |
| stress program | `library/mojo/stress/` | `pixi run new-stress <name> "<what it exercises>"` | Realistic Mojo that stresses the interpreter. Its `# expect-stdout:` lines are the interpreter's prediction; godbolt confirms or refutes it. | no |
The loop is the same for all three:
```bash
pixi run new-probe int-floordiv-negative "does // floor toward negative infinity on Int?"
# edit mojo/probes/int-floordiv-negative.mojo
# edit library/mojo/probes/int-floordiv-negative.mojo
pixi run mojo-verify-item 'probes/int-floordiv-negative' # ask godbolt (network)
pixi run test # the checker and interpreter scoreboards read the new answer
```
@@ -59,7 +61,7 @@ Header lines (`# probe:`, `# mode:`, …) are for `verify`; only those keys are
## Reading the status
`pixi run mojo-status` judges every item from the stored answers, offline, and writes `mojo/golden/STATUS.md` (a table per corpus, every failure with its reason, and every probe's question and verdict) and `status.json` (`git diff mojo/golden/status.json` shows exactly what changed).
`pixi run mojo-status` judges every item from the stored answers, offline, and writes `library/mojo/golden/STATUS.md` (a table per corpus, every failure with its reason, and every probe's question and verdict) and `status.json` (`git -C library diff mojo/golden/status.json` shows exactly what changed).
Each item says what the pinned compiler should do with it: `pass` (compiles, runs, exits 0, and prints what it should or contains the idiom it should), `fail` (that check must not hold: a starter), `compile-error` (must be rejected, with each expected error at its line), `observe` (a probe: the answer is the point), or `skip` (with a reason: newer syntax than the pin, no `main`, not reproducible on public godbolt). Verdicts are `pass`, `fail`, `observed`, `skipped`, and `unverified` (no golden yet).
@@ -71,17 +73,17 @@ The dialect markers in the pin are a cheap first look: `newer` patterns are synt
| Path | What it is | Who writes it |
|---|---|---|
| `mojo/pin.json` | The Mojo version, the godbolt compiler id, each source repository and the ref to use, and the dialect markers | You |
| `mojo/mojo.lock.json` | What each ref resolved to: commit, zip sha256, and the directory inside the zip | `fetch`/`harvest` |
| `mojo/corpus/` | Processed material, one directory per processor, each item with its provenance | `harvest` |
| `mojo/corpus/SOURCES.md` | Which commit and license everything came from, what each processor found, every warning | `harvest` |
| `mojo/probes/` | Our questions about the pinned compiler's behavior, one small program each | You (`new-probe`) |
| `mojo/local/` | Exercise programs, shipped in the app | You (`new-exercise`) |
| `mojo/stress/` | Realistic programs that test the interpreter, not shipped | You (`new-stress`) |
| `mojo/golden/<key>.json` | Godbolt's answer for one program, keyed by compiler, mode, arguments, stdin and source | `verify` |
| `mojo/golden/STATUS.md` | Every item's verdict against the goldens | `status` (also `harvest`, `verify`) |
| `src/mojo/generated/` | What the app reads at run time: `pin.ts` (version, compiler), `tokens.ts` (the words, removed words and the compiler's messages for them), `editor.ts` (comment and indentation rules), `programs.ts` (`mojo/local/` programs with their verdicts), `stdlib.ts` and `types.ts` (which package exports each name, which types are copyable), and `listings.ts` (exercise listings, written by `scripts/mojo/listings.mjs` during `play`) | `harvest`/`emit`; `verify` and `status` refresh `programs.ts` |
| `syntax/mojo.xml` | The same words as a KDE syntax for Pandoc listings (pplus-web copies it into its Quarto extension with its quarto-css task) | `harvest`/`emit` |
| `library/mojo/pin.json` | The Mojo version, the godbolt compiler id, each source repository and the ref to use, and the dialect markers | You |
| `library/mojo/mojo.lock.json` | What each ref resolved to: commit, zip sha256, and the directory inside the zip | `fetch`/`harvest` |
| `library/mojo/corpus/` | Processed material, one directory per processor, each item with its provenance | `harvest` |
| `library/mojo/corpus/SOURCES.md` | Which commit and license everything came from, what each processor found, every warning | `harvest` |
| `library/mojo/probes/` | Our questions about the pinned compiler's behavior, one small program each | You (`new-probe`) |
| `library/mojo/local/` | Exercise programs, shipped in the app | You (`new-exercise`) |
| `library/mojo/stress/` | Realistic programs that test the interpreter, not shipped | You (`new-stress`) |
| `library/mojo/golden/<key>.json` | Godbolt's answer for one program, keyed by compiler, mode, arguments, stdin and source | `verify` |
| `library/mojo/golden/STATUS.md` | Every item's verdict against the goldens | `status` (also `harvest`, `verify`) |
| `src/mojo/generated/` | What the app reads at run time: `pin.ts` (version, compiler), `tokens.ts` (the words, removed words and the compiler's messages for them), `editor.ts` (comment and indentation rules), `programs.ts` (`library/mojo/local/` programs with their verdicts), `stdlib.ts` and `types.ts` (which package exports each name, which types are copyable), and `listings.ts` (exercise listings, written by `scripts/mojo/listings.mjs` during `play`) | `harvest`/`emit`; `verify` and `status` refresh `programs.ts` |
| `library/syntax/mojo.xml` | The same words as a KDE syntax for Pandoc listings (pplus-web copies it into its Quarto extension with its quarto-css task) | `harvest`/`emit` |
Commit all of it. The stored answers are what make `pixi run test` and `mojo-status` work offline.
@@ -116,11 +118,11 @@ A processor whose input files are missing at the pinned ref is skipped with a wa
## Moving to a new Mojo version
1. Find the release. Mojo releases are tagged `mojo/vX.Y.Z` in `modular/modular` (the platform's own tags are `modular/vX.Y`). Find godbolt's compiler id for it, `mojo_X_Y_Z`, in `https://godbolt.org/api/compilers/mojo`.
2. Edit `mojo/pin.json`: `mojo.version`, `godbolt.compiler`, and `sources.modular.ref`. Move the other refs to the commits that match: Mojo Quest's commit whose `src/lib/compile.ts` names the new compiler id; the vscode-mojo, mojo-syntax and skills commits released alongside it.
2. Edit `library/mojo/pin.json`: `mojo.version`, `godbolt.compiler`, and `sources.modular.ref`. Move the other refs to the commits that match: Mojo Quest's commit whose `src/lib/compile.ts` names the new compiler id; the vscode-mojo, mojo-syntax and skills commits released alongside it.
3. Review the dialect markers against the new release notes. Anything in the notes' **Removed** or **Language changes** sections belongs in `older`; anything in `newer` that this release now includes comes out. (`newer` matters only when harvesting a snapshot past a release.)
4. `pixi run mojo-harvest`, then read `mojo/corpus/SOURCES.md`: every warning is something to look at, and a `release` line saying `snapshot newer` means the ref is not the release tag.
4. `pixi run mojo-harvest`, then read `library/mojo/corpus/SOURCES.md`: every warning is something to look at, and a `release` line saying `snapshot newer` means the ref is not the release tag.
5. `pixi run mojo-verify`. A new compiler id changes every key, so every item is asked again: about 440 requests, two at a time. Run it once, not in a loop; the public server is shared.
6. `pixi run mojo-status`, then read `mojo/golden/STATUS.md`. `git diff mojo/golden/status.json` shows exactly which verdicts and probe answers changed between versions. A changed probe answer is a change Programming+'s Mojo engine has to make; a failing Mojo Quest solution or manual sample usually means a language change the markers should know about.
6. `pixi run mojo-status`, then read `library/mojo/golden/STATUS.md`. `git -C library diff mojo/golden/status.json` shows exactly which verdicts and probe answers changed between versions. A changed probe answer is a change Programming+'s Mojo engine has to make; a failing Mojo Quest solution or manual sample usually means a language change the markers should know about.
7. `pixi run test`, then commit the pin, the lock, the corpus, the goldens, and `src/mojo/generated/` together.
## Licenses and provenance