Whether the engines' changes went back to the course or the course took them from here was an open question (docs/engines.md). Soundness fixes to the prover cannot wait for a round trip through a repository with no engine tests, so this repository now owns the engines. - pixi run engines-export <course>/static builds and writes every engine module into the course's static/js/: JavaScript modules as written, TypeScript ones (logic, solver, vcgen) as built, without source maps. - pixi run engines-diff now shows what the course has not taken yet, comparing the file the course would receive with its copy. Both scripts share the list of modules in scripts/engines-files.mjs. - docs/engines.md describes the ownership, the prover's three modules and its number model, the soundness tests and the bugs found so far, the changes the course receives with the next export, and how to export. - AGENTS.md gains a hard rule: never let the prover call something proved that a run breaks, with how to add a soundness regression first. - docs/app.md's Known limits no longer sends engine fixes upstream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgvbRdoEheAHqsRmr2WuKZ
60 lines
8.4 KiB
Markdown
60 lines
8.4 KiB
Markdown
# The engines
|
||
|
||
Programming+ runs the language engines that the course site also runs, and draws the hardware views with the course's stylesheets, so a program behaves the same in the app as on a course page. **This repository owns the engines.** Changes are made and tested here, and the course copies them from here (`pixi run engines-export`). The engines started as a copy of the course's modules; the course no longer sends changes the other way.
|
||
|
||
| Here | The course receives it as | What it is |
|
||
|---|---|---|
|
||
| `src/engines/pseudo/` | `static/js/pseudo/` | The pseudo language: parser, checker, interpreter, the English (lingo) reader, the prover (`logic.ts`, `solver.ts`, `vcgen.ts`), translation |
|
||
| `src/engines/asm86/` | `static/js/asm86/` | The x86-64 model machine, its caches, pipeline and out-of-order core, and the widgets that draw them |
|
||
| `src/engines/shared/` | `static/js/shared/` | DOM helpers, URL state, text distance, charts |
|
||
| `styles/course/` | `static/css/base/`, `static/css/widgets/` | The widget stylesheets ([styles/course/README.md](../styles/course/README.md)) |
|
||
|
||
The engines are wrapped behind one `LanguageService` interface (`src/core/types.ts`, adapters in `src/services/`). Modules the app does not use are excluded from the build in `tsconfig.json`. A change in `src/engines/` is a change to the course's language, so it needs a test here and goes to the course by export.
|
||
|
||
## The prover
|
||
|
||
The prover is three TypeScript modules under strict mode; the rest of the engines are still JavaScript.
|
||
|
||
- `logic.ts`: the term language (a discriminated union, `Term`), the smart constructors that simplify as they build, keys, substitution, linear forms and printing.
|
||
- `solver.ts`: the decision procedure, `prove(facts, goal)`. It answers `proved` or `unknown`, never "false". Equalities by union-find with congruence, linear arithmetic over whole numbers by Fourier-Motzkin with integer tightening, case splits, and FORALL used at the positions a case mentions. It is sound and incomplete: products of two unknowns, MOD by an unknown, and most facts about collections are opaque to it.
|
||
- `vcgen.ts`: the proof obligations, by symbolic execution, with loops cut at their invariants and simple invariants found by Houdini-style inference.
|
||
|
||
"Proved" is about the notation's own number model. INTEGER is a whole number of any size and a FLOAT is a finite number whose rounding the prover does not see into. A run of the interpreter stops where a value leaves that model (a whole number past 2^53 − 1, a FLOAT that is not finite, a division by zero), so every run stays inside what a proof covers. The Mojo made from a program has other limits (a 64-bit `Int` that wraps, a `Float64` that becomes `inf` or `nan`), and a unit whose proof leans on the number model carries an `int-model` or `float-model` note saying so.
|
||
|
||
### Soundness
|
||
|
||
A "proved" must be a proof. Two test files hold the prover to that, and a change to `logic.ts`, `solver.ts` or `vcgen.ts` must keep both green:
|
||
|
||
- `tests/prover-soundness.test.mjs`: every regression program in `tests/prover/` names, in a `// MUST-NOT-PROVE: <kind>@<line>` header, an obligation that a run breaks and that an unsound rule once proved; it may never be proved again, and the program must still break it. A program whose fix was to the language names where its run now stops, `// RUN-STOPS: <kind>@<line>`. Across every exercise, corpus program, lesson solution and example, nothing marked proved may be broken by the fuzzer or by running the program, and `ASSERT 1 == 2` appended to a program that runs to its end may never be proved.
|
||
- `tests/engine-solver.test.mjs`: a table of goals the solver must prove and goals it must leave unknown, 400 random linear integer formulas checked against every small assignment, and obligations from small programs.
|
||
|
||
When a soundness bug is found, add the program that shows it to `tests/prover/` first, watch the test fail, then fix the rule. The bugs found so far, each with its program:
|
||
|
||
| Program | Rule | Fix |
|
||
|---|---|---|
|
||
| `houdini-cap.pseudo` | Invariant inference stopped after four rounds and assumed the candidates left, even when the last round still dropped some | Keep inferred invariants only at a fixed point |
|
||
| `rand-zero.pseudo`, `rand-negative.pseudo` | `RAND(X)` assumed `0 <= R < X` for every X, false for X ≤ 0, so the checker held a contradiction and proved anything | A fact that covers all three signs of X |
|
||
| `fill-negative.pseudo`, `randint-empty.pseudo` | `FILL` and `RANDINT` assumed facts about arguments the interpreter checks by stopping, without asking whether they hold | An `argument` obligation, and guarded facts |
|
||
| `float-overflow.pseudo` | The prover treats FLOAT as numbers (`Y == Y`), while a run went on with infinity and NaN | The interpreter stops on a FLOAT that is not finite |
|
||
| `foreach-empty.pseudo` | The way out of `FOR EACH` kept "X is in the collection", false after zero passes over an empty list | The fact holds only inside a pass |
|
||
|
||
In the app, an obligation the prover marks proved that a generated input breaks at the same place is shown as broken, not proved.
|
||
|
||
## What this repository changed
|
||
|
||
Changes made here since the engines left the course, which the course receives with the next export:
|
||
|
||
- **pseudo:** the text built-ins `SPLIT`, `JOIN`, `INDEX_OF`, `REPLACE` and the numerical built-ins (`FLOOR`, `CEIL`, `LOG10`, `SIN`, `COS`, `TAN`, `ATAN2`, `PI`, the statistics) in `lang`, `parse`, `check`, `interp`, `translate`, `vcgen`; `PARALLEL FOR` with its race detector; an indexed read `L[I]` recorded as a read of one position, not all of `L` (`interp`); one `reserved-name` error for a keyword used as a name (`parse`); `english.js` exporting `COMPARISONS` and `IN_PHRASES` so the reference can read them; `analyze.js` and `coach.js` treating a variable passed `BYREF` to a procedure that writes it first as an out-parameter, not a read; `coach.js` no longer warning "growing bound" on a queue drained by `WHILE LENGTH(Q) > 0`; the soundness fixes above, with the new `argument` obligation kind (`vcgen`, and `views-analysis.js` for the course's pages); a run that stops on a FLOAT that is not finite (`interp`); and `logic`, `solver` and `vcgen` rewritten in TypeScript with the same behavior.
|
||
- **asm86:** what compiled Mojo uses beyond the course's subset (`interp.js`, about 630 lines): the BMI bit instructions, atomic read-modify-writes with `lock`, bit tests that write, `bswap`, `ud2`, the 128-bit AVX spellings, scalar double precision, and the double-width shifts; the parity flag (`interp`, `uops`, `views`); the floating-point units and their latencies (`uops`); a small change in `sandbox.js`; and in `hwviews.js` the whole core's arrows (`machineView`): only a lit edge carries an arrowhead, in its own color and on the end its data reached, each view has its own marker ids, and the result's edge into the registers starts at the port that produced it.
|
||
|
||
One addition to the pseudo language lives outside the engine, in app code: `DECLARE <name> : <type> <- <value>`, a declaration with its starting value on one line (Mojo's `var total: Int = 0`). `src/services/pseudo-study.ts` reads it by doing what the engine's `study()` does (parse, check, analyze, coach) with one step between parse and check: the `<- value` is blanked out of the line, columns kept, so the engine parses a plain DECLARE, and the value is parsed on its own and put in as an ordinary assignment on the same line. Every app reader of pseudo (`studyOf`, and the lingo, matho and Mojo translators) goes through it; pseudo-to-mojo writes the pair as one `var` line. The two-line form reads as before. It can move into the engine's parser now that the engine lives here.
|
||
|
||
## Giving the course the engines
|
||
|
||
1. `pixi run engines-diff <course>/static` shows what the course has not taken yet: for each module, the file the course would receive (`src/engines/` as written, `dist/engines/` for a TypeScript module) against the course's copy, whitespace ignored, and the files only one side has.
|
||
2. `pixi run engines-export <course>/static` builds and writes those files into the course's `static/js/`. A TypeScript module arrives as plain ES-module JavaScript, so the course's pages import it unchanged. Source maps are not copied.
|
||
3. In the course repository, review the diff and run its pages before committing.
|
||
4. For the stylesheets, follow [styles/course/README.md](../styles/course/README.md); they are still identical on both sides.
|
||
|
||
A change someone makes in the course's copy is not picked up from there. Bring it here as a change with a test, then export.
|