Make this repository the engines' home; the course copies from here

Whether the engines' changes went back to the course or the course took
them from here was an open question (docs/engines.md). Soundness fixes to
the prover cannot wait for a round trip through a repository with no engine
tests, so this repository now owns the engines.

- pixi run engines-export <course>/static builds and writes every engine
  module into the course's static/js/: JavaScript modules as written,
  TypeScript ones (logic, solver, vcgen) as built, without source maps.
- pixi run engines-diff now shows what the course has not taken yet,
  comparing the file the course would receive with its copy. Both scripts
  share the list of modules in scripts/engines-files.mjs.
- docs/engines.md describes the ownership, the prover's three modules and
  its number model, the soundness tests and the bugs found so far, the
  changes the course receives with the next export, and how to export.
- AGENTS.md gains a hard rule: never let the prover call something proved
  that a run breaks, with how to add a soundness regression first.
- docs/app.md's Known limits no longer sends engine fixes upstream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgvbRdoEheAHqsRmr2WuKZ
This commit is contained in:
alexmandClaude committed 2026-10-05 03:33:58 +00:00
1 parent 2416d7fc8e
commit 0d313f989d
8 files changed
+151 -42

No files matched your search

+4 -2
View File
@@ -27,12 +27,13 @@ The full suite takes about two minutes. While iterating, build and run one file:
3. **Never claim Mojo compiles, runs, or prints something without the real compiler's answer.** Mojo is pinned to 1.1.0. The checker may never report an error on a program the compiler accepts, and the interpreter must print exactly what godbolt printed or decline; `tests/mojo-check.test.mjs` and `tests/mojo-run.test.mjs` hold them to that. Every Mojo construct you write into the app must pass `checkMojo`, and every standard-library name must exist in `src/mojo/generated/stdlib.ts` or `types.ts`.
4. **No new dependencies.** TypeScript is the only npm package. There is no bundler, no framework, and no runtime dependency; the corpus pipeline is Python's standard library only. CodeMirror is vendored as one built module (`src/vendor/`), rebuilt only by `scripts/vendor/codemirror.sh`.
5. **You have no network** in most sandboxes. Godbolt (`mojo-verify`, Compile & run) and GitHub (`mojo-harvest`, `mojo-fetch`) need one. Never fake their results; leave those steps to the owner and list them.
6. **Keep the workspace working while you change it.** Code panes are always the CodeMirror editor (`Editor` in `src/ui/editor.ts`, over the vendored `src/vendor/codemirror.js`), never a `<textarea>`; `tests/editor.test.mjs` checks it. Keep the workspace's parts as they are unless the task changes them: the pane tabs over one editor, the stepper docked under the editor, the side column (the inspect panel, whose first tab is the Run view), and the dock; in the Hardware area, the code on the left and the arrangeable board on the right. Restyle or rearrange them only when asked, and after any UI change open the app (`pixi run dev`) and check that the editor has line numbers and highlighting, the stepper steps, and the styles loaded.
6. **Never let the prover call something proved that a run breaks.** "Proved" in the app must be a proof. A change to `src/engines/pseudo/logic.ts`, `solver.ts` or `vcgen.ts` keeps `tests/prover-soundness.test.mjs` and `tests/engine-solver.test.mjs` green, and a fact the prover assumes must hold for every value a run can produce, or be guarded by an obligation. When you find a soundness bug, add the program that shows it to `tests/prover/` with a `// MUST-NOT-PROVE:` (or `// RUN-STOPS:`) header first, see the test fail, then fix the rule ([docs/engines.md](docs/engines.md#soundness)).
7. **Keep the workspace working while you change it.** Code panes are always the CodeMirror editor (`Editor` in `src/ui/editor.ts`, over the vendored `src/vendor/codemirror.js`), never a `<textarea>`; `tests/editor.test.mjs` checks it. Keep the workspace's parts as they are unless the task changes them: the pane tabs over one editor, the stepper docked under the editor, the side column (the inspect panel, whose first tab is the Run view), and the dock; in the Hardware area, the code on the left and the arrangeable board on the right. Restyle or rearrange them only when asked, and after any UI change open the app (`pixi run dev`) and check that the editor has line numbers and highlighting, the stepper steps, and the styles loaded.
## Conventions
- **Commands are pixi tasks.** Name them as `pixi run <task>` in docs, comments, and messages. A new command is a task in `pixi.toml` with a `description` and a row in the README's Commands table; `tests/tasks.test.mjs` fails otherwise, and also fails on any `pixi run <name>` that names no task. `package.json` keeps only `build`, `watch`, `check`, `dev`, `test` (what CI uses).
- **Docs.** The README is the front door: keep it short. Put behavior in [docs/app.md](docs/app.md), exercises in [docs/exercises.md](docs/exercises.md), Mojo of our own and the pipeline in [docs/mojo-corpus.md](docs/mojo-corpus.md), the engines in [docs/engines.md](docs/engines.md), deployment in [docs/deploy.md](docs/deploy.md), and the Quarto extension in [quarto/README.md](quarto/README.md). Update the document a change belongs in, in the same commit.
- **Docs.** The README is the front door: keep it short. Put behavior in [docs/app.md](docs/app.md), exercises in [docs/exercises.md](docs/exercises.md), Mojo of our own and the pipeline in [docs/mojo-corpus.md](docs/mojo-corpus.md), the engines and the prover's soundness in [docs/engines.md](docs/engines.md), deployment in [docs/deploy.md](docs/deploy.md), and the Quarto extension in [quarto/README.md](quarto/README.md). Update the document a change belongs in, in the same commit.
- **Tests run against `dist/` with no DOM.** Put logic in DOM-free modules (`src/core/`, `src/services/`, `src/mojo/`, `src/quarto/`) and test it there. The views in `src/ui/views/` also run outside a browser, under the small document of `src/quarto/dom.ts`, when the Quarto extension renders them: a view may use the selectors that document knows (compound selectors and the descendant combinator) and must not need layout to produce its markup. When a test needs `localStorage`, install a fake on `globalThis` and remove it in `finally`, as `tests/state.test.mjs` does. For a UI change, write a short manual check list in the commit message.
- **TypeScript.** Strict mode, ES modules, imports with the `.js` suffix (`from './state.js'`). Indent like the file you are editing (some files use two spaces, some four).
- **Stable ids.** Diagnostic ids (`matho-refused`, `reserved-name`, ...) and reference section ids are asserted by tests and linked from guides. Change one only on purpose, and update every test and link that names it.
@@ -47,6 +48,7 @@ The full suite takes about two minutes. While iterating, build and run one file:
| A question about the compiler | `pixi run new-probe <name> "<question>"` |
| Realistic Mojo for the interpreter | `pixi run new-stress <name> "<what it exercises>"`, with the output you predict in `# expect-stdout:` lines |
| A pseudo regression program | a file in `tests/corpus/` whose `// EXPECT:` lines are the output it must print |
| A prover soundness regression | a file in `tests/prover/` whose `// MUST-NOT-PROVE: <kind>@<line>` names an obligation a run breaks, or `// RUN-STOPS: <kind>@<line>` where a run must stop |
| A command | a task in `pixi.toml` and a row in the README's Commands table |
| A language | a `LanguageService` (`src/core/types.ts`) in `src/services/`, registered in `src/services/index.ts` |
| A Quarto shortcode | a function in the table `quarto/_extensions/pplus/pplus.lua` returns, a row in `SHORTCODES` (`src/quarto/shortcodes.ts`) and in `scripts/check-pages.mjs`, a sample in the gallery (`src/ui/quarto.ts`) and a use in `quarto/example/`; a rendered view is a case of `renderStatic` (`src/quarto/render.ts`); a view of the machine is a name in `HW_NAMES` (`src/quarto/machine.ts`), `HW_VIEWS` in `pplus.lua` and `HW_VIEWS` in `scripts/check-pages.mjs` |
+4 -3
View File
@@ -40,7 +40,8 @@ Every command is a pixi task (`pixi task list` shows them with their description
| `dev` | Build, then serve on http://localhost:4173 |
| `test` | Build, then run every test, the corpus pipeline's Python tests included |
| `check-pages <dir>` | Run every program embedded in the Quarto pages under a folder ([quarto/README.md](quarto/README.md)) |
| `engines-diff <static>` | Compare the vendored course code with a course checkout ([docs/engines.md](docs/engines.md)) |
| `engines-diff <static>` | Show what a course checkout has not taken yet from the engines and stylesheets ([docs/engines.md](docs/engines.md)) |
| `engines-export <static>` | Copy the engines into a course checkout, TypeScript modules as built ([docs/engines.md](docs/engines.md)) |
| **Write content** | |
| `new-exercise <id>` | Start an exercise whose Mojo lives in `mojo/local/<id>.mojo` ([docs/exercises.md](docs/exercises.md)) |
| `new-probe <name> <question>` | Start `mojo/probes/<name>.mojo`: one question about the compiler ([docs/mojo-corpus.md](docs/mojo-corpus.md)) |
@@ -90,7 +91,7 @@ src/
ui/hw/ the Hardware area's machine board, over the course's x86-64 views
exercises/ one file per catalog problem
curriculum/ a lesson corpus kept as an interpreter fixture (tests/curriculum.test.mjs); no UI
engines/ the course site's language engines (docs/engines.md); do not edit here
engines/ the language engines this repository owns and the course copies (docs/engines.md)
vendor/ CodeMirror 6 as one built module; scripts/vendor/ rebuilds it
styles/course/ the course site's widget stylesheets, copied unchanged (and bridge.css, ours)
tests/ node --test against dist/; tests/corpus/ holds pseudo regression programs
@@ -112,6 +113,6 @@ scratch/ ignored by git: a place for experiments
| [docs/exercises.md](docs/exercises.md) | Adding a catalog problem |
| [docs/mojo-corpus.md](docs/mojo-corpus.md) | Writing Mojo of our own (probes, exercise programs, stress programs) and moving to a new Mojo version |
| [docs/asm-coverage.md](docs/asm-coverage.md) | Which instructions compiled Mojo uses, against what the machine models |
| [docs/engines.md](docs/engines.md) | The vendored course engines and stylesheets, and how they differ from the course |
| [docs/engines.md](docs/engines.md) | The language engines this repository owns, their tests, and how the course takes them |
| [docs/deploy.md](docs/deploy.md) | Building the site and publishing it to the droplet |
| [quarto/README.md](quarto/README.md) | Embedding views in a Quarto site |
+1 -1
View File
@@ -218,6 +218,6 @@ These laws are tested in `tests/sync-speco.test.mjs`, for `count-base-proved`, `
## Known limits
- The engines are the course's modules as they stand, including the issues in the toolchain review (no engine tests of their own, two unsound verifier rules, English lowered through text). Fixes belong in the engines, upstream; this app will pick them up by re-vendoring. The vendored copy carries a few small changes, each listed in [engines.md](engines.md): the four text built-ins `SPLIT`, `JOIN`, `INDEX_OF`, `REPLACE` (lang, check, interp, translate, vcgen), the numerical built-ins and `PARALLEL FOR` with its race detector (lang, parse, check, interp, translate, vcgen), an indexed read `L[I]` recorded as a read of one position rather than of the whole of `L` (interp), and a single `reserved-name` error for a keyword used as a name (parse); and three earlier ones: `english.js` exports `COMPARISONS` and `IN_PHRASES` so the reference can read them; `analyze.js` tags a variable passed to a `BYREF` parameter and `coach.js` no longer reports it as "read before set" when the procedure writes the parameter before reading it (an out-parameter); and `coach.js` no longer warns "growing bound" on a `WHILE LENGTH(Q) > 0` whose body also `REMOVE`s from `Q` (a queue).
- The engines live in this repository ([engines.md](engines.md)). The prover is sound and incomplete: it proves only what follows on every path it examined, and `tests/prover-soundness.test.mjs` holds it to that; English is still lowered to pseudo through text, and the parser, checker and interpreter are still JavaScript without tests of their own beyond the app's.
- The assembly pane takes the course's Intel-syntax subset; directives and `[rip + label]` are not supported yet.
- Student work persists in the browser only.
+44 -16
View File
@@ -1,31 +1,59 @@
# The vendored course code
# The engines
Programming+ runs the course site's own language engines and draws the hardware views with the course's own stylesheets, so a program behaves the same in the app as on a course page.
Programming+ runs the language engines that the course site also runs, and draws the hardware views with the course's stylesheets, so a program behaves the same in the app as on a course page. **This repository owns the engines.** Changes are made and tested here, and the course copies them from here (`pixi run engines-export`). The engines started as a copy of the course's modules; the course no longer sends changes the other way.
| Here | From the course repository | What it is |
| Here | The course receives it as | What it is |
|---|---|---|
| `src/engines/pseudo/` | `static/js/pseudo/` | The pseudo language: parser, checker, interpreter, the English (lingo) reader, the prover's verification conditions, translation |
| `src/engines/pseudo/` | `static/js/pseudo/` | The pseudo language: parser, checker, interpreter, the English (lingo) reader, the prover (`logic.ts`, `solver.ts`, `vcgen.ts`), translation |
| `src/engines/asm86/` | `static/js/asm86/` | The x86-64 model machine, its caches, pipeline and out-of-order core, and the widgets that draw them |
| `src/engines/shared/` | `static/js/shared/` | DOM helpers, URL state, text distance, charts |
| `styles/course/` | `static/css/base/`, `static/css/widgets/` | The widget stylesheets ([styles/course/README.md](../styles/course/README.md)) |
The engines are wrapped behind one `LanguageService` interface (`src/core/types.ts`, adapters in `src/services/`). Modules the app does not use are excluded from the build in `tsconfig.json`. Do not edit `src/engines/` casually: a change there is a change to the course's language.
The engines are wrapped behind one `LanguageService` interface (`src/core/types.ts`, adapters in `src/services/`). Modules the app does not use are excluded from the build in `tsconfig.json`. A change in `src/engines/` is a change to the course's language, so it needs a test here and goes to the course by export.
## How the copies differ from the course
## The prover
`pixi run engines-diff <course>/static` compares every file, ignoring whitespace. As of the last check the stylesheets are identical; most engine modules differ only in formatting (they were reindented); and fourteen differ in content, every one of them a change made here and not yet sent back:
The prover is three TypeScript modules under strict mode; the rest of the engines are still JavaScript.
- **pseudo:** the text built-ins `SPLIT`, `JOIN`, `INDEX_OF`, `REPLACE` and the numerical built-ins (`FLOOR`, `CEIL`, `LOG10`, `SIN`, `COS`, `TAN`, `ATAN2`, `PI`, the statistics) in `lang`, `parse`, `check`, `interp`, `translate`, `vcgen`; `PARALLEL FOR` with its race detector; an indexed read `L[I]` recorded as a read of one position, not all of `L` (`interp`); one `reserved-name` error for a keyword used as a name (`parse`); `english.js` exporting `COMPARISONS` and `IN_PHRASES` so the reference can read them; `analyze.js` and `coach.js` treating a variable passed `BYREF` to a procedure that writes it first as an out-parameter, not a read; and `coach.js` no longer warning "growing bound" on a queue drained by `WHILE LENGTH(Q) > 0`.
- `logic.ts`: the term language (a discriminated union, `Term`), the smart constructors that simplify as they build, keys, substitution, linear forms and printing.
- `solver.ts`: the decision procedure, `prove(facts, goal)`. It answers `proved` or `unknown`, never "false". Equalities by union-find with congruence, linear arithmetic over whole numbers by Fourier-Motzkin with integer tightening, case splits, and FORALL used at the positions a case mentions. It is sound and incomplete: products of two unknowns, MOD by an unknown, and most facts about collections are opaque to it.
- `vcgen.ts`: the proof obligations, by symbolic execution, with loops cut at their invariants and simple invariants found by Houdini-style inference.
"Proved" is about the notation's own number model. INTEGER is a whole number of any size and a FLOAT is a finite number whose rounding the prover does not see into. A run of the interpreter stops where a value leaves that model (a whole number past 2^53 − 1, a FLOAT that is not finite, a division by zero), so every run stays inside what a proof covers. The Mojo made from a program has other limits (a 64-bit `Int` that wraps, a `Float64` that becomes `inf` or `nan`), and a unit whose proof leans on the number model carries an `int-model` or `float-model` note saying so.
### Soundness
A "proved" must be a proof. Two test files hold the prover to that, and a change to `logic.ts`, `solver.ts` or `vcgen.ts` must keep both green:
- `tests/prover-soundness.test.mjs`: every regression program in `tests/prover/` names, in a `// MUST-NOT-PROVE: <kind>@<line>` header, an obligation that a run breaks and that an unsound rule once proved; it may never be proved again, and the program must still break it. A program whose fix was to the language names where its run now stops, `// RUN-STOPS: <kind>@<line>`. Across every exercise, corpus program, lesson solution and example, nothing marked proved may be broken by the fuzzer or by running the program, and `ASSERT 1 == 2` appended to a program that runs to its end may never be proved.
- `tests/engine-solver.test.mjs`: a table of goals the solver must prove and goals it must leave unknown, 400 random linear integer formulas checked against every small assignment, and obligations from small programs.
When a soundness bug is found, add the program that shows it to `tests/prover/` first, watch the test fail, then fix the rule. The bugs found so far, each with its program:
| Program | Rule | Fix |
|---|---|---|
| `houdini-cap.pseudo` | Invariant inference stopped after four rounds and assumed the candidates left, even when the last round still dropped some | Keep inferred invariants only at a fixed point |
| `rand-zero.pseudo`, `rand-negative.pseudo` | `RAND(X)` assumed `0 <= R < X` for every X, false for X ≤ 0, so the checker held a contradiction and proved anything | A fact that covers all three signs of X |
| `fill-negative.pseudo`, `randint-empty.pseudo` | `FILL` and `RANDINT` assumed facts about arguments the interpreter checks by stopping, without asking whether they hold | An `argument` obligation, and guarded facts |
| `float-overflow.pseudo` | The prover treats FLOAT as numbers (`Y == Y`), while a run went on with infinity and NaN | The interpreter stops on a FLOAT that is not finite |
| `foreach-empty.pseudo` | The way out of `FOR EACH` kept "X is in the collection", false after zero passes over an empty list | The fact holds only inside a pass |
In the app, an obligation the prover marks proved that a generated input breaks at the same place is shown as broken, not proved.
## What this repository changed
Changes made here since the engines left the course, which the course receives with the next export:
- **pseudo:** the text built-ins `SPLIT`, `JOIN`, `INDEX_OF`, `REPLACE` and the numerical built-ins (`FLOOR`, `CEIL`, `LOG10`, `SIN`, `COS`, `TAN`, `ATAN2`, `PI`, the statistics) in `lang`, `parse`, `check`, `interp`, `translate`, `vcgen`; `PARALLEL FOR` with its race detector; an indexed read `L[I]` recorded as a read of one position, not all of `L` (`interp`); one `reserved-name` error for a keyword used as a name (`parse`); `english.js` exporting `COMPARISONS` and `IN_PHRASES` so the reference can read them; `analyze.js` and `coach.js` treating a variable passed `BYREF` to a procedure that writes it first as an out-parameter, not a read; `coach.js` no longer warning "growing bound" on a queue drained by `WHILE LENGTH(Q) > 0`; the soundness fixes above, with the new `argument` obligation kind (`vcgen`, and `views-analysis.js` for the course's pages); a run that stops on a FLOAT that is not finite (`interp`); and `logic`, `solver` and `vcgen` rewritten in TypeScript with the same behavior.
- **asm86:** what compiled Mojo uses beyond the course's subset (`interp.js`, about 630 lines): the BMI bit instructions, atomic read-modify-writes with `lock`, bit tests that write, `bswap`, `ud2`, the 128-bit AVX spellings, scalar double precision, and the double-width shifts; the parity flag (`interp`, `uops`, `views`); the floating-point units and their latencies (`uops`); a small change in `sandbox.js`; and in `hwviews.js` the whole core's arrows (`machineView`): only a lit edge carries an arrowhead, in its own color and on the end its data reached, each view has its own marker ids, and the result's edge into the registers starts at the port that produced it.
One addition to the pseudo language lives outside the engine, in app code: `DECLARE <name> : <type> <- <value>`, a declaration with its starting value on one line (Mojo's `var total: Int = 0`). `src/services/pseudo-study.ts` reads it by doing what the engine's `study()` does (parse, check, analyze, coach) with one step between parse and check: the `<- value` is blanked out of the line, columns kept, so the engine parses a plain DECLARE, and the value is parsed on its own and put in as an ordinary assignment on the same line. Every app reader of pseudo (`studyOf`, and the lingo, matho and Mojo translators) goes through it; pseudo-to-mojo writes the pair as one `var` line. The two-line form reads as before. If the course takes the form into its parser, this module can go.
One addition to the pseudo language lives outside the engine, in app code: `DECLARE <name> : <type> <- <value>`, a declaration with its starting value on one line (Mojo's `var total: Int = 0`). `src/services/pseudo-study.ts` reads it by doing what the engine's `study()` does (parse, check, analyze, coach) with one step between parse and check: the `<- value` is blanked out of the line, columns kept, so the engine parses a plain DECLARE, and the value is parsed on its own and put in as an ordinary assignment on the same line. Every app reader of pseudo (`studyOf`, and the lingo, matho and Mojo translators) goes through it; pseudo-to-mojo writes the pair as one `var` line. The two-line form reads as before. It can move into the engine's parser now that the engine lives here.
Deciding what happens to these (send them to the course, or make this repository the engines' home and have the course copy from here) is open; until then, `engines-diff` is how to see the gap.
## Giving the course the engines
## Updating from the course
1. `pixi run engines-diff <course>/static` shows what the course has not taken yet: for each module, the file the course would receive (`src/engines/` as written, `dist/engines/` for a TypeScript module) against the course's copy, whitespace ignored, and the files only one side has.
2. `pixi run engines-export <course>/static` builds and writes those files into the course's `static/js/`. A TypeScript module arrives as plain ES-module JavaScript, so the course's pages import it unchanged. Source maps are not copied.
3. In the course repository, review the diff and run its pages before committing.
4. For the stylesheets, follow [styles/course/README.md](../styles/course/README.md); they are still identical on both sides.
1. `pixi run engines-diff <course>/static` and read the list.
2. For a file that differs only in whitespace, or that changed only on the course side, copy the course's file over the one here.
3. For a file that differs in content on both sides, merge by hand, keeping the changes listed above.
4. `pixi run test`. The engines have no tests of their own in the course repository; the tests here (`tests/services.test.mjs`, `tests/corpus.test.mjs`, the curriculum and reference tests, `tests/asm-coverage.test.mjs`) are what catch a regression.
5. For the stylesheets, follow [styles/course/README.md](../styles/course/README.md).
A change someone makes in the course's copy is not picked up from there. Bring it here as a change with a test, then export.
+2 -1
View File
@@ -28,7 +28,8 @@ check = { cmd = "npx tsc -p tsconfig.json --noEmit", depends-on = ["install"], d
dev = { cmd = "python -m http.server 4173", depends-on = ["build"], description = "Build, then serve the app on http://localhost:4173" }
test = { cmd = "node --test tests/*.test.mjs", depends-on = ["build"], description = "Build, then run every test (the corpus pipeline's Python tests included)" }
check-pages = { cmd = "node scripts/check-pages.mjs {{ dir }}", args = ["dir"], depends-on = ["build"], description = "Run every program embedded in the Quarto pages under <dir> (quarto/README.md)" }
engines-diff = { cmd = "node scripts/engines-diff.mjs {{ static }}", args = ["static"], description = "Compare src/engines/ and styles/course/ with a course checkout's static/ directory (docs/engines.md)" }
engines-diff = { cmd = "node scripts/engines-diff.mjs {{ static }}", args = ["static"], depends-on = ["build"], description = "Show what a course checkout's static/ directory has not taken yet from src/engines/ and styles/course/ (docs/engines.md)" }
engines-export = { cmd = "node scripts/engines-export.mjs {{ static }}", args = ["static"], depends-on = ["build"], description = "Copy the engines this repository owns into a course checkout's static/js/, TypeScript modules as built (docs/engines.md)" }
# --- write content ---------------------------------------------------------
new-exercise = { cmd = "node scripts/new-exercise.mjs {{ id }}", args = ["id"], description = "Start an exercise: mojo/local/<id>.mojo and src/exercises/<id>.ts, registered in index.ts" }
+28 -19
View File
@@ -1,19 +1,23 @@
#!/usr/bin/env node
/**
* Compare the vendored course code with a checkout of the course site.
* Compare this repository's engines with a checkout of the course site.
*
* pixi run engines-diff ../course/static
*
* <static> is the course repository's static/ directory. For each engine
* module in src/engines/ (pseudo, asm86, shared) and each stylesheet in
* styles/course/, prints whether it is the same as the course's copy, differs
* only in whitespace and line endings, or differs in content (with the number
* of changed lines, whitespace ignored), and lists files only one side has.
* It changes nothing. docs/engines.md says what to do with the answer.
* <static> is the course repository's static/ directory. This repository owns
* the engines and the course copies from here (pixi run engines-export), so
* this shows what the course has not taken yet. For each engine module (pseudo,
* asm86, shared) it compares the file the course would receive (src/engines/
* as written, or dist/engines/ for a module written in TypeScript) with the
* course's copy, and each stylesheet in styles/course/ with the course's;
* prints whether they are the same, differ only in whitespace and line
* endings, or differ in content (with the number of changed lines, whitespace
* ignored), and lists files only one side has. It changes nothing.
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { engineModules, ENGINE_DIRS } from './engines-files.mjs';
const ROOT = fileURLToPath(new URL('..', import.meta.url));
const dir = process.argv[2];
@@ -22,9 +26,6 @@ if (!dir || !fs.existsSync(path.join(dir, 'js'))) {
process.exit(2);
}
const PAIRS = [
...['pseudo', 'asm86', 'shared'].map((f) => [`src/engines/${f}`, `js/${f}`, '.js']),
];
const CSS = { 'tokens.css': 'css/base/tokens.css', 'kit.css': 'css/widgets/kit.css', 'asm86.css': 'css/widgets/asm86.css' };
const squash = (s) => s.replace(/\r/g, '').split('\n').map((l) => l.replace(/\s+/g, ' ').trim()).filter(Boolean);
@@ -47,15 +48,23 @@ function compare(ours, theirs) {
}
const rows = [];
for (const [mine, course, ext] of PAIRS) {
const m = path.join(ROOT, mine), c = path.join(dir, course);
const names = new Set([...(fs.existsSync(m) ? fs.readdirSync(m) : []), ...(fs.existsSync(c) ? fs.readdirSync(c) : [])].filter((f) => f.endsWith(ext)));
for (const f of [...names].sort()) {
const a = path.join(m, f), b = path.join(c, f);
const label = `${mine}/${f}`;
if (!fs.existsSync(a)) rows.push([label, 'only in the course']);
else if (!fs.existsSync(b)) rows.push([label, 'only here']);
else { const r = compare(a, b); rows.push([label, r.kind === 'content' ? `differs: ${r.n} lines` : r.kind === 'whitespace' ? 'whitespace only' : 'same']); }
const strip = (t) => t.replace(/\n\/\/# sourceMappingURL=.*\n?$/, '\n');
const mine = engineModules(ROOT);
for (const m of mine) {
const theirs = path.join(dir, 'js', m.dir, m.file);
if (!fs.existsSync(m.ours)) rows.push([m.label, 'not built: run pixi run build']);
else if (!fs.existsSync(theirs)) rows.push([m.label, 'only here']);
else {
const a = strip(fs.readFileSync(m.ours, 'utf8')), b = fs.readFileSync(theirs, 'utf8');
const r = a === b ? { kind: 'same' } : (changed(squash(a), squash(b)) === 0 ? { kind: 'whitespace' } : { kind: 'content', n: changed(squash(a), squash(b)) });
rows.push([m.label, r.kind === 'content' ? `differs: ${r.n} lines` : r.kind === 'whitespace' ? 'whitespace only' : 'same']);
}
}
for (const d of ENGINE_DIRS) {
const c = path.join(dir, 'js', d);
if (!fs.existsSync(c)) continue;
for (const f of fs.readdirSync(c).filter((x) => x.endsWith('.js')).sort()) {
if (!mine.some((m) => m.dir === d && m.file === f)) rows.push([`js/${d}/${f}`, 'only in the course']);
}
}
for (const [f, course] of Object.entries(CSS)) {
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env node
/**
* Copy this repository's language engines into a checkout of the course site.
*
* pixi run engines-export ../course/static
*
* This repository owns the engines (docs/engines.md); the course copies from
* here. <static> is the course repository's static/ directory. For each
* engine directory (pseudo, asm86, shared) every module goes to
* <static>/js/<directory>/: a JavaScript module as written in src/engines/,
* a TypeScript one (logic, solver, vcgen) as built into dist/engines/, which
* is plain ES-module JavaScript the course's pages can import unchanged.
* Source maps are not copied. Prints what it wrote, and changes nothing else
* in the course. Review the course's diff before committing it there.
*/
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { engineModules } from './engines-files.mjs';
const ROOT = fileURLToPath(new URL('..', import.meta.url));
const dir = process.argv[2];
if (!dir || !fs.existsSync(path.join(dir, 'js'))) {
console.error('usage: pixi run engines-export <the course repository\'s static/ directory>');
process.exit(2);
}
let wrote = 0, same = 0, added = 0;
for (const m of engineModules(ROOT)) {
if (!fs.existsSync(m.ours)) {
console.error(`${m.label}: ${path.relative(ROOT, m.ours)} is missing; run pixi run build first`);
process.exit(1);
}
const target = path.join(dir, 'js', m.dir, m.file);
const text = fs.readFileSync(m.ours, 'utf8').replace(/\n\/\/# sourceMappingURL=.*\n?$/, '\n');
const before = fs.existsSync(target) ? fs.readFileSync(target, 'utf8') : null;
if (before === text) { same++; continue; }
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.writeFileSync(target, text);
if (before === null) { added++; console.log(`added js/${m.dir}/${m.file}${m.built ? ' (built from TypeScript)' : ''}`); }
else { wrote++; console.log(`updated js/${m.dir}/${m.file}${m.built ? ' (built from TypeScript)' : ''}`); }
}
console.log(`\n${wrote} updated, ${added} added, ${same} already the same`);
+25
View File
@@ -0,0 +1,25 @@
/**
* The engine modules this repository owns, and the file of each that the course receives:
* src/engines/<dir>/<name>.js as written, or dist/engines/<dir>/<name>.js for a module written in
* TypeScript. Shared by engines-diff and engines-export.
*/
import fs from 'node:fs';
import path from 'node:path';
export const ENGINE_DIRS = ['pseudo', 'asm86', 'shared'];
/** Every engine module: { dir, file, label, ours, built }. */
export function engineModules(root) {
const out = [];
for (const dir of ENGINE_DIRS) {
const src = path.join(root, 'src/engines', dir);
for (const f of fs.readdirSync(src).sort()) {
if (f.endsWith('.js')) out.push({ dir, file: f, label: `src/engines/${dir}/${f}`, ours: path.join(src, f), built: false });
else if (f.endsWith('.ts') && !f.endsWith('.d.ts')) {
const js = f.replace(/\.ts$/, '.js');
out.push({ dir, file: js, label: `src/engines/${dir}/${f}`, ours: path.join(root, 'dist/engines', dir, js), built: true });
}
}
}
return out;
}