8.0 KiB
CI
CI runs what a developer runs. Every workflow installs pixi, restores the environment pixi.lock pins (Node, Python, the Mojo compiler) with pixi install --locked, and runs a pixi task, so the gate on the runner and the gate on a laptop are the same command in the same environment. Five weeks once passed with main broken in two places and nobody told: CI then ran npm ci && npm test with whatever Node the host had, no pandoc, no Mojo compiler and no Dafny, and a missing tool skipped tests in silence.
| Workflow | When | Runs | Time here |
|---|---|---|---|
.gitea/workflows/test.yml |
every push and pull request | pixi run ci, then checks that nothing the gate regenerates differs from what is committed |
4 minutes here, 2.5 with mojo-local's answers cached |
.gitea/workflows/nightly.yml |
every day at 06:17 UTC, and by hand | pixi run nightly |
12 minutes here |
pplus-library's .gitea/workflows/test.yml |
every push and pull request to the library | core's default branch with that library commit in library/, then pixi run ci (below) |
as core's |
pixi run ci is test (every test), mojo-status-strict (no Mojo program of ours fails, from godbolt's stored answers), library-check (every library routine at every level, Dafny included), mojo-local (every Mojo program core holds or generates, built and run by the pinned compiler) and ci-fuzz (below). pixi run nightly is mojo-status-strict and the fuzzers for longer.
The runner
The workflows are written for Gitea Actions on a runner in host mode (no container: steps run on the host's shell); pplus-web's docs/deploy.md has the setup. They use no container: and no services:, and each needs one change to run on GitHub Actions: runs-on: ubuntu-latest in place of the host's label.
- pixi.
PIXI_VERSIONat the top of each workflow is the pixi that reads the lock. The first step uses the host's pixi when it is that version; otherwise it downloads that release from GitHub into$PPLUS_CI_CACHE/pixi-<version>/(default~/.cache/pplus-ci), checks its SHA-256 (written in the workflow), and puts it on the PATH. No sudo; a host runner fetches it once. To move pixi: changePIXI_VERSIONand both hashes in every workflow, in one commit, and check thatpixi install --lockedstill accepts the lock. - The environment.
pixi install --lockedfails ifpixi.tomlandpixi.lockdisagree, so the runner never resolves anything new. pixi's package cache stays on the host, so a host runner downloads each package once. .tools/. The pinned Dafny (pixi run dafny-fetch, about 150 MB) andmojo-local's answers live in.tools/; in CI.toolsis a link to$PPLUS_CI_CACHE/core-tools, so neither is fetched or rebuilt on every push.mojo-localkeys an answer by the compiler's version, the arguments, the input and the whole source, so a cached answer is the compiler's answer for exactly that program. Two jobs that share the cache at once could write the same answer file together; the runner takes one job at a time unless itscapacityis raised.CI. Each workflow setsCI=true(Gitea and GitHub set it too). Under it a test that would skip for a missing tool fails instead, because a skip in CI is how a gap hides: python3 (tests/mojo.test.mjs,tests/dafny.test.mjs,tests/library.test.mjs), the fetched Dafny (tests/dafny.test.mjs), a git checkout to read the library pointer from (tests/chain.test.mjs), and in pplus-web pandoc for the Quarto tests.- What the host needs besides pixi: git, curl, tar,
sha256sum, bash, and Node foractions/checkoutitself; a C compiler (cc) for the C referenceslibrary-checkbuilds. Withoutccthose checks are recorded as skipped,library/status.jsonthen differs from the committed one, and the last step oftest.ymlfails and says so.
The fuzzers' seeds
pixi run ci-fuzz runs mojo-fuzz (25 programs of expressions), mojo-fuzz --statements (25 whole programs) and prove-fuzz (300 programs) on two seeds each: seed 1, so a failure that comes back is the same failure, and a seed that moves, so every run looks somewhere new: the CI run's number (GITHUB_RUN_NUMBER, which Gitea sets too) or, outside CI, today's date in UTC as YYYYMMDD. pixi run nightly runs all three on four seeds, the date's number and the three numbers after it, with 100, 100 and 3,000 programs. The size matters: the statement mode's first run at 100 programs a seed found a translator fault (an argument evaluated once per pass of a loop) that many seeds of 40 had not. Every round prints its seed and the command that reproduces it:
== prove-fuzz seed 20261113 (reproduce: node scripts/prove-fuzz.mjs --seed 20261113 --programs 3000)
and the run ends with a summary, one line per round, clean, FOUND or FAILED.
When CI or the nightly run fails
- Find the round. The summary at the end of the log names it, with its command.
- Reproduce it here. Run the same command as a pixi task, with the same seed and size:
pixi run prove-fuzz --seed 20261113 --programs 3000, orpixi run mojo-fuzz --seed 20261113 --programs 100. The same seed makes the same programs on every machine. The prover has a time budget for each routine (--budget-ms, 1,500 ms), so a slower machine may prove less than the runner did, never more; if a prove-fuzz finding does not come back, raise the budget. - A prove-fuzz finding is a soundness bug: an obligation marked proved that a run breaks. The report names the program's number and prints the smallest program that still shows it, with the call and the run's seed.
pixi run prove-fuzz --seed S --only-program N --keep work/writes it as a regression program with its// CALL:and// MUST-NOT-PROVE:header. Check it by hand (pixi run cli prove <file>says proved; running it with that call breaks it), put it inlibrary/pseudo/prover/, watchtests/prover-soundness.test.mjsfail, then fix the rule (AGENTS.md, rule 5; engines.md).--without <feature>switches off one construct at a time to see which one the finding needs. - A mojo-fuzz finding is printed once per shape with the smallest expression that shows it; its header (
scripts/mojo/fuzz.mjs) says what each kind means. A translator fix is pinned intests/sync-mojo.test.mjs; a finding that is the compiler's own is listed there as such, with what was observed. - mojo-status-strict failing means a program we wrote fails against godbolt's stored answers: see mojo-corpus.md.
- Anything else in
pixi run cifails the same way on a developer's machine: runpixi run ciand read the first failure.
A failure opens no issue and sends nothing: the run is red on the repository's Actions page, for whoever looks. Look after a push, and look at the nightly runs at least weekly.
The pointer to the library
Core pins pplus-library as library/ by commit. tests/chain.test.mjs fails when that commit is not in the library checkout, or the checkout is behind it, and says which command fixes it (git submodule update --init). The whole chain (web pins core, core pins library; each pointer pushed to its branch; the generated files current; Mojo pinned alike) is checked by pplus-web's release-check task, and moved in order by its release-bump task (pplus-web's docs/deploy.md).
pplus-library has no tests of its own: its checks are core's. Its workflow checks out core's default branch (or the ref in the repository variable CORE_REF), puts the library commit being tested where core's submodule goes, moves core's pointer to it in that throwaway checkout and regenerates what core derives from the library (library-emit, mojo-emit), and runs pixi run ci. A green run shows that core's default branch, given this library and what core regenerates from it, passes core's whole gate. It does not show that core pins this commit (that is a later commit in core), that the site works with it, or anything about a core branch other than the one tested; and a red run can be core's own, so look at core's latest run first.