A gate that does not depend on its machine, on a runner of its own
Test / gate (push) Canceled after 0s
Test / gate (push) Canceled after 0s
The gate first ran on the server that deploys the site: a slow container with no C compiler and no ICU. Ten tests failed there on correct code, and no Mojo program could be built at all. - The workflows ask for a runner labeled heavy, not the deploy server: the gate builds and runs some 300 Mojo programs and fuzzes. docs/ci.md says how to register one. With none registered a run waits. - The pinned environment has a C compiler on Linux (gcc, with its C library pinned to glibc 2.34): mojo build links with the C compiler it finds, and conda-forge's default links against a C library too old for Mojo's runtime. mojo-local reports a program that could not be linked as that, and does not keep the answer. - The pinned Dafny runs without ICU, and dafny-fetch fails when the Dafny it fetched does not start. - A test's time limit is processor time at the machine's pace (tests/helpers/pace.mjs). - The prover's time on the clock is the caller's: provePseudo and Workspace.proofBudgetMs take it, the Prove tab's 3 seconds stay the default, and the command line, library-check and the tests use none, so their answers are the counted budgets' alone. - The library pointer moves to its workflow change (e5ae090).
This commit is contained in:
31 files changed
+427
-104
No files matched your search
+9
-2
@@ -6,10 +6,14 @@
|
||||
*
|
||||
* pixi run cli check <file> what the language's own reader says
|
||||
* pixi run cli run <file> [--call C] [--input F] run it, print what it prints
|
||||
* pixi run cli prove <file> [--speco F] every obligation: proved, broken or open
|
||||
* pixi run cli prove <file> [--speco F] [--budget-ms N] every obligation: proved, broken or open
|
||||
* pixi run cli fuzz <file> [--seed N] [--budget N] the fuzzer's report per routine
|
||||
* pixi run cli translate <file.pseudo> --to mojo|lingo|matho|speco
|
||||
*
|
||||
* prove has no time limit here unless --budget-ms gives one (milliseconds for each routine): the prover's counted
|
||||
* budgets decide alone, so the answer is the same on every machine. The Prove tab of the app gives each routine 3
|
||||
* seconds as well, and on a slow machine can leave open what this proves; `--budget-ms 3000` asks for what it does.
|
||||
*
|
||||
* The language comes from the extension: .pseudo, .lingo, .matho (or .tex), .speco, .mojo, .asm (or .s). Add --json
|
||||
* for machine-readable output. Exit status: 0 when nothing is wrong, 1 when the command found a problem (an error,
|
||||
* a broken obligation, a failing input, a run that stopped), 2 for a usage error. Build first (`pixi run build`):
|
||||
@@ -23,7 +27,7 @@ const ROOT = fileURLToPath(new URL('..', import.meta.url));
|
||||
const dist = (p) => pathToFileURL(path.join(ROOT, 'dist', p)).href;
|
||||
const usage = (why) => {
|
||||
if (why) console.error(`pplus: ${why}`);
|
||||
console.error('usage: pplus <check|run|prove|fuzz|translate> <file> [--call C] [--input F] [--speco F] [--seed N] [--budget N] [--to LANG] [--json]');
|
||||
console.error('usage: pplus <check|run|prove|fuzz|translate> <file> [--call C] [--input F] [--speco F] [--seed N] [--budget N] [--budget-ms N] [--to LANG] [--json]');
|
||||
process.exit(2);
|
||||
};
|
||||
if (!fs.existsSync(path.join(ROOT, 'dist', 'common', 'workspace.js'))) usage('dist/ is missing; run `pixi run build` first');
|
||||
@@ -66,6 +70,9 @@ switch (cmd) {
|
||||
}
|
||||
case 'prove': {
|
||||
if (lang !== 'pseudo') usage('prove reads a .pseudo file (with --speco for a separate spec)');
|
||||
const clock = opt('budget-ms');
|
||||
if (clock !== undefined && !(Number(clock) > 0)) usage('--budget-ms takes a number of milliseconds above 0');
|
||||
ws.proofBudgetMs = clock === undefined ? Infinity : Number(clock);
|
||||
const p = ws.proof();
|
||||
if (!p) { out({ error: 'the pseudo does not parse' }, ['pplus: the pseudo does not parse; run `pplus check` first']); process.exit(1); }
|
||||
const { report } = p;
|
||||
|
||||
+16
-2
@@ -13,6 +13,12 @@ it into .tools/dafny-<version>/ (ignored by git). The zip is self-contained: it
|
||||
builds Dafny was tested with, so the solver is pinned too. To move to another Dafny, change VERSION and every hash in
|
||||
RELEASES in one commit, and expect every stored proof answer to be asked again.
|
||||
|
||||
One thing the zip does not carry is ICU, which .NET loads for culture-aware text; on a machine without it (a slim
|
||||
container image) Dafny aborts before it prints its version. Dafny is run here with DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
|
||||
(`env()` below), which tells .NET not to look for ICU: verifying a program compares no text by culture, so the
|
||||
verdicts are the same and the machine needs nothing installed. Whatever else runs the executable (scripts/library.mjs,
|
||||
tests/dafny.test.mjs) sets the same variable.
|
||||
|
||||
Standard library only, like the corpus pipeline.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
@@ -98,16 +104,24 @@ def fetch(quiet: bool = False) -> Path:
|
||||
return EXE
|
||||
|
||||
|
||||
def env() -> dict[str, str]:
|
||||
"""The environment Dafny runs in: this one, and .NET told not to look for ICU (the module's docstring)."""
|
||||
return {**os.environ, "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT": "1"}
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if argv[:1] == ["fetch"]:
|
||||
fetch()
|
||||
print(subprocess.run([str(EXE), "--version"], capture_output=True, text=True).stdout.strip())
|
||||
v = subprocess.run([str(EXE), "--version"], capture_output=True, text=True, env=env())
|
||||
if v.returncode != 0 or not v.stdout.startswith(VERSION):
|
||||
raise SystemExit(f"dafny: the fetched Dafny does not start here (exit {v.returncode}): {(v.stderr or v.stdout).strip()[:400]}")
|
||||
print(v.stdout.strip())
|
||||
return 0
|
||||
if argv[:1] == ["path"]:
|
||||
print(fetch(quiet=True))
|
||||
return 0
|
||||
exe = fetch()
|
||||
return subprocess.run([str(exe), *argv]).returncode
|
||||
return subprocess.run([str(exe), *argv], env=env()).returncode
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+4
-2
@@ -473,7 +473,8 @@ export async function checkLibrary(dir = LIB, only = null) {
|
||||
}
|
||||
rec.checks.pseudo = caseFail ? bad(caseFail) : ok(`${calls.length} case(s) run`);
|
||||
// prover
|
||||
const pr = svc.provePseudo(st);
|
||||
// no time on the clock: what is recorded is the counted budgets' answer, the same on every machine
|
||||
const pr = svc.provePseudo(st, { budgetMs: Infinity });
|
||||
const obs = pr.units.flatMap((u) => u.obligations);
|
||||
const broken = obs.filter((o) => o.status === 'broken');
|
||||
const notProved = obs.filter((o) => o.status !== 'proved');
|
||||
@@ -591,7 +592,8 @@ export async function checkLibrary(dir = LIB, only = null) {
|
||||
const src = r.proof.source;
|
||||
const flags = ['verify', '--resource-limit', '20000000'];
|
||||
if (dafny) {
|
||||
const p = spawnSync(dafny, [...flags, path.join(rdir, r.proof.file)], { encoding: 'utf8', timeout: 600000 });
|
||||
// without ICU on the machine .NET aborts unless told not to look for it (scripts/dafny.py)
|
||||
const p = spawnSync(dafny, [...flags, path.join(rdir, r.proof.file)], { encoding: 'utf8', timeout: 600000, env: { ...process.env, DOTNET_SYSTEM_GLOBALIZATION_INVARIANT: '1' } });
|
||||
const summary = /Dafny program verifier finished with (.*)/.exec(p.stdout || '')?.[1] || (p.stdout || p.stderr || '').trim().split('\n').pop();
|
||||
const pass = p.status === 0 && / 0 errors/.test(summary);
|
||||
rec.proof = pass ? ok(`Dafny ${dafnyVersion}: ${summary}`) : bad(`Dafny ${dafnyVersion}: ${summary}`);
|
||||
|
||||
+15
-4
@@ -176,6 +176,14 @@ const exec = (cmd, args, { cwd, input = '', timeout = 60000 } = {}) => new Promi
|
||||
const SIGNALS = os.constants.signals;
|
||||
const HAVE_PRLIMIT = spawnSync('prlimit', ['--version'], { encoding: 'utf8' }).status === 0;
|
||||
|
||||
/**
|
||||
* `mojo build` compiles a program and then has the C compiler it finds on the PATH link it. When that step fails (no
|
||||
* C compiler, or one that links against a C library older than Mojo's runtime needs) the compiler never said
|
||||
* anything about the program: the answer is the machine's, so it is not kept in the cache, not held against the
|
||||
* checker or godbolt, and reported as what it is.
|
||||
*/
|
||||
const NOT_LINKED = /unable to find suitable c compiler|failed to link executable/;
|
||||
|
||||
/** Build one program and run it: { compiled, buildStderr, executed, exitCode, stdout, stderr, timedOut }, in godbolt's shape. */
|
||||
async function buildAndRun(p, workdir) {
|
||||
const dir = fs.mkdtempSync(path.join(workdir, 'p-'));
|
||||
@@ -186,7 +194,8 @@ async function buildAndRun(p, workdir) {
|
||||
const extra = p.args.split(/\s+/).filter(Boolean);
|
||||
const b = await exec('mojo', ['build', ...extra, 'example.mojo', '-o', 'example'], { cwd: dir, timeout: 180000 });
|
||||
if (b.status !== 0 || !fs.existsSync(path.join(dir, 'example'))) {
|
||||
return { compiled: false, buildStderr: app(b.stderr || b.stdout), executed: false, exitCode: null, stdout: '', stderr: '', timedOut: b.timedOut };
|
||||
const buildStderr = app(b.stderr || b.stdout);
|
||||
return { compiled: false, buildStderr, executed: false, exitCode: null, stdout: '', stderr: '', timedOut: b.timedOut, ...(NOT_LINKED.test(buildStderr) ? { notLinked: true } : {}) };
|
||||
}
|
||||
// a trap must not become a core dump on someone's desktop: limit 1 tells the kernel not to pipe it
|
||||
const r = HAVE_PRLIMIT ? await exec('prlimit', ['--core=1:1', './example'], { cwd: dir, input: p.stdin, timeout: 30000 })
|
||||
@@ -300,7 +309,7 @@ const jobs = Math.max(1, Number(opt('jobs', '2')) || 2);
|
||||
const useCache = !flag('no-cache');
|
||||
|
||||
const findings = [];
|
||||
const counts = { programs: programs.length, compiled: 0, rejected: 0, cached: 0, interpreterRan: 0, interpreterDeclined: 0, godboltCompared: 0 };
|
||||
const counts = { programs: programs.length, compiled: 0, rejected: 0, notLinked: 0, cached: 0, interpreterRan: 0, interpreterDeclined: 0, godboltCompared: 0 };
|
||||
let next = 0;
|
||||
async function worker() {
|
||||
while (next < programs.length) {
|
||||
@@ -309,7 +318,8 @@ async function worker() {
|
||||
const file = path.join(cacheDir, `${key}.json`);
|
||||
let r = useCache && fs.existsSync(file) ? readJson(file) : null;
|
||||
if (r) { counts.cached++; for (const k of ['stdout', 'stderr', 'buildStderr']) r[k] = String(r[k] ?? '').replace(/(?:\/[^\s:"']+)?\/example\.mojo/g, '/app/example.mojo'); }
|
||||
else { r = await buildAndRun(p, workdir); if (useCache && !r.timedOut) fs.writeFileSync(file, `${JSON.stringify(r)}\n`); }
|
||||
else { r = await buildAndRun(p, workdir); if (useCache && !r.timedOut && !r.notLinked) fs.writeFileSync(file, `${JSON.stringify(r)}\n`); }
|
||||
if (r.notLinked) { counts.notLinked++; findings.push({ question: 'linker', id: p.id, detail: `not linked, so the compiler's answer is not known: ${firstError(r.buildStderr)}` }); continue; }
|
||||
if (r.compiled) counts.compiled++; else counts.rejected++;
|
||||
findings.push(...judge(p, r));
|
||||
if (p.interp === 'ran') counts.interpreterRan++;
|
||||
@@ -331,7 +341,8 @@ if (flag('json')) console.log(JSON.stringify({ compiler: `Mojo ${version}`, coun
|
||||
else {
|
||||
console.log(`Mojo ${version} on this machine: ${counts.programs} programs (${counts.compiled} compiled, ${counts.rejected} rejected, ${counts.cached} from the cache)`);
|
||||
console.log(`the interpreter ran ${counts.interpreterRan} of them and declined ${counts.interpreterDeclined}; ${counts.godboltCompared} compared with godbolt's stored answer`);
|
||||
for (const q of ['checker', 'interpreter', 'translator', 'godbolt']) {
|
||||
if (counts.notLinked) console.log(`\n${counts.notLinked} program(s) were compiled and could not be linked. That is this machine's C compiler, which mojo build calls to link, not Mojo's answer: under pixi it is the pinned gcc and C library (pixi.toml, docs/ci.md).`);
|
||||
for (const q of ['linker', 'checker', 'interpreter', 'translator', 'godbolt']) {
|
||||
const fs_ = unexpected.filter((f) => f.question === q);
|
||||
if (!fs_.length) continue;
|
||||
console.log(`\n${q}: ${fs_.length}`);
|
||||
|
||||
Reference in new issue
Block a user