Files
alexm 401084bab3
Test / gate (push) Failing after 19m0s
Initial commit
2026-11-12 22:23:54 -05:00

6.7 KiB

The model machine against compiled Mojo

"Open in Hardware" puts a function from the real compiler's listing on the model machine (src/engines/asm86), translated from godbolt's AT&T syntax by src/mojo/att.ts. The machine implements a teaching subset of x86-64, so the question is how much of what the Mojo compiler actually emits it can take. This is the census: every function in every listing the corpus holds (library/mojo/golden/*.json answered with asm), sent exactly as the button sends it, and what the assembler said. node scripts/asm-census.mjs reruns it (add --ops for every mnemonic); tests/asm-coverage.test.mjs holds the numbers as a ratchet, so a new corpus item that brings a new instruction fails the build rather than a student's session.

Where it stands

before now
distinct instructions in the listings 97 104
of which outside the model's set 38 6 (all 256-bit AVX2: vextracti128, vinserti128, vpblendvb, vperm2i128, vpalignr, vptest)
the user's own functions that assemble 3 of 7 (main never did) 17 of 17
standard-library and runtime functions that assemble — 698 of 733

The "before" column is the census of the listings the corpus held when this work began; the "now" column is today's census of today's listings, and tests/documents.test.mjs fails when it no longer matches node scripts/asm-census.mjs. Every listing is built for --target-cpu x86-64-v3 (the pin's build arguments, mojo-corpus.md), so it holds AVX2 at most and never AVX-512, whichever of godbolt's machines answered; tests/build-args.test.mjs fails on a listing with a zmm or mask register. Asked again under those arguments, every listing the corpus held came back with the same instructions; vptest and four modeled ones (adc, cmovle, jns, tzcnt) came with probes/godbolt-target-cpu, String's == at the default optimization level.

main() now steps up to its first call into print and stops there with "stopped: std_io_io_print: standard library, not on the model machine", which is the honest answer: the model has no standard library, and says exactly where the real program would enter it.

What was wrong, and what changed

The translator dropped size suffixes only for the mnemonics it knew. AT&T writes andnq, mulxq, shlxq, xaddq, cmpxchgq, btsq, bswapq; the Intel pane reads andn, mulx, … The BMI and atomic mnemonics were missing from the suffix table, so the machine was asked for an instruction called andnq, which exists nowhere. They are in the table now, and lock (a prefix) is passed through as a prefix rather than treated as a mnemonic with an operand called xaddq.

The machine lacked the instructions the optimizer likes. Added, each with a one-sentence gloss for the datapath view and the micro-op model told what it reads and writes:

  • BMI1/BMI2: andn, mulx, shlx, shrx, sarx, bzhi, blsr, blsi, blsmsk, rorx, bextr (pdep/pext are named as not modeled)
  • the atomics behind reference counts: xadd, cmpxchg, and the lock prefix, accepted on the read-modify-writes the ISA allows it on
  • bit tests that write: bts, btr, btc; bswap; cwd, cwde, cbw
  • ud2: the trap the compiler plants on paths that must never run; reaching it stops the run with "the processor raised an invalid-opcode fault"
  • the 128-bit AVX spellings of what the subset had: vmovdqu, vmovdqa, vmovups, vmovaps, vmovd, vmovq, vpxor, vxorps, vpor, vpand, vpandn, vpaddb, vpsubb, vpcmpeqb, vpmovmskb, vzeroupper; three-operand form (vop dst, a, b), sources untouched
  • lanes: vpinsr{b,w,d,q}, vpextr{b,w,d,q}, vpshufb, vpshufd, vpbroadcast{b,d}, vpmin/max{ub,ud}, vpcmpeq{w,d,q}, vpadd{w,d}, vpsubd, vpsubusb, vpsll/vpsrl{w,d,q}

A call into the standard library refused the whole listing. The assembler resolves every label up front, so one call std::io::print meant the function could not be stepped at all, even to the instruction before the call. The translator now appends a stub for every such symbol (label: then ud2 ; …: standard library, not on the model machine), so the function assembles, steps up to the call, and the stop says what it called.

Read-only strings were unreachable. lea rax, [rip + static_string_…] named a label that was data, not code, and the machine has no data directives. The translator now finds the .asciz behind each such label, places the bytes in memory (memory 0x6000 "…", octal and hex escapes decoded, from 0x6000 up), writes the address in the instruction's place, and hands the lines to the setup block, where the student can see them.

Scalar doubles, parity, shld (added)

  • addsd subsd mulsd divsd sqrtsd movsd cvtsi2sd ucomisd comisd and their v forms: IEEE binary64, round to nearest even, default NaN 0xFFF8000000000000, a NaN operand is quieted and the first operand wins. MXCSR exception flags are not modeled.
  • ucomisd/comisd set ZF/PF/CF as hardware does (unordered sets all three; OF, SF clear).
  • PF is a real flag now: set by logic, add/sub and shifts, cleared by popcnt, untouched by rotates. jp jnp jpe jpo, cmovp… and setp… follow.
  • shld/shrd on 32- and 64-bit operands; count masked to 5/6 bits; count 0 changes nothing, flags included.
  • Latencies: add/mul 4, div 14, sqrt 18 (unpipelined on port 0), convert 5, compare 3.

What stays outside, and why

  • 256-bit AVX2 (ymm): the model keeps sixteen 128-bit xmm registers and no ymm halves; the six instructions above are used only in that width. A ymm operand is refused with that sentence. Modeling it means widening every vector register and every lane helper; worth doing only when a user function needs it, and none in the corpus does.
  • Symbols the listing addresses but does not define: KGEN_… runtime globals, …@GOTPCREL function pointers, a .zero constant. The assembler names the symbol in its refusal.
  • Constant-pool blocks with a symbol in them (jump tables): not placed; the instruction that loads one is refused by name. Numeric .LCPI… blocks (.quad, .long, .double, .zero, …) are placed, 16-byte aligned from 0x6000, when the code addresses them, and .LCPI0_0+8(%rip) works.

Keeping it honest

The census runs against whatever the corpus holds: every stored answer that carries a listing (the probes and exercise programs whose header asks for mode: asm or mode: both). pixi run mojo-verify on a new such program adds its listing, and each addition either assembles or fails tests/asm-coverage.test.mjs with the instruction named. That test's ratchet (LIBRARY_AT_LEAST) is a floor under the table's library count; raise it when the machine learns more, never lower it.