244 lines
21 KiB
Markdown
244 lines
21 KiB
Markdown
# Programming+ for Quarto
|
||
|
||
A Quarto extension that puts [Programming+](https://pplus.scient.ing) in a page. A program lives in a file beside the `.qmd`, and a shortcode says where it appears and how. The app's **With Quarto** page (`#/quarto`) is a gallery of every shortcode beside what it makes. [`example/`](example/) is a site that uses every shortcode on real pages (`pixi run quarto-example`): rendered views, the machine, and live views.
|
||
|
||
There are two kinds of shortcode (the machine's are a family of the first):
|
||
|
||
- **Rendered with the page.** While Quarto renders, the extension runs the program through Programming+'s own engines and writes what its views draw into the page. The result has no iframe and needs no network, it prints, and a program that stops printing its `EXPECT` lines is reported on the spot. These need Node 20 or later and a built Programming+ on the machine that renders the page (see [The engine](#the-engine)).
|
||
- **Live.** The app itself, in an `<iframe>`, with the program carried in its URL. The reader edits, steps and runs it. These need nothing at render time, but readers need the network, and the app's host has to allow your site to frame it.
|
||
|
||
| Shortcode | Kind | Shows |
|
||
|---|---|---|
|
||
| `pplus-run` | rendered | what the program prints and returns, checked against its `EXPECT` lines; a Mojo file runs on the real `mojo` when one is on the PATH |
|
||
| `pplus-steps` | rendered | the listing beside a stepper: every step, its line, what it wrote, the values after it, and what has been printed (a small script moves through a run recorded at render time) |
|
||
| `pplus-table` | rendered | the table of values: a column per step that changed something, a row per name |
|
||
| `pplus-flow` | rendered | the flowchart, each box shaded by how often its line ran |
|
||
| `pplus-dataflow` | rendered | the data-flow graph: which names feed which |
|
||
| `pplus-lifetimes` | rendered | each Mojo value made, copied, moved and destroyed |
|
||
| `pplus-proof` | rendered | every claim of the specification: proved, broken or open |
|
||
| `pplus-tests` | rendered | the fuzzer's report for every routine |
|
||
| `pplus-check` | rendered | what the checker finds in each file, on the file's own lines |
|
||
| `pplus-translate` | rendered | the procedure in every notation, with Mojo generated from the pseudo and labeled so (as Quarto tabs, columns, or the app's panes) |
|
||
| `pplus-machine` | rendered | the x86-64 run on the model machine: instructions, cycles, cache misses, branches |
|
||
| `pplus-view` | rendered | any of the above, by `view=` |
|
||
| `pplus-hw` | machine | the Hardware area's board over an x86-64 run: the listing beside a preset's cards (`preset=regs`, `caches`, `pipeline`, `chip`) or the cards you name (`views=`), each drawn at every step, with a stepper |
|
||
| `pplus-hw-<view>` | machine | one card of the board: `code`, `explain`, `registers`, `flags`, `address`, `memory`, `stack`, `dataflow`, `alu`, `branches`, `vectors`, `lines`, `cache`, `hierarchy`, `paging`, `pipeline`, `uops`, `timeline`, `bottleneck`, `counters`, `chip`, `core`, `encoding` (see [The machine](#the-machine)) |
|
||
| `pplus-playground` | live | the whole playground |
|
||
| `pplus-embed` | live | one view of it (`view=trace`, `table`, `flow`, `data`, `life`, `problems`, `proof`, `tests`, `compiler`, `asm`, or a pane) |
|
||
| `pplus` | live | the original shortcode: a live embed, a listing, or both (`show=`); pages written for it keep working |
|
||
| `pplus-listing` | text | the files as highlighted code blocks; a `.tex` file typeset as equations |
|
||
| `pplus-tabs` | text | the files as a tabset, one tab per notation |
|
||
| `pplus-link` | text | a link that opens the program in the app |
|
||
|
||
Every rendered view also takes `live="click"`. That adds a **make it live** button, which puts the live view in its place.
|
||
|
||
## Install
|
||
|
||
1. Copy `quarto/_extensions/pplus` into your site's `_extensions/`.
|
||
2. Name the syntax definitions it ships, so listings of lingo, pseudo and Mojo are highlighted. Then, for the rendered views, name the engine (next section):
|
||
|
||
```yaml
|
||
# _quarto.yml
|
||
syntax-definitions:
|
||
- _extensions/pplus/syntax/pseudo.xml
|
||
- _extensions/pplus/syntax/lingo.xml
|
||
- _extensions/pplus/syntax/mojo.xml
|
||
|
||
pplus-engine: ../programming-plus # a built Programming+ (its folder, or its dist/)
|
||
pplus-url: "https://pplus.scient.ing" # where the live views point (this is the default)
|
||
pplus-strict: true # optional: a view that cannot be rendered, or a run that does not print its EXPECT lines, stops the render
|
||
```
|
||
|
||
`mojo.xml` is generated from the same words as the Mojo pane's highlighter (the compiler's own token table, at the pinned Mojo version). A printed listing therefore colors `mut`, `deinit`, the transfer `^`, and removed words such as `fn` exactly as the pane does. If the project already has its own `pseudo.xml`, keep that and add only `lingo.xml`.
|
||
|
||
The stylesheets (`pplus-views.css`, `pplus.css`) and the script (`pplus.js`) are attached to every page that uses a shortcode.
|
||
|
||
### The engine
|
||
|
||
A rendered view runs `node _extensions/pplus/render.mjs <engine>` once per shortcode. The extension looks for a built Programming+ in this order:
|
||
|
||
1. `pplus-engine` in the project's or the page's metadata, relative to the project. Set it to `none` to render nothing with the page: each rendered view then shows its listing and the live view.
|
||
2. The `PPLUS_ENGINE` environment variable.
|
||
3. `_extensions/pplus/engine/`. `pixi run quarto-engine <your site>/_extensions/pplus` copies one there, so the site needs no checkout beside it. Run it again after rebuilding.
|
||
4. The repository the extension sits in, when it is the one in this repository (`quarto/_extensions/pplus`). This is how `example/` renders.
|
||
|
||
`pplus-node` (or `PPLUS_NODE`) names the node to use. Results are cached in `.quarto/pplus/`, keyed by the request and the engine's build. Set `pplus-cache: false` to render every view again. With no engine found, the page still renders: each rendered view becomes its listing and the live view, and Quarto prints one warning.
|
||
|
||
A Mojo file asked to run (`pplus-run`) is also given to the real compiler when `mojo` is on the PATH (inside `pixi`, the pinned one). The view then shows that compiler's output and names it. Without one, Programming+'s interpreter runs it, and the view says so. `compiler="off"` always uses the interpreter, and `compiler="on"` fails without a compiler.
|
||
|
||
### Live views on your site
|
||
|
||
The live views and **make it live** load the app in an iframe. The app's host decides which sites may frame it: `pplus.scient.ing` allows `scient.ing` and its subdomains, and a local `quarto preview` ([docs/deploy.md](../docs/deploy.md)). For any other site, point `pplus-url` at a copy of the app you host, or add the site to the host's `frame-ancestors`. The rendered views need neither.
|
||
|
||
## Use
|
||
|
||
Programs are files next to the page: `count-base.lingo`, `match.pseudo`, `spec.speco`, `mean.tex`, `count_base.mojo`, `two-pointers.asm` (with `two-pointers.setup`). Paths are relative to the page.
|
||
|
||
```markdown
|
||
{{< pplus-run pseudo="count-base.pseudo" code="1" >}} the program, then what it printed
|
||
{{< pplus-steps lingo="count-base.lingo" t="6" >}} a stepper, parked at step 6
|
||
{{< pplus-steps asm="two.asm" setup="two.setup" max="40" >}} the machine's registers, step by step
|
||
{{< pplus-steps pseudo="count-base.pseudo" follow="Total,I" >}} a stepper over two names only
|
||
{{< pplus-table pseudo="count-base.pseudo" >}} the table of values
|
||
{{< pplus-flow pseudo="binary-search.pseudo" live="click" >}} the flowchart, and a button to go live
|
||
{{< pplus-dataflow pseudo="count-base.pseudo" >}}
|
||
{{< pplus-lifetimes mojo="lifecycle.mojo" >}}
|
||
{{< pplus-proof pseudo="count-base.pseudo" speco="count-base.speco" >}}
|
||
{{< pplus-tests pseudo="match-off-by-one.pseudo" >}}
|
||
{{< pplus-check pseudo="five-bugs.pseudo" code="1" numbers="1" >}}
|
||
{{< pplus-translate matho="mean.tex" >}} tabs: matho, lingo, pseudo, Mojo
|
||
{{< pplus-translate lingo="x.lingo" layout="grid" to="pseudo,mojo" >}}
|
||
{{< pplus-machine asm="two.asm" setup="two.setup" >}}
|
||
{{< pplus-view view="flow" pseudo="x.pseudo" >}}
|
||
|
||
{{< pplus-playground lingo="x.lingo" pseudo="x.pseudo" panel="flow" >}}
|
||
{{< pplus-embed pseudo="x.pseudo" view="trace" t="9" >}}
|
||
{{< pplus-listing pseudo="x.pseudo" numbers="1" >}}
|
||
{{< pplus-tabs lingo="x.lingo" pseudo="x.pseudo" mojo="x.mojo" >}}
|
||
{{< pplus-link pseudo="x.pseudo" text="Open it in Programming+" >}}
|
||
```
|
||
|
||
Every option is documented at the top of `_extensions/pplus/pplus.lua`. These work with every shortcode:
|
||
|
||
| option | does |
|
||
|---|---|
|
||
| `lingo`, `matho`, `pseudo`, `speco`, `mojo`, `asm` | files beside the page whose text fills that pane |
|
||
| `setup` | a file holding the x86-64 pane's setup block (registers, memory) |
|
||
| `lang` | the pane that runs (default: the first of lingo, matho, pseudo, mojo, asm given) |
|
||
| `call`, `input` | the call to run, and lines INPUT reads separated by `\|` (default: the file's own `// CALL:` and `// INPUT:` lines) |
|
||
| `machine` | the x86-64 machine model preset |
|
||
| `t` | the step the stepper starts on (live: is parked at) |
|
||
| `title`, `caption` | a title over the view, a sentence under it |
|
||
| `collapse` | `1` folds it into a block titled with `title` |
|
||
| `area` | `bridge` (default), `dev` or `hw`: where "open in the playground" goes, and the area the live playground opens in |
|
||
|
||
And these work with the rendered ones:
|
||
|
||
| option | does |
|
||
|---|---|
|
||
| `code` | `1` prints the program inside the view's frame, above it |
|
||
| `numbers` | `1` numbers the listing's lines |
|
||
| `live` | `click`: a button that puts the live view in place of the rendered one |
|
||
| `height` | the live frame's height when it is put in place (default 520; 720 for translate and the machine's views, which open the whole playground) |
|
||
| `follow` | steps: the variables to show, comma-separated |
|
||
| `max` | steps: the most steps kept (default 400; from 10 to 5000) |
|
||
| `to`, `layout` | translate: the notations to show, comma-separated; `tabs` (default), `columns` or `grid` |
|
||
| `compiler` | run, for a Mojo file: `auto`, `on` or `off` |
|
||
| `column` | Quarto's column for the view (`page`, `screen`, `margin`, ...) |
|
||
|
||
In a format other than HTML (a PDF, a Word file), a rendered view becomes Markdown. The run is printed as a code block; the steps, the table of values and the machine's counters as tables; the proof, tests and checker's findings as lists; and translate as one listing per notation. The flowchart, the data flow and the lifetimes say they are drawn in the HTML version. A view of the machine gives the registers the run has written by step `t` and names the cards drawn in the HTML version. Each view except translate is followed by a link to the app.
|
||
|
||
### The live views, arranged
|
||
|
||
`pplus-playground` puts the full playground in the frame: every pane, the stepper, the variables, every inspect tab, the dock, Run (Compile & run on the Mojo pane), Share and the theme switch. `pplus-embed` (and `pplus`) shows one view with the code it reads. A reader's edits are kept in their browser for that example, and Reset brings the page's version back. The options arrange it the way the playground's own controls do:
|
||
|
||
```markdown
|
||
{{< pplus-playground mojo="demo.mojo" asm="demo.asm" setup="demo.setup" area="hw" >}}
|
||
{{< pplus-playground problem="demo" panel="compiler" build="default" dock="open" >}}
|
||
{{< pplus-playground mojo="demo.mojo" area="dev" editor="0.6" dockheight="0.75" panel="life" >}}
|
||
{{< pplus-embed mojo="demo.mojo" view="compiler" >}}
|
||
{{< pplus-embed pseudo="x.pseudo" view="flow" height="auto" showcall="0" >}}
|
||
{{< pplus pseudo="match.pseudo" show="both" title="MATCH" >}} the listing, then the embed
|
||
```
|
||
|
||
| option | does |
|
||
|---|---|
|
||
| `view` | `pplus-embed`, `pplus`: the view (default `trace`) |
|
||
| `panel` | the tab it opens on: `trace table flow data life proof tests` beside the editor, or `output problems compiler asm setup` in the dock |
|
||
| `build` | `O0` (as written) or `default` (as `mojo build` does) |
|
||
| `dock`, `docktab` | `open`/`closed`, and which dock tab |
|
||
| `editor`, `side`, `dockheight` | the playground's splits, as fractions |
|
||
| `theme` | `light`/`dark` (default: follows the page's switch) |
|
||
| `ro` | `1`: read-only, and nothing is saved |
|
||
| `showcall` | `0` hides the run box above a flow, data or table view |
|
||
| `split` | the share of a fixed-height frame the code takes above the panel (default 0.7) |
|
||
| `height` | the frame's height in px (default 520, or 720 for the playground), or `auto` to fit its content (not for the playground, which fills its frame) |
|
||
| `problem` | a catalog problem's id instead of files |
|
||
| `state` | a state from the playground's Share dialog |
|
||
| `show`, `mode`, `text` | `pplus`: `listing` or `both`; `mode="link"` for only a link, with `text` |
|
||
|
||
Or arrange it in the playground itself and paste the result: **Share** → *embed which view* → **Quarto** gives a `{{< pplus-playground state="…" >}}` or `{{< pplus-embed state="…" view="…" >}}` shortcode for exactly what you see, layout and step included.
|
||
|
||
### A Mojo program from `core/library/mojo/local/`, with its assembly
|
||
|
||
```bash
|
||
pixi run --manifest-path core/pixi.toml play demo # verify it, and store the compiler's listing
|
||
pixi run quarto-export demo ../site/programmingplus/demo
|
||
```
|
||
|
||
This writes `demo.mojo`, `demo.asm` (the x86-64 pane's program) and `demo.setup` next to the page, and prints a `pplus-playground` shortcode that embeds them, and `pplus-run` and `pplus-lifetimes` shortcodes that render the program with the page. For a program with no stored listing it writes only `demo.mojo` and says so. The `# origin:`/`# probe:`/... header of a `core/library/mojo/local/` file is left out of the pane and the listing. Its `# expect-stdout:` lines count as `EXPECT` lines.
|
||
|
||
## The machine
|
||
|
||
`pplus-hw` and the `pplus-hw-<view>` shortcodes show what the Hardware area's board shows, over an x86-64 file beside the page (`asm=`, with its `setup=` for the registers and memory). While Quarto renders, the extension runs the program on the model machine and draws each card at every step. The cards are the course's own views (pplus-core's `core/src/engines/asm86/`) and, for `lines` and `counters`, the app's own, mounted as the board mounts them. The page keeps each card's distinct drawings and which one each step shows, compressed (deflate: for the 97 steps of the example's two-pointers run most cards take 1 to 12 KB, the pipeline about 30 KB and the out-of-order timeline about 80 KB), and `pplus.js` steps through them. Nothing runs in the reader's browser beyond that, and the step the page opens on (`t=`) is in the markup, so the page prints and reads without script.
|
||
|
||
```markdown
|
||
{{< pplus-hw asm="two-pointers.asm" setup="two-pointers.setup" >}} registers, flags, address, memory, stack
|
||
{{< pplus-hw asm="two.asm" setup="two.setup" preset="caches" t="12" >}} the caches, opening on step 12
|
||
{{< pplus-hw asm="two.asm" setup="two.setup" views="explain,registers,encoding" >}} the cards you name, in order
|
||
{{< pplus-hw-pipeline asm="two.asm" setup="two.setup" listing="0" max="40" >}} one card, no listing, at most 40 steps drawn
|
||
{{< pplus-hw-paging asm="two.asm" setup="two.setup" >}} virtual memory is switched on for it
|
||
{{< pplus-hw-encoding asm="two.asm" setup="two.setup" t="9" stepper="0" >}} one step, for a printed figure
|
||
```
|
||
|
||
| view | shows |
|
||
|---|---|
|
||
| `code` | the machine code: each instruction with its address and bytes |
|
||
| `explain` | the last instruction in words |
|
||
| `registers`, `flags`, `address` | the registers the program touches (with every narrower name it uses), the flags, base + index × scale + displacement |
|
||
| `memory`, `stack` | each region byte by byte with the load or store marked, and the stack's slots with rsp and rbp |
|
||
| `dataflow`, `alu` | where the last instruction's values came from and went, and the ALU at bit level |
|
||
| `branches`, `vectors` | each branch and whether the predictor guessed right, and the xmm registers lane by lane |
|
||
| `lines`, `cache`, `hierarchy` | the cache line each access touched, the L1 data cache as sets and ways, the path through L1, L2, L3 and memory |
|
||
| `paging` | the page walk and the TLB |
|
||
| `pipeline`, `uops`, `timeline`, `bottleneck` | the five-stage pipeline, micro-ops and ports, the out-of-order timeline, what limits the speed |
|
||
| `counters` | the run's totals, in the style of `perf stat` (the same at every step) |
|
||
| `chip`, `core`, `encoding` | the whole core with the parts the last instruction used lit up, inside the out-of-order core, the instruction's bytes field by field |
|
||
|
||
| option | does |
|
||
|---|---|
|
||
| `preset`, `views` | `pplus-hw`: the board's arrangement (`regs`, the default, `caches`, `pipeline`, `chip`), or the views, comma-separated |
|
||
| `machine` | the machine model: `teaching` (default), `realistic`, `ideal`; `+paging` for virtual memory |
|
||
| `t` | the step the page opens on |
|
||
| `max` | the most steps drawn (default 400); a longer run is sampled evenly, its last step always included |
|
||
| `listing` | `0` leaves out the x86-64 listing beside the cards |
|
||
| `stepper` | `0` draws only step `t` |
|
||
| `column` | Quarto's column for the figure; `pplus-hw` is `page`-wide by default, one card as wide as the text |
|
||
|
||
The machine's stylesheets are the course's own (`course/tokens.css`, `kit.css`, `asm86.css`, written for Quarto pages) and the board's adjustments to them (`pplus-hw.css`); they are attached only to a page that shows the machine. `pixi run quarto-css` copies the three course files from `styles/course/` and writes `pplus-hw.css` from `styles/course/bridge.css`.
|
||
|
||
For a Mojo program, `pixi run quarto-export <name> <folder>` writes the compiler's x86-64 for it and a setup for its arguments beside the page; [example/machine.qmd](example/machine.qmd) shows the demo's `cell` function this way.
|
||
|
||
## The header of a program file
|
||
|
||
A program file may start with lines the checker reads. The views honor them, and no listing, pane or editor shows them:
|
||
|
||
```
|
||
// CALL: MATCH("ATT", "GATTACA") the call to run
|
||
// INPUT: 4000 a line INPUT reads (repeatable)
|
||
// EXPECT: MATCH returned [1] a line of output, in order (repeatable); pplus-run checks it
|
||
// ERRORS: 2 pseudo, LaTeX: meant to be refused with this many errors (Mojo: meant not to compile)
|
||
// QUESTION: range-ends lingo: the reader is meant to ask this question
|
||
// STOPS: index the run is meant to stop with this kind of error
|
||
```
|
||
|
||
A `.tex` file uses `%` in place of `//`. A Mojo file's header is its leading block of `# key: value` lines (`# EXPECT: 3`, or the corpus's `# expect-stdout: 3`). Only that block is left out of a listing, so keep every header line in it.
|
||
|
||
## Check
|
||
|
||
```bash
|
||
pixi run check-pages /path/to/site
|
||
```
|
||
|
||
(It builds first. To check several folders at once, run `node scripts/check-pages.mjs <folder> [...]` after `pixi run build`.)
|
||
|
||
Every `.lingo`, `.pseudo` and `.tex` file under the folders (names that start with `_` or `.` are skipped) is parsed and checked, and lingo and LaTeX are lowered to pseudo. A file with `// CALL:` or `// EXPECT:` lines is run with its `// CALL:` and `// INPUT:` lines and compared with its `// EXPECT:` lines; a lingo file with `// QUESTION:` lines is only held to asking them. A `.speco` file is only checked to exist where a shortcode names it. A `.mojo` file is checked for syntax the pinned Mojo removed; when a `mojo` compiler is on the PATH it is also run, with `mojo run` and the pin's build arguments, and must exit 0 and print its `EXPECT` lines (or, with `ERRORS`, not compile). Every shortcode must be one the extension has, and every file it names must exist. The exit status is 1 on any problem. A rendered page checks itself as well: with `pplus-strict: true`, `quarto render` stops at the first view that cannot be rendered (no engine, or the engine fails) or whose run does not print its `EXPECT` lines, which includes a program with `EXPECT` lines that does not run at all. A program without `EXPECT` lines that does not run is a warning.
|
||
|
||
## How it is built
|
||
|
||
- `pplus.lua` holds the shortcodes. It reads the files, writes the live views' URLs (the state as `~` + base64url JSON, which the app reads without decompression), and asks `render.mjs` for the rendered views.
|
||
- `render.mjs` loads the engine's `dist/quarto/render.js`. That is `src/quarto/render.ts` in Programming+: it builds the same `Session` the app does and draws the app's own views (`src/ui/views/`) into the small document of `src/quarto/dom.ts`, then returns their HTML and a Markdown form.
|
||
- `pplus-views.css` holds the app's rules for those views, scoped to `.pplus-static`, and is generated by `pixi run quarto-css`, which also copies the course's stylesheets into `course/`, writes `pplus-hw.css`, and copies core's `syntax/mojo.xml`. `pplus.css` holds the extension's own rules (the frames, the run, the stepper, the translations).
|
||
- `src/quarto/machine.ts` draws the machine's cards at every step for `pplus-hw`, with the board's own `mountView` and presets (`src/ui/hw/board.ts`, `src/core/hardware.ts`); `render.mjs` compresses the drawings.
|
||
- `pplus.js` moves the steppers (the machine's too, decompressing its drawings with the browser's `DecompressionStream`), puts a live view in place on **make it live**, sizes `height="auto"` frames (to what the frame asks for, between 120 and 4000 pixels: a frame cannot make the page longer than that), and passes the page's light/dark switch to live frames.
|