Files
pplus-web/docs/deploy.md
T
alexm 5202c96318 The home page shows at once
It was blank until every module had loaded and the page's examples had
been run, a fuzzer's report among them (about a second of work): first
paint came 1.3 seconds into a repeat visit and about 3 into a first one.

- The assembler draws the home page once, in Node, with the app's own
  homeView, and puts it inside #app. The browser paints it with the
  HTML and the stylesheet.
- When the app has loaded it draws the page again, with its controls,
  and puts it in the prerendered one's place; the fuzzer's reports and
  the proof's counts are copied from the prerendered page, not worked
  out again.
- The prerendered page shows only on the home address: the page's first
  inline script marks it before the first paint, and without the mark it
  stays hidden, so an embed or a problem's address never shows it. That
  script changed, and its hash in docs/Caddyfile with it.

Served locally, first paint is at 0.1 to 0.2 seconds and the page with
its controls at 0.2 to 0.3; both were at 1.2 to 1.4.
2026-11-13 07:28:38 -05:00

57 lines
13 KiB
Markdown

# Deploying
The site builds against pplus-core, the git submodule `core/`, at the commit this repository pins. A checkout needs it (`git clone --recurse-submodules`, or `git submodule update --init`); `npm run build` builds core first. To take a newer core: `git -C core pull` (or check out a tag), `pixi run test`, and commit the new `core` pointer with what it changes on this side.
`pixi run site` tests the app and assembles `site/`; `pixi run deploy [slug] [user@host]` does that and publishes it. CI (`.gitea/workflows/deploy.yml`) runs `pixi run ci` (the site's tests and core's `mojo-status-strict`) on every push and pull request, and on a push to `main` checks the chain of pointers (`pixi run release-check`), assembles and publishes.
The site is seven things: `index.html`, `styles.css`, `fonts/` (IBM Plex, served by the site so a visit asks no font host for anything; `fonts/README.md` says where the files came from), `styles/` (the course's widget styles, which the Hardware area's machine views use), `dist/`, `core/` (pplus-core's built `core/dist/`, which `index.html`'s import map names as `#core/`), and the Quarto extension (`quarto/_extensions/pplus`, without an engine copy), which the With Quarto page loads (`pplus.css`, `pplus.js`) and links to file by file. `scripts/assemble.sh` copies them into `site/` with a `VERSION` file holding the commit, a `build.json` naming the three commits the site was built from (below), and the license and notices (`LICENSE.md`, `NOTICE.md`, `THIRD_PARTY.md`, which the footer links), then `scripts/assemble-page.mjs` adds three things to `site/index.html`: the commit as `<meta name="pplus-version">`, which the footer shows; a `<link rel="modulepreload">` for every module the page's imports reach, after the import map, so a first visit asks for all of them at once instead of one level of imports at a time; and the home page already drawn, inside `#app` (`tests/load.test.mjs` holds the list to the import graph, the inline scripts to their hashes, and the prerendered page to its place). **The home page is prerendered** because drawing it runs the engines on its own examples, a fuzzer's report among them, about a second of work that came after 150 modules had loaded: a visitor saw nothing for 1.3 to 3 seconds. The assembler draws it once in Node with the app's own `homeView` (on the small document the Quarto views use), so the browser paints it with the HTML and the stylesheet, in about a tenth of a second; when the app has loaded it draws the page again, with its controls, and puts it in the prerendered one's place, copying the two costly parts from it instead of working them out again (`src/ui/prerender.ts`). The prerendered page shows only on the home address: the page's first inline script marks it before the first paint, and without the mark it stays hidden, so an embed or a problem's address never shows the home page first. A checkout served as it is (`pixi run dev`) has no prerendered page and draws the home page as before. And `scripts/deploy.sh <slug> [user@host]` publishes `site/` to `/srv/www/<slug>` as an atomic release:
```
/srv/www/pplus -> symlink to the live release
/srv/www/pplus.releases/ one directory per deployed commit, the five newest kept
```
The swap is one rename, so nobody sees a half-copied site, and a rollback is `ln -sfn /srv/www/pplus.releases/<sha> /srv/www/pplus`. A pre-existing plain directory at `/srv/www/<slug>` is moved to `<slug>.releases/before-releases` on the first run.
**Caddy**: [`Caddyfile`](Caddyfile) is the site block (root on the symlink, `file_server`, a missing path that looks like a page sent to `/` (the app's routes are after the `#`, and the page names its files relative to itself, so it cannot be served in place at a deeper path) while a missing file is a plain 404, and caching: `no-cache` on the page, the modules and the stylesheets, so the browser asks again on every load and gets a bodiless 304 while a file is unchanged; 30 days on the fonts, which change only when `fonts/` is replaced; `no-store` on `VERSION`; nothing is `immutable`, since no file name here changes with its content; the license and notices are served as plain text) and a snippet of security headers it imports: a Content-Security-Policy (everything from the site itself, the two inline scripts of `index.html` by their hashes, `connect-src` open only to godbolt.org for Compile & run, and `frame-ancestors` so course pages can embed views), `Referrer-Policy: no-referrer`, `X-Content-Type-Options: nosniff`, and a `Permissions-Policy` that turns off the camera, microphone, location, payments and USB. `tests/csp.test.mjs` fails when an inline script of `index.html` changes without its hash, or when the page names another origin; a new backend for Compile & run needs its origin in `connect-src`. The live shortcodes and `live="click"` put the app in an iframe, so a Quarto site on another origin needs that origin added to `frame-ancestors`; the rendered views need nothing from the host. Caddy issues the certificate itself; add the block to `/etc/caddy/Caddyfile` and `systemctl reload caddy`. The tests read the file as text; `pixi run caddy-check` serves an assembled `site/` with a real Caddy under it (localhost, plain HTTP) and holds the headers, the caching and both missing-path answers. Run it after changing the Caddyfile.
**Gitea Actions** (`.gitea/workflows/deploy.yml`): the deploy job runs on the label `quarto-deploy`. On the droplet's runner that label is a container image (`infra/quarto-deploy` on the same Gitea: Node, git, rsync, curl, Quarto and a pixi) with the droplet's `/srv/www` bound into it. A job gets a new container and runs in it as root; nothing it writes outside `/srv/www` is there for the next job. The image is general on purpose, so everything else the job needs comes from `pixi.lock`:
1. `actions/checkout` with `submodules: recursive` (the job's token must be able to read pplus-core and pplus-library while they are private);
2. **pixi**, at the version `PIXI_VERSION` names in the workflow: the image's when it is that version, else that release from GitHub, checked by SHA-256 (on this runner, every job);
3. `pixi install --locked`: Node, Python, pandoc and the Mojo compiler, exactly as `pixi.lock` has them. With no cache between jobs the whole environment is downloaded each time;
4. `pixi run ci`: every test of the site (the Quarto ones run in pandoc and fail, not skip, if it is missing), and core's `mojo-status-strict` at the pinned commit (offline, a second);
5. on `main`: `pixi run release-check`, `scripts/assemble.sh`, and `scripts/deploy.sh` into `/srv/www`.
**The deploy job is light on purpose.** The droplet is a server with other work, and slow: the site's tests take 2.5 minutes there (half a minute on a workstation), and core's tests, which the job ran until November 2026, took 15 to 22. So the job holds the site to its own tests and to core's stored compiler answers, and nothing in it builds a Mojo program. Core's tests, `library-check`, `mojo-local` and the fuzzers are pplus-core's gate (`pixi run ci` there), which runs on a second runner with the label `heavy` (core's `docs/ci.md` has its setup) and on a developer's machine. Before moving the `core` pointer, run `pixi run core-test` here, or let `pixi run release-bump` run core's gate (below): the deploy job will not catch a core that fails its own tests.
What the runner has to bring is small: git and Node (for `actions/checkout`), bash, curl, tar, `sha256sum`, and a `/srv/www` the job can write (or rsync and SSH to the machine that has it, below). The same workflow runs on a runner in host mode (a label registered as `quarto-deploy:host`: steps run on the machine's own shell as the runner's user, which then needs those tools and write access to `/srv/www`); there pixi's package cache and the fetched pixi in `~/.cache` last from job to job.
`SITE_SLUG` (repository variable) names the directory under `/srv/www` and defaults to `pplus`, the directory the Caddyfile serves (it defaulted to the repository's name, and the first job on this repository published to `/srv/www/pplus-web`, which nothing serves). After publishing, a smoke test fetches `/VERSION` from the live site, at `SITE_URL` (repository variable, `https://pplus.scient.ing` when unset), and fails the job if it is not the deployed commit, which is what catches a job that publishes where nothing is served. On GitHub Actions the build and test steps run unchanged with `runs-on: ubuntu-latest`; publishing needs the droplet.
**Once the repositories are public**, a pull request from a fork runs its own workflow file and code on this runner, in a container that can write `/srv/www`. Run pull requests only from branches of the repository itself (each workflow's job has an `if:` that does that), or require approval for fork runs in Gitea's repository settings where the version offers it, or give pull requests a separate runner that cannot write the web root.
If the runner ever moves off the droplet, `scripts/deploy.sh pplus deploy@droplet` rsyncs over SSH instead; give that runner a key whose public half is in `deploy`'s `authorized_keys`.
By hand, on the droplet: `pixi run deploy pplus`. From elsewhere: `pixi run deploy pplus deploy@droplet`.
## Moving the pointers
The site pins pplus-core by commit (`core/`), and core pins pplus-library (`core/library/`). A change at the bottom reaches the site in three commits, pushed in order: library, then core with its new `library` pointer and what it regenerates from it, then the site with its new `core` pointer. Done by hand the first time, it broke twice: a pointer named a commit from before a move, and a pointer named a commit that existed on one machine only. Two tasks (`scripts/release.mjs`) hold the order:
- `pixi run release-check` says whether the chain is consistent, and what to run when it is not: each pointer's commit is in the checkout, the checkout is at it and has no uncommitted change, and the commit is on the submodule's branch (`main`) on its remote. It asks the remote with `git ls-remote` (never a fetch); with `--offline`, or when the remote cannot be asked, it reads the last fetch and says "cannot tell" where that cannot settle it. It also checks that the generated files are current (core's `library.mjs emit --check`, `corpus.py emit` writing nothing new, `quarto-css --check`) and that Mojo is pinned alike in `core/library/mojo/pin.json`, both `pixi.toml` files and both `pixi.lock` files, with Node and Python alike in core and the site. Exit status 1 on a problem.
- `pixi run release-bump`, after a change committed in `core/library` (or in `core`): moves core's pointer to the library checkout, regenerates core's registry and the corpus's generated files, runs core's gate and `library-check`; then builds the site against that core, regenerates the Quarto extension's files, runs the site's tests; and prints the commits to make and push, in order. With `--commit` it makes the commits; it never pushes, tags or fetches. `--no-gate` skips the gates.
`tests/chain.test.mjs` runs the cheap half of `release-check` with every `npm test`: it fails when a pointer names a commit its checkout does not have, or a checkout is behind its pointer, and names the command that fixes it (`git submodule update --init --recursive`). Core's `tests/chain.test.mjs` does the same for its own pointer.
## Versions and tags
Today core is 0.2.0 and the site 0.1.0 (`package.json` and `pixi.toml` say the same, and `tests/tasks.test.mjs` keeps them so); the library has no version, and nothing is tagged. A built site says what it is made of: `site/build.json` (`scripts/build-info.mjs`) holds each repository's version, commit and `git describe --tags --always --dirty`, and the Mojo the corpus is pinned to. The app does not show it yet; an About line that reads `build.json` is the remaining step.
Proposed for the first public release (the owner's to decide; nothing here creates a tag):
- Each repository keeps its own version, in its `package.json` and `pixi.toml` (the library, which has neither, in its tag only), and a release is an annotated tag `vX.Y.Z` on the commit the repository above pins.
- First release: pplus-library `v0.1.0`; pplus-core `v0.3.0`, because its exports changed since 0.2.0 (`linear()`, `LIBRARY_COMMIT`); the site `v0.2.0`. The version bumps are commits of their own, made before the tags.
- Tag bottom up, after `pixi run release-check` is clean and each commit is pushed: the library at the commit core pins, core at the commit the site pins, then the site. `git describe` in `build.json` then reads `v0.3.0` instead of a hash.
- `release-bump` stays out of tagging: moving a pointer is not a release. What it could do later is refuse to move a pointer onto a commit that is behind a pinned tag, and `release-check --release` could require each pinned commit to carry a `v*` tag that matches its `package.json`.