Files
pplus-web/docs/deploy.md
T
alexm e092cb029c
Build, test, deploy / site (push) Failing after 15m13s
Initial commit
2026-11-12 22:24:02 -05:00

11 KiB

Deploying

The site builds against pplus-core, the git submodule core/, at the commit this repository pins. A checkout needs it (git clone --recurse-submodules, or git submodule update --init); npm run build builds core first. To take a newer core: git -C core pull (or check out a tag), pixi run test, and commit the new core pointer with what it changes on this side.

pixi run site tests the app and assembles site/; pixi run deploy [slug] [user@host] does that and publishes it. CI (.gitea/workflows/deploy.yml) runs pixi run ci on every push and pull request, and on a push to main checks the chain of pointers (pixi run release-check), assembles and publishes.

The site is seven things: index.html, styles.css, fonts/ (IBM Plex, served by the site so a visit asks no font host for anything; fonts/README.md says where the files came from), styles/ (the course's widget styles, which the Hardware area's machine views use), dist/, core/ (pplus-core's built core/dist/, which index.html's import map names as #core/), and the Quarto extension (quarto/_extensions/pplus, without an engine copy), which the With Quarto page loads (pplus.css, pplus.js) and links to file by file. scripts/assemble.sh copies them into site/ with a VERSION file holding the commit, a build.json naming the three commits the site was built from (below), and the license and notices (LICENSE.md, NOTICE.md, THIRD_PARTY.md, which the footer links), then scripts/assemble-page.mjs adds two things to site/index.html: the commit as <meta name="pplus-version">, which the footer shows, and a <link rel="modulepreload"> for every module the page's imports reach, after the import map, so a first visit asks for all of them at once instead of one level of imports at a time (tests/load.test.mjs holds the list to the import graph and the inline scripts to their hashes). And scripts/deploy.sh <slug> [user@host] publishes site/ to /srv/www/<slug> as an atomic release:

/srv/www/pplus            -> symlink to the live release
/srv/www/pplus.releases/  one directory per deployed commit, the five newest kept

The swap is one rename, so nobody sees a half-copied site, and a rollback is ln -sfn /srv/www/pplus.releases/<sha> /srv/www/pplus. A pre-existing plain directory at /srv/www/<slug> is moved to <slug>.releases/before-releases on the first run.

Caddy: Caddyfile is the site block (root on the symlink, file_server, a missing path that looks like a page sent to / (the app's routes are after the #, and the page names its files relative to itself, so it cannot be served in place at a deeper path) while a missing file is a plain 404, and caching: no-cache on the page, the modules and the stylesheets, so the browser asks again on every load and gets a bodiless 304 while a file is unchanged; 30 days on the fonts, which change only when fonts/ is replaced; no-store on VERSION; nothing is immutable, since no file name here changes with its content; the license and notices are served as plain text) and a snippet of security headers it imports: a Content-Security-Policy (everything from the site itself, the two inline scripts of index.html by their hashes, connect-src open only to godbolt.org for Compile & run, and frame-ancestors so course pages can embed views), Referrer-Policy: no-referrer, X-Content-Type-Options: nosniff, and a Permissions-Policy that turns off the camera, microphone, location, payments and USB. tests/csp.test.mjs fails when an inline script of index.html changes without its hash, or when the page names another origin; a new backend for Compile & run needs its origin in connect-src. The live shortcodes and live="click" put the app in an iframe, so a Quarto site on another origin needs that origin added to frame-ancestors; the rendered views need nothing from the host. Caddy issues the certificate itself; add the block to /etc/caddy/Caddyfile and systemctl reload caddy. The tests read the file as text; pixi run caddy-check serves an assembled site/ with a real Caddy under it (localhost, plain HTTP) and holds the headers, the caching and both missing-path answers. Run it after changing the Caddyfile.

Gitea Actions without a container image (.gitea/workflows/deploy.yml, and pplus-core's and pplus-library's .gitea/workflows/): the act_runner on the droplet runs jobs on the host shell instead of in docker. Once, on the droplet:

# tools the jobs use: git and node for actions/checkout, curl, tar and sha256sum to fetch the pinned pixi,
# rsync for SSH deploys, a C compiler for the C references core's library-check builds
apt install -y git rsync curl tar coreutils build-essential
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && apt install -y nodejs   # any node >= 20, for actions/checkout
# a user for the runner that owns the web root
useradd -m -s /bin/bash deploy
mkdir -p /srv/www && chown deploy:deploy /srv/www
# register the runner with a host-mode label (the :host suffix is the whole trick)
sudo -u deploy act_runner register --instance https://<your-gitea> --token <runner token> --labels "quarto-deploy:host" --no-interactive
# run it as a service (act_runner generate-config > config.yaml; systemd unit per the Gitea docs, User=deploy)

With that label, runs-on: quarto-deploy executes steps directly as deploy. Every job then does what a developer does:

  1. actions/checkout with submodules: recursive (the job's token must be able to read pplus-core and pplus-library while they are private);
  2. pixi, at the version PIXI_VERSION names in the workflow: the host's when it is that version, else the release fetched once into ~deploy/.cache/pplus-ci/pixi-<version>/, checked by SHA-256. Nothing needs installing for it; to have it on the host anyway, as deploy: curl -fsSL https://pixi.sh/install.sh | PIXI_VERSION=v0.83.0 bash;
  3. pixi install --locked: Node, Python, pandoc and the Mojo compiler, exactly as pixi.lock has them (the package cache in ~deploy/.cache/rattler keeps about 2 GB after the first run);
  4. pixi run ci: every test of the site (the Quarto ones run in pandoc and fail, not skip, if it is missing), and core's tests and mojo-status-strict at the pinned commit;
  5. on main: pixi run release-check, scripts/assemble.sh, and scripts/deploy.sh into /srv/www.

SITE_SLUG (repository variable) names the directory and defaults to the repository name; set it to pplus. SITE_URL turns on a smoke test that fetches /VERSION from the live site and fails the job if it is not the deployed commit. pplus-core's workflows (its docs/ci.md) use the same runner and also keep the pinned Dafny and the Mojo build answers in ~deploy/.cache/pplus-ci/core-tools; the runner takes one job at a time (the default capacity: 1), which those caches assume. On GitHub Actions the build and test steps run unchanged with runs-on: ubuntu-latest; publishing needs the droplet.

Once the repositories are public, a pull request from a fork runs its own workflow file and code on this runner, as deploy, which can write /srv/www. Run pull requests only from branches of the repository itself (each workflow's job has an if: that does that), or require approval for fork runs in Gitea's repository settings where the version offers it, or give pull requests a separate runner that cannot write the web root.

If the runner ever moves off the droplet, scripts/deploy.sh pplus deploy@droplet rsyncs over SSH instead; give that runner a key whose public half is in deploy's authorized_keys.

By hand, on the droplet: pixi run deploy pplus. From elsewhere: pixi run deploy pplus deploy@droplet.

Moving the pointers

The site pins pplus-core by commit (core/), and core pins pplus-library (core/library/). A change at the bottom reaches the site in three commits, pushed in order: library, then core with its new library pointer and what it regenerates from it, then the site with its new core pointer. Done by hand the first time, it broke twice: a pointer named a commit from before a move, and a pointer named a commit that existed on one machine only. Two tasks (scripts/release.mjs) hold the order:

  • pixi run release-check says whether the chain is consistent, and what to run when it is not: each pointer's commit is in the checkout, the checkout is at it and has no uncommitted change, and the commit is on the submodule's branch (main) on its remote. It asks the remote with git ls-remote (never a fetch); with --offline, or when the remote cannot be asked, it reads the last fetch and says "cannot tell" where that cannot settle it. It also checks that the generated files are current (core's library.mjs emit --check, corpus.py emit writing nothing new, quarto-css --check) and that Mojo is pinned alike in core/library/mojo/pin.json, both pixi.toml files and both pixi.lock files, with Node and Python alike in core and the site. Exit status 1 on a problem.
  • pixi run release-bump, after a change committed in core/library (or in core): moves core's pointer to the library checkout, regenerates core's registry and the corpus's generated files, runs core's gate and library-check; then builds the site against that core, regenerates the Quarto extension's files, runs the site's tests; and prints the commits to make and push, in order. With --commit it makes the commits; it never pushes, tags or fetches. --no-gate skips the gates.

tests/chain.test.mjs runs the cheap half of release-check with every npm test: it fails when a pointer names a commit its checkout does not have, or a checkout is behind its pointer, and names the command that fixes it (git submodule update --init --recursive). Core's tests/chain.test.mjs does the same for its own pointer.

Versions and tags

Today core is 0.2.0 and the site 0.1.0 (package.json and pixi.toml say the same, and tests/tasks.test.mjs keeps them so); the library has no version, and nothing is tagged. A built site says what it is made of: site/build.json (scripts/build-info.mjs) holds each repository's version, commit and git describe --tags --always --dirty, and the Mojo the corpus is pinned to. The app does not show it yet; an About line that reads build.json is the remaining step.

Proposed for the first public release (the owner's to decide; nothing here creates a tag):

  • Each repository keeps its own version, in its package.json and pixi.toml (the library, which has neither, in its tag only), and a release is an annotated tag vX.Y.Z on the commit the repository above pins.
  • First release: pplus-library v0.1.0; pplus-core v0.3.0, because its exports changed since 0.2.0 (linear(), LIBRARY_COMMIT); the site v0.2.0. The version bumps are commits of their own, made before the tags.
  • Tag bottom up, after pixi run release-check is clean and each commit is pushed: the library at the commit core pins, core at the commit the site pins, then the site. git describe in build.json then reads v0.3.0 instead of a hash.
  • release-bump stays out of tagging: moving a pointer is not a release. What it could do later is refuse to move a pointer onto a commit that is behind a pinned tag, and release-check --release could require each pinned commit to carry a v* tag that matches its package.json.