11 KiB
Deploying
The site builds against pplus-core, the git submodule core/, at the commit this repository pins. A checkout needs it (git clone --recurse-submodules, or git submodule update --init); npm run build builds core first. To take a newer core: git -C core pull (or check out a tag), pixi run test, and commit the new core pointer with what it changes on this side.
pixi run site tests the app and assembles site/; pixi run deploy [slug] [user@host] does that and publishes it. CI (.gitea/workflows/deploy.yml) runs pixi run ci on every push and pull request, and on a push to main checks the chain of pointers (pixi run release-check), assembles and publishes.
The site is seven things: index.html, styles.css, fonts/ (IBM Plex, served by the site so a visit asks no font host for anything; fonts/README.md says where the files came from), styles/ (the course's widget styles, which the Hardware area's machine views use), dist/, core/ (pplus-core's built core/dist/, which index.html's import map names as #core/), and the Quarto extension (quarto/_extensions/pplus, without an engine copy), which the With Quarto page loads (pplus.css, pplus.js) and links to file by file. scripts/assemble.sh copies them into site/ with a VERSION file holding the commit, a build.json naming the three commits the site was built from (below), and the license and notices (LICENSE.md, NOTICE.md, THIRD_PARTY.md, which the footer links), then scripts/assemble-page.mjs adds two things to site/index.html: the commit as <meta name="pplus-version">, which the footer shows, and a <link rel="modulepreload"> for every module the page's imports reach, after the import map, so a first visit asks for all of them at once instead of one level of imports at a time (tests/load.test.mjs holds the list to the import graph and the inline scripts to their hashes). And scripts/deploy.sh <slug> [user@host] publishes site/ to /srv/www/<slug> as an atomic release:
/srv/www/pplus -> symlink to the live release
/srv/www/pplus.releases/ one directory per deployed commit, the five newest kept
The swap is one rename, so nobody sees a half-copied site, and a rollback is ln -sfn /srv/www/pplus.releases/<sha> /srv/www/pplus. A pre-existing plain directory at /srv/www/<slug> is moved to <slug>.releases/before-releases on the first run.
Caddy: Caddyfile is the site block (root on the symlink, file_server, a missing path that looks like a page sent to / (the app's routes are after the #, and the page names its files relative to itself, so it cannot be served in place at a deeper path) while a missing file is a plain 404, and caching: no-cache on the page, the modules and the stylesheets, so the browser asks again on every load and gets a bodiless 304 while a file is unchanged; 30 days on the fonts, which change only when fonts/ is replaced; no-store on VERSION; nothing is immutable, since no file name here changes with its content; the license and notices are served as plain text) and a snippet of security headers it imports: a Content-Security-Policy (everything from the site itself, the two inline scripts of index.html by their hashes, connect-src open only to godbolt.org for Compile & run, and frame-ancestors so course pages can embed views), Referrer-Policy: no-referrer, X-Content-Type-Options: nosniff, and a Permissions-Policy that turns off the camera, microphone, location, payments and USB. tests/csp.test.mjs fails when an inline script of index.html changes without its hash, or when the page names another origin; a new backend for Compile & run needs its origin in connect-src. The live shortcodes and live="click" put the app in an iframe, so a Quarto site on another origin needs that origin added to frame-ancestors; the rendered views need nothing from the host. Caddy issues the certificate itself; add the block to /etc/caddy/Caddyfile and systemctl reload caddy. The tests read the file as text; pixi run caddy-check serves an assembled site/ with a real Caddy under it (localhost, plain HTTP) and holds the headers, the caching and both missing-path answers. Run it after changing the Caddyfile.
Gitea Actions without a container image (.gitea/workflows/deploy.yml, and pplus-core's and pplus-library's .gitea/workflows/): the act_runner on the droplet runs jobs on the host shell instead of in docker. Once, on the droplet:
# tools the jobs use: git and node for actions/checkout, curl, tar and sha256sum to fetch the pinned pixi,
# rsync for SSH deploys, a C compiler for the C references core's library-check builds
apt install -y git rsync curl tar coreutils build-essential
curl -fsSL https://deb.nodesource.com/setup_22.x | bash - && apt install -y nodejs # any node >= 20, for actions/checkout
# a user for the runner that owns the web root
useradd -m -s /bin/bash deploy
mkdir -p /srv/www && chown deploy:deploy /srv/www
# register the runner with a host-mode label (the :host suffix is the whole trick)
sudo -u deploy act_runner register --instance https://<your-gitea> --token <runner token> --labels "quarto-deploy:host" --no-interactive
# run it as a service (act_runner generate-config > config.yaml; systemd unit per the Gitea docs, User=deploy)
With that label, runs-on: quarto-deploy executes steps directly as deploy. Every job then does what a developer does:
actions/checkoutwithsubmodules: recursive(the job's token must be able to read pplus-core and pplus-library while they are private);- pixi, at the version
PIXI_VERSIONnames in the workflow: the host's when it is that version, else the release fetched once into~deploy/.cache/pplus-ci/pixi-<version>/, checked by SHA-256. Nothing needs installing for it; to have it on the host anyway, asdeploy:curl -fsSL https://pixi.sh/install.sh | PIXI_VERSION=v0.83.0 bash; pixi install --locked: Node, Python, pandoc and the Mojo compiler, exactly aspixi.lockhas them (the package cache in~deploy/.cache/rattlerkeeps about 2 GB after the first run);pixi run ci: every test of the site (the Quarto ones run in pandoc and fail, not skip, if it is missing), and core's tests andmojo-status-strictat the pinned commit;- on
main:pixi run release-check,scripts/assemble.sh, andscripts/deploy.shinto/srv/www.
SITE_SLUG (repository variable) names the directory and defaults to the repository name; set it to pplus. SITE_URL turns on a smoke test that fetches /VERSION from the live site and fails the job if it is not the deployed commit. pplus-core's workflows (its docs/ci.md) use the same runner and also keep the pinned Dafny and the Mojo build answers in ~deploy/.cache/pplus-ci/core-tools; the runner takes one job at a time (the default capacity: 1), which those caches assume. On GitHub Actions the build and test steps run unchanged with runs-on: ubuntu-latest; publishing needs the droplet.
Once the repositories are public, a pull request from a fork runs its own workflow file and code on this runner, as deploy, which can write /srv/www. Run pull requests only from branches of the repository itself (each workflow's job has an if: that does that), or require approval for fork runs in Gitea's repository settings where the version offers it, or give pull requests a separate runner that cannot write the web root.
If the runner ever moves off the droplet, scripts/deploy.sh pplus deploy@droplet rsyncs over SSH instead; give that runner a key whose public half is in deploy's authorized_keys.
By hand, on the droplet: pixi run deploy pplus. From elsewhere: pixi run deploy pplus deploy@droplet.
Moving the pointers
The site pins pplus-core by commit (core/), and core pins pplus-library (core/library/). A change at the bottom reaches the site in three commits, pushed in order: library, then core with its new library pointer and what it regenerates from it, then the site with its new core pointer. Done by hand the first time, it broke twice: a pointer named a commit from before a move, and a pointer named a commit that existed on one machine only. Two tasks (scripts/release.mjs) hold the order:
pixi run release-checksays whether the chain is consistent, and what to run when it is not: each pointer's commit is in the checkout, the checkout is at it and has no uncommitted change, and the commit is on the submodule's branch (main) on its remote. It asks the remote withgit ls-remote(never a fetch); with--offline, or when the remote cannot be asked, it reads the last fetch and says "cannot tell" where that cannot settle it. It also checks that the generated files are current (core'slibrary.mjs emit --check,corpus.py emitwriting nothing new,quarto-css --check) and that Mojo is pinned alike incore/library/mojo/pin.json, bothpixi.tomlfiles and bothpixi.lockfiles, with Node and Python alike in core and the site. Exit status 1 on a problem.pixi run release-bump, after a change committed incore/library(or incore): moves core's pointer to the library checkout, regenerates core's registry and the corpus's generated files, runs core's gate andlibrary-check; then builds the site against that core, regenerates the Quarto extension's files, runs the site's tests; and prints the commits to make and push, in order. With--commitit makes the commits; it never pushes, tags or fetches.--no-gateskips the gates.
tests/chain.test.mjs runs the cheap half of release-check with every npm test: it fails when a pointer names a commit its checkout does not have, or a checkout is behind its pointer, and names the command that fixes it (git submodule update --init --recursive). Core's tests/chain.test.mjs does the same for its own pointer.
Versions and tags
Today core is 0.2.0 and the site 0.1.0 (package.json and pixi.toml say the same, and tests/tasks.test.mjs keeps them so); the library has no version, and nothing is tagged. A built site says what it is made of: site/build.json (scripts/build-info.mjs) holds each repository's version, commit and git describe --tags --always --dirty, and the Mojo the corpus is pinned to. The app does not show it yet; an About line that reads build.json is the remaining step.
Proposed for the first public release (the owner's to decide; nothing here creates a tag):
- Each repository keeps its own version, in its
package.jsonandpixi.toml(the library, which has neither, in its tag only), and a release is an annotated tagvX.Y.Zon the commit the repository above pins. - First release: pplus-library
v0.1.0; pplus-corev0.3.0, because its exports changed since 0.2.0 (linear(),LIBRARY_COMMIT); the sitev0.2.0. The version bumps are commits of their own, made before the tags. - Tag bottom up, after
pixi run release-checkis clean and each commit is pushed: the library at the commit core pins, core at the commit the site pins, then the site.git describeinbuild.jsonthen readsv0.3.0instead of a hash. release-bumpstays out of tagging: moving a pointer is not a release. What it could do later is refuse to move a pointer onto a commit that is behind a pinned tag, andrelease-check --releasecould require each pinned commit to carry av*tag that matches itspackage.json.