182 lines
10 KiB
Markdown
182 lines
10 KiB
Markdown
<<<<<<< HEAD
|
|
# Deploying the website and the Python package
|
|
|
|
This folder holds what it takes to serve the paperspec website at <https://paperspec.scient.ing>.
|
|
The last sections say how the Python package `paperspec` is published to the Gitea package registry, and how other repositories install it from there.
|
|
The site is static: `paperspec site` builds it into `_site/`, and any web server can serve that folder.
|
|
|
|
## What the site serves
|
|
|
|
The site mirrors the repository, so a file at `schemas/anchor.schema.json` in the repository is served at `https://paperspec.scient.ing/schemas/anchor.schema.json`.
|
|
Three addresses that the formats themselves name depend on that:
|
|
|
|
| Address | What is there |
|
|
| --- | --- |
|
|
| `https://paperspec.scient.ing/schemas/<file>` | Each JSON Schema, at the address in its `$id`. |
|
|
| `https://paperspec.scient.ing/context/scienting-1.jsonld` | The JSON-LD context every layer and annotation lists. |
|
|
| `https://paperspec.scient.ing/vocab` | The vocabulary page, whose fragments are the IRIs of the context's terms. |
|
|
|
|
Beside every Markdown file the builder writes a rendered page: `README.md` becomes its folder's `index.html`, and `X.md` becomes `X.html`.
|
|
A folder without a README gets a listing page, so every fixture and vector can be browsed.
|
|
The pages need no server-side code; search runs in the browser over a small index of headings, `search.json`.
|
|
|
|
## Build it
|
|
|
|
```sh
|
|
pixi run site # writes _site/
|
|
pixi run site --out /tmp/paperspec-site
|
|
```
|
|
|
|
`pixi run check` builds the site into a temporary folder and checks that every Markdown file has a page, that every internal link and fragment resolves, that no Markdown is left unrendered, and that each schema's `$id`, the context's address and the vocabulary's IRIs are files of the site.
|
|
|
|
To look at it locally, serve the folder with any static server, for example:
|
|
|
|
```sh
|
|
pixi run python -m http.server --directory _site 8000
|
|
```
|
|
|
|
## Serve it with Caddy
|
|
|
|
[Caddyfile](Caddyfile) serves `/srv/www/paperspec` at `paperspec.scient.ing`.
|
|
It sets the media types the data needs (`application/ld+json` for the context, `application/schema+json` for schemas, `application/jsonl` for vectors and package files), allows every file to be fetched from any origin, and serves `404.html` for missing paths.
|
|
Add it to the server's Caddy configuration, for example with an `import` line, and reload Caddy.
|
|
|
|
## Publish from CI
|
|
|
|
The workflow `.gitea/workflows/site.yml` runs on every push to `main`.
|
|
It installs the environment from `pixi.lock`, runs `pixi run quick`, builds the site, and copies it with `rsync` to `/srv/www/paperspec`, the folder Caddy serves.
|
|
A failed check stops the publish, so the site never serves a release that does not pass its own checks.
|
|
|
|
The copy is local: the `linux-base` runner starts each job in a container with the host's `/srv/www` mounted at the same path, so no key or secret is involved.
|
|
On a runner without that mount the workflow still builds and checks the site, and says that it did not publish.
|
|
`SITE_ROOT` and `SITE_SLUG` at the top of the workflow name the folder.
|
|
|
|
Only changed files are copied, so every other file keeps its `Last-Modified` and `ETag` and caches stay valid across releases.
|
|
The environment in `.pixi/envs` is cached between runs under the hash of `pixi.lock`, so a fresh container installs it only when the lock file changes.
|
|
|
|
## Publish the Python package
|
|
|
|
The Python package `paperspec` holds the reference implementation and the `paperspec` command.
|
|
The workflow `.gitea/workflows/publish.yml` publishes it to the PyPI registry of the Gitea instance, `https://git.scient.ing/api/packages/infra/pypi`, when a version tag such as `v0.1.1` is pushed.
|
|
It checks the repository, builds the sdist and the wheel with `pixi run -e dev dist`, confirms that the version setuptools-scm computed is the tag's, checks the metadata with twine, and uploads both files.
|
|
The package version is the version of this code; the formats keep the version that `CHANGELOG.md` and `pixi.toml` state.
|
|
|
|
### Set it up once
|
|
|
|
1. Pick the account the workflow publishes as, a bot account or your own, and make it a member of a team in the `infra` organization with write access to packages.
|
|
2. Signed in as that account, create an access token under Settings, Applications, with the `write:package` scope.
|
|
3. In this repository's Settings, Actions, Secrets, add `PACKAGE_USER` with the account's user name and `PACKAGE_TOKEN` with the token.
|
|
Organization secrets work as well, for every repository in `infra`.
|
|
Gitea refuses secret names that start with `GITEA_` or `GITHUB_`.
|
|
4. After the first upload, link the package to this repository in the package's settings, so it shows on the repository's Packages tab.
|
|
|
|
### Release a version
|
|
|
|
Tag a commit on `main` and push the tag:
|
|
|
|
```sh
|
|
git tag -a v0.1.1 -m "Release the paperspec package, version 0.1.1"
|
|
git push origin v0.1.1
|
|
```
|
|
|
|
The job refuses to upload when the built version differs from the tag, for example when the tag is not on the commit it checked out.
|
|
The registry never replaces a version, so to publish a fix, push a new tag.
|
|
A rerun on the same tag skips the files already uploaded, which makes it safe to rerun a job that failed on its secrets: open the workflow under Actions and run it on the tag.
|
|
|
|
### Install from the registry
|
|
|
|
The registry does not mirror PyPI, so an installer needs two indexes: the registry for `paperspec`, and PyPI for `jsonschema`, `referencing` and `cryptography`.
|
|
Tie `paperspec` to the registry wherever the installer allows it, so that a package of the same name on PyPI is never chosen instead.
|
|
|
|
With pixi, in `pixi.toml`:
|
|
|
|
```toml
|
|
[pypi-dependencies]
|
|
paperspec = { version = ">=0.1.1", extras = ["tools"], index = "https://git.scient.ing/api/packages/infra/pypi/simple" }
|
|
```
|
|
|
|
With uv, in `pyproject.toml`:
|
|
|
|
```toml
|
|
[[tool.uv.index]]
|
|
name = "scienting"
|
|
url = "https://git.scient.ing/api/packages/infra/pypi/simple"
|
|
explicit = true
|
|
|
|
[tool.uv.sources]
|
|
paperspec = { index = "scienting" }
|
|
```
|
|
|
|
With pip, which cannot tie a package to an index and takes the highest version from either:
|
|
|
|
```sh
|
|
pip install --index-url https://git.scient.ing/api/packages/infra/pypi/simple/ --extra-index-url https://pypi.org/simple/ "paperspec[tools]"
|
|
```
|
|
|
|
If the organization or the package is private, the installer also needs a token with the `read:package` scope, for example in a `~/.netrc` entry for `git.scient.ing`.
|
|
A git pin to a commit, as `pixi.toml` and `pyproject.toml` can also express, keeps working and needs no registry.
|
|
|
|
## Distributing document packages
|
|
||||||| parent of 4fa326d (First implementation (#1))
|
|
=======
|
|
# Deploying the website
|
|
|
|
This folder holds what it takes to serve the paperspec website at <https://paperspec.scient.ing>.
|
|
The site is static: `tools/site.py` builds it into `_site/`, and any web server can serve that folder.
|
|
|
|
## What the site serves
|
|
|
|
The site mirrors the repository, so a file at `schemas/anchor.schema.json` in the repository is served at `https://paperspec.scient.ing/schemas/anchor.schema.json`.
|
|
Three addresses that the formats themselves name depend on that:
|
|
|
|
| Address | What is there |
|
|
| --- | --- |
|
|
| `https://paperspec.scient.ing/schemas/<file>` | Each JSON Schema, at the address in its `$id`. |
|
|
| `https://paperspec.scient.ing/context/scienting-1.jsonld` | The JSON-LD context every layer and annotation lists. |
|
|
| `https://paperspec.scient.ing/vocab` | The vocabulary page, whose fragments are the IRIs of the context's terms. |
|
|
|
|
Beside every Markdown file the builder writes a rendered page: `README.md` becomes its folder's `index.html`, and `X.md` becomes `X.html`.
|
|
A folder without a README gets a listing page, so every fixture and vector can be browsed.
|
|
The pages need no server-side code; search runs in the browser over a small index of headings, `search.json`.
|
|
|
|
## Build it
|
|
|
|
```sh
|
|
pixi run python tools/site.py # writes _site/
|
|
pixi run python tools/site.py --out /tmp/paperspec-site
|
|
```
|
|
|
|
`pixi run check` builds the site into a temporary folder and checks that every Markdown file has a page, that every internal link and fragment resolves, that no Markdown is left unrendered, and that each schema's `$id`, the context's address and the vocabulary's IRIs are files of the site.
|
|
|
|
To look at it locally, serve the folder with any static server, for example:
|
|
|
|
```sh
|
|
pixi run python -m http.server --directory _site 8000
|
|
```
|
|
|
|
## Serve it with Caddy
|
|
|
|
[Caddyfile](Caddyfile) serves `/srv/www/paperspec` at `paperspec.scient.ing`.
|
|
It sets the media types the data needs (`application/ld+json` for the context, `application/schema+json` for schemas, `application/jsonl` for vectors and package files), allows every file to be fetched from any origin, and serves `404.html` for missing paths.
|
|
Add it to the server's Caddy configuration, for example with an `import` line, and reload Caddy.
|
|
|
|
## Publish from CI
|
|
|
|
The workflow `.gitea/workflows/site.yml` runs on every push to `main`.
|
|
It installs the environment from `pixi.lock`, runs `pixi run quick`, builds the site, and copies it with `rsync` to `/srv/www/paperspec`, the folder Caddy serves.
|
|
A failed check stops the publish, so the site never serves a release that does not pass its own checks.
|
|
|
|
The copy is local: the `linux-base` runner starts each job in a container with the host's `/srv/www` mounted at the same path, so no key or secret is involved.
|
|
On a runner without that mount the workflow still builds and checks the site, and says that it did not publish.
|
|
`SITE_ROOT` and `SITE_SLUG` at the top of the workflow name the folder.
|
|
|
|
Only changed files are copied, so every other file keeps its `Last-Modified` and `ETag` and caches stay valid across releases.
|
|
The environment in `.pixi/envs` is cached between runs under the hash of `pixi.lock`, so a fresh container installs it only when the lock file changes.
|
|
|
|
## Distributing packages
|
|
>>>>>>> 4fa326d (First implementation (#1))
|
|
|
|
A zipped package is an ordinary file, and the site serves the fixtures as such.
|
|
For a team's own packages, the Gitea instance speaks the OCI distribution protocol, so packages can be pushed to and pulled from its package registry as OCI artifacts with ORAS, and signed there with cosign; the media type of the artifact is the package's, `application/vnd.scienting.doc+zip`.
|
|
Nothing in the formats depends on how packages travel.
|