bvandeusenandClaude Opus 5
5c1ae574f6
The desktop crate is a binary, and binaries commit their lockfile. Without one every run re-resolved the graph: a tagged .deb/.AppImage/.exe couldn't be rebuilt from its tag, any semver-compatible upstream release landed automatically on the next build — the failure mode hardest to read, because the commit that broke it changed nothing relevant — and Renovate had no lockfile to bump, leaving Rust dependency movement invisible to the Dashboard. Generated with cargo generate-lockfile inside ci-tauri:1.97, the same image CI builds in, so the format and the picked versions are what CI would have chosen itself. That takes the artifact-upload round-trip the issue proposed off the table: ci-requirements.md already blesses the image for cargo fmt, and resolving a dependency graph is no more a build than formatting is. 503 packages. Enforcement goes on each job's FIRST cargo invocation rather than the bundle build: cargo clippy --locked on Linux, and its own cargo fetch --locked step on Windows, whose only crate-graph command is otherwise the cross-compile itself. Drift fails in the first thirty seconds instead of thirty minutes in, and everything after the gate in that job compiles the recorded versions anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
257 lines
13 KiB
Markdown
257 lines
13 KiB
Markdown
# CI Requirements — ThoughtSync
|
||
|
||
> Spec lives in [`docs/process.md`](https://git.fabledsword.com/bvandeusen/CI-runner/src/branch/main/docs/process.md)
|
||
> in the CI-Runner repo.
|
||
|
||
## Runtime image
|
||
|
||
```
|
||
git.fabledsword.com/bvandeusen/ci-python:3.14
|
||
```
|
||
|
||
Selected via `container.image` (not a `runs-on` label) on all four jobs in
|
||
`.forgejo/workflows/ci.yml`: typecheck (Vue/TS), lint (ruff), test (pytest),
|
||
build (docker buildx).
|
||
|
||
## Image deps used
|
||
|
||
- python 3.12+ (the runtime `Dockerfile` targets python:3.12-slim; tests run on
|
||
the image's 3.14 — both >=3.12, so results stay representative)
|
||
- node 24 — `npm ci` + `vue-tsc` in the typecheck job, and the frontend builder
|
||
stage inside the production `Dockerfile`. (Also required by the JS-based
|
||
`actions/checkout` action — a Node-less runner fails every job at checkout.)
|
||
- ruff — lint job runs `ruff check src/` with zero install overhead
|
||
- uv — test job creates the venv (`uv venv /opt/venv`) and installs the package
|
||
with dev deps
|
||
- docker CLI + buildx — build job pushes the dev/release image to the Fabled-Git
|
||
registry
|
||
|
||
## Per-job tool installs
|
||
|
||
Nothing installed at job time beyond what the image provides — all four jobs run
|
||
entirely on `ci-python:3.14`.
|
||
|
||
## Notes
|
||
|
||
- **No `actions/cache`.** Deliberately omitted for npm/uv: it's a GitHub-fetched
|
||
JS action and on a cold runner concurrent jobs race fetching it. We lean on the
|
||
pinned `ci-python` image's pre-installed toolchain instead; `npm ci` / `uv pip
|
||
install` cold cost is a non-blocker.
|
||
- Build gates on `typecheck` + `lint` only. The `test` job runs in parallel for
|
||
visibility but does not block the dev image push. DB-backed / integration tests
|
||
run against the dev image manually — ThoughtSync's unit tests are DB-free (no
|
||
Postgres service lane in CI yet).
|
||
- `dev` push -> `:dev` + `:<sha>`; `v*` tag -> `:latest` + `:<version>` + `:<sha>`
|
||
(family rule 46).
|
||
- The production runtime `Dockerfile` tracks python:3.12 so test results stay
|
||
representative of the deployed image.
|
||
- **Artifacts — use the mirrored upload action, never `actions/upload-artifact`.**
|
||
```yaml
|
||
uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
|
||
```
|
||
Upstream's `actions/upload-artifact@v4` cannot work against this instance and
|
||
no server-side change will help: its `isGhes()` rejects any hostname that isn't
|
||
`github.com` / `*.ghe.com` / `*.localhost` and throws before it opens a
|
||
connection, so the server is never asked what it supports. `@v3` is worse — it
|
||
reports success, and Gitea then serves artifacts back only through the v4 API
|
||
(`content_encoding = application/zip`), so a v3 upload is stored but invisible
|
||
to every retrieval path. A green job producing nothing retrievable.
|
||
|
||
`bvandeusen/upload-artifact` is our pull mirror of `forgejo/upload-artifact`
|
||
(the Forgejo project's fork, one commit on upstream v5.0.0 disabling that
|
||
check). Mirrored so CI depends on a commit we hold; pinned by SHA because the
|
||
mirror auto-syncs and a moved upstream tag would otherwise change what runs.
|
||
|
||
Both desktop upload steps also set `if-no-files-found: error` and carry **no**
|
||
`continue-on-error`. They previously had both defaults inverted, which is how
|
||
110 unreachable artifacts accumulated on this repo without anyone noticing —
|
||
the upload could fail or match nothing and the run still went green. Scribe
|
||
issues 2255 / 2270 have the full teardown.
|
||
|
||
Download: `GET /api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/artifacts`
|
||
for the id (global run id, not the repo-scoped run number), then
|
||
`…/actions/artifacts/{id}/zip`. Note the workstation has no `unzip` — use
|
||
`python3 -m zipfile -e`.
|
||
|
||
## Desktop (Tauri) lane — separate workflow
|
||
|
||
The Tauri desktop client (`desktop/`) builds in its own workflow,
|
||
`.forgejo/workflows/desktop.yml`, NOT in `ci.yml` — it's a heavy Rust + AppImage
|
||
build (~20–40 min) that should only run on `desktop/**` changes, not on every
|
||
backend/frontend push.
|
||
|
||
- **Image:** `git.fabledsword.com/bvandeusen/ci-tauri:1.97` (Rust + Node +
|
||
WebKitGTK 4.1 + Tauri v2 Linux deps + `tauri-cli`). Selected via
|
||
`container.image`; `runs-on: python-ci` is only a scheduling label.
|
||
- **Steps:** build the shared frontend (embedded by `generate_context!`) →
|
||
`cargo tauri icon app-icon.png` (platform icon set from the committed 1024px
|
||
source) → `cargo fmt --check` → `cargo clippy -D warnings` → `cargo test` →
|
||
`cargo tauri build` (produces `.deb` + `.AppImage`) → de-bundle the AppImage's
|
||
graphics libs → verify the `.deb` → repackage for pacman.
|
||
- **`APPIMAGE_EXTRACT_AND_RUN=1`** is set: AppImage tooling FUSE-mounts by default
|
||
and CI containers have no `/dev/fuse`.
|
||
- **Packaging tools used from the image** (none installed at job time, rule 5):
|
||
`dpkg-deb` / `dpkg-query` / `apt-cache` and `dpkg-shlibdeps` (from `dpkg-dev`,
|
||
pulled in by `build-essential`) for `desktop/packaging/deb/verify.sh`; `tar` +
|
||
a compressor for `desktop/packaging/arch/package-prebuilt.sh`. Both scripts
|
||
degrade gracefully rather than hard-failing on an absent optional tool:
|
||
`bsdtar` (`libarchive-tools`) is used for the pacman package's `.MTREE` when
|
||
present and skipped when not, compression falls back zstd → xz → gzip, and the
|
||
`.deb` clean-container install test runs only if a docker CLI is available.
|
||
Run 2872 confirmed all three optional tools are ABSENT today, so the current
|
||
build takes every fallback: the pacman package ships as `.pkg.tar.xz` with no
|
||
`.MTREE`, and the `.deb` clean-container install test is skipped. All three are
|
||
functional outcomes — pacman installs an `.xz` package fine, and only
|
||
`pacman -Qkk` file verification needs `.MTREE`. Adding `libarchive-tools` +
|
||
`zstd` + a docker CLI to `ci-tauri` would upgrade these paths; none of them
|
||
block a green build.
|
||
- **`libssl-dev` + `pkg-config` are load-bearing** (both already in `ci-tauri`).
|
||
Since M10.6 the desktop crate depends on `reqwest` with the **`native-tls`**
|
||
backend, which on Linux compiles against OpenSSL. Do NOT drop either package
|
||
from `ci-tauri` in a future slim-down — the Rust build fails at `openssl-sys`.
|
||
(They're part of Tauri's own documented Linux prerequisites, so they should
|
||
stay regardless.)
|
||
- **`libssl3` is covered transitively, on purpose — don't "fix" it.** Since
|
||
M10.6 `dpkg-shlibdeps` lists `libssl3` among the binary's needs, but the
|
||
`.deb` declares only `libwebkit2gtk-4.1-0` + `libgtk-3-0`. `verify.sh` passes
|
||
it because webkit's own recursive dependency closure includes OpenSSL, so apt
|
||
installs it either way. Declaring it explicitly would be *worse*: the package
|
||
name is release-dependent (`libssl3` on bookworm, `libssl3t64` after the
|
||
64-bit-time_t transition in trixie/Ubuntu 24.04), so a hardcoded name freezes
|
||
the package to the build distro. Leaning on webkit's closure adapts. If webkit
|
||
ever stops pulling OpenSSL, `verify.sh` fails the build loudly — that guard is
|
||
what makes the indirection safe.
|
||
- **Not verifiable in CI:** the runner is Debian, so the pacman package cannot be
|
||
`pacman -U`-tested here. That step logs `.PKGINFO` + the full file listing so
|
||
the package is auditable from the run log; a real Arch install is the operator's
|
||
confirm.
|
||
|
||
### Windows lane — second job, second image
|
||
|
||
`desktop.yml` also runs a `windows` job that cross-compiles the NSIS installer.
|
||
|
||
- **Image:** `git.fabledsword.com/bvandeusen/ci-tauri-win:1.97` (Rust + Node +
|
||
`cargo-xwin` + LLVM/`lld` + NSIS). A separate image from `ci-tauri` per
|
||
CI-Runner's `docs/process.md` fork rule — the MSVC CRT/SDK cache alone is >1 GB.
|
||
Its pins are held in lockstep with `ci-tauri`; bump them together, since both
|
||
lanes compile the same source.
|
||
- **Why cross-compile:** there is no Windows build host, and a Windows container
|
||
cannot run on a Linux host (containers share the host kernel). `cargo-xwin`,
|
||
`lld-link` and `makensis` are Linux programs that emit Windows PE output.
|
||
- **NSIS only.** `.msi` requires WiX v3, a Windows program — per Tauri, "`.msi`
|
||
installers can only be created on Windows."
|
||
- **Separate job on purpose:** a Windows failure must not block the Linux
|
||
artifacts, which are the primary product today.
|
||
- **Weakest verification of any lane.** Tauri documents this path as "not as
|
||
straight forward as compiling on Windows directly and is not tested as much",
|
||
to be used "only as a last resort" — and a Linux runner cannot execute a
|
||
Windows binary. Green means it *built*. A real Windows machine check is
|
||
mandatory before trusting a release.
|
||
- **Unsigned.** Installers will trip SmartScreen until a code-signing
|
||
certificate exists; that is a purchasing decision, not a CI one.
|
||
- **TLS backend is chosen for this lane's sake.** The desktop crate pins
|
||
`reqwest` to `native-tls`, which on `x86_64-pc-windows-msvc` resolves to
|
||
`schannel` — pure-Rust bindings to the OS TLS stack. That keeps C/assembly out
|
||
of the cross-compile entirely. Switching to `rustls` would pull in
|
||
`ring`/`aws-lc-rs` and their assembler, which is exactly the class of
|
||
dependency that broke this lane before (`libsqlite3-sys` → `llvm-lib`). Treat
|
||
a TLS-backend change as a change to *this lane*, not just a dependency bump.
|
||
- No Postgres lane (unchanged): the desktop app's local store + sync behavior is
|
||
verified on the operator's machine, not in CI.
|
||
|
||
## Formatting the Rust lane before pushing
|
||
|
||
`cargo fmt --check` runs in CI and had failed on four consecutive desktop pushes
|
||
by itself, each costing a full cycle to learn a whitespace nit. There is no Rust
|
||
toolchain on the workstation (rule 10), but the CI image is pullable, and running
|
||
a formatter is neither a test run nor a local stack:
|
||
|
||
```
|
||
docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
|
||
-v "$PWD/desktop/src-tauri:/w" -w /w \
|
||
git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo fmt --check
|
||
```
|
||
|
||
Drop `--check` to apply. `--user` keeps the container from leaving root-owned
|
||
files behind; `CARGO_HOME` points somewhere writable for that user.
|
||
|
||
**Don't infer formatting from existing code.** Several lines in `local/store.rs`
|
||
exceed 100 characters and survive only because rustfmt cannot break a string
|
||
literal — copying that shape caused one of the four failures.
|
||
|
||
## The desktop lockfile
|
||
|
||
`desktop/src-tauri/Cargo.lock` is **committed**, per Cargo's own guidance for
|
||
binary crates. Without it every CI run re-resolved the graph, which meant a
|
||
released `.deb`/`.AppImage`/`.exe` couldn't be rebuilt from its tag, a build
|
||
could break with no repo change, and Renovate had nothing to bump (issue 2102).
|
||
|
||
Enforced by `--locked` on each job's **first** cargo invocation — `cargo clippy
|
||
--locked` on Linux, a dedicated `cargo fetch --locked --target
|
||
x86_64-pc-windows-msvc` step on Windows. If the manifest and the lockfile
|
||
disagree, the run fails there instead of silently re-resolving; everything after
|
||
it in the same job then compiles the recorded versions, so the flag isn't
|
||
repeated on the bundle build. The Windows step exists separately because that
|
||
job's only crate-graph command is the cross-compile itself, and drift is cheaper
|
||
to learn in the first thirty seconds than thirty minutes in.
|
||
|
||
To regenerate it after a dependency change — same reasoning as `cargo fmt`
|
||
above, and resolution is neither a test run nor a build:
|
||
|
||
```
|
||
docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
|
||
-v "$PWD/desktop/src-tauri:/w" -w /w \
|
||
git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo generate-lockfile
|
||
```
|
||
|
||
Resolving inside the CI image rather than against some other cargo is what keeps
|
||
the lockfile format and the picked versions identical to what CI would have
|
||
chosen. Commit the result in the same change as the `Cargo.toml` edit — a
|
||
manifest change pushed without it fails the gate.
|
||
|
||
## Pushing: `dev` is both a branch and a tag
|
||
|
||
`git push origin dev` fails in this repo:
|
||
|
||
```
|
||
error: src refspec dev matches more than one
|
||
```
|
||
|
||
The rolling update channel is a release on a **fixed tag named `dev`** (the tag
|
||
never moves — Fabled-Git has no `/releases/latest/download/<asset>` route, so the
|
||
updater needs a permanent URL). Once that tag is fetched locally, the short name
|
||
`dev` resolves to both `refs/heads/dev` and `refs/tags/dev`. Fully qualify it:
|
||
|
||
```
|
||
git push origin refs/heads/dev:refs/heads/dev
|
||
```
|
||
|
||
## Shell scripts have no CI lane
|
||
|
||
Nothing lints `desktop/packaging/*.sh`, and a broken installer or publish script
|
||
fails at the moment a user runs it, not in a build. Check them before pushing —
|
||
`install.sh` is POSIX sh, the rest are bash:
|
||
|
||
```
|
||
dash -n desktop/packaging/install.sh # or: sh -n
|
||
bash -n desktop/packaging/publish-release.sh
|
||
```
|
||
|
||
Where a script resolves URLs from the Fabled-Git API, exercise the resolution
|
||
against the live instance (plain `curl` reads, no install) rather than trusting
|
||
the regex by eye. Both channel paths in `install.sh` were verified that way.
|
||
|
||
**Hand-assembled JSON: parse it before you push it.** `publish-release.sh` builds
|
||
its request bodies as shell strings, and quoting context decides what survives
|
||
into the JSON — a `` \ `` inside an unquoted heredoc loses its backslash to the
|
||
shell, the same `` \ `` inside a single-quoted variable does not, and reaches
|
||
Fabled-Git as an illegal escape (HTTP 422, one wasted build). `sh -n` cannot see
|
||
this. Extract the body block and parse it for every branch it can take:
|
||
|
||
```
|
||
sed -n '/^# The install command printed/,/^JSON$/p' desktop/packaging/publish-release.sh > /tmp/body.sh
|
||
echo ')' >> /tmp/body.sh
|
||
bash -c 'GITHUB_SERVER_URL=https://git.fabledsword.com GITHUB_REPOSITORY=o/r \
|
||
TAG=dev RELEASE_PRERELEASE=true; . /tmp/body.sh; printf "%s" "$BODY" | python3 -m json.tool >/dev/null'
|
||
```
|