thoughtsync/ci-requirements.md

CI Requirements — ThoughtSync

Spec lives in docs/process.md in the CI-Runner repo.

Runtime image

git.fabledsword.com/bvandeusen/ci-python:3.14

Selected via container.image (not a runs-on label) on all four jobs in .forgejo/workflows/ci.yml: typecheck (Vue/TS), lint (ruff), test (pytest), build (docker buildx).

Image deps used

• python 3.12+ (the runtime Dockerfile targets python:3.12-slim; tests run on the image's 3.14 — both >=3.12, so results stay representative)
• node 24 — npm ci + vue-tsc in the typecheck job, and the frontend builder stage inside the production Dockerfile. (Also required by the JS-based actions/checkout action — a Node-less runner fails every job at checkout.)
• ruff — lint job runs ruff check src/ with zero install overhead
• uv — test job creates the venv (uv venv /opt/venv) and installs the package with dev deps
• docker CLI + buildx — build job pushes the dev/release image to the Fabled-Git registry

Per-job tool installs

Nothing installed at job time beyond what the image provides — all four jobs run entirely on ci-python:3.14.

Notes

• No actions/cache. Deliberately omitted for npm/uv: it's a GitHub-fetched JS action and on a cold runner concurrent jobs race fetching it. We lean on the pinned ci-python image's pre-installed toolchain instead; npm ci / uv pip install cold cost is a non-blocker.

• Build gates on typecheck + lint only. The test job runs in parallel for visibility but does not block the dev image push. DB-backed / integration tests run against the dev image manually — ThoughtSync's unit tests are DB-free (no Postgres service lane in CI yet).

• dev push -> :dev + :<sha>; v* tag -> :latest + :<version> + :<sha> (family rule 46).

• The production runtime Dockerfile tracks python:3.12 so test results stay representative of the deployed image.

• Artifacts — use the mirrored upload action, never actions/upload-artifact.

  uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
  

  Upstream's actions/upload-artifact@v4 cannot work against this instance and no server-side change will help: its isGhes() rejects any hostname that isn't github.com / *.ghe.com / *.localhost and throws before it opens a connection, so the server is never asked what it supports. @v3 is worse — it reports success, and Gitea then serves artifacts back only through the v4 API (content_encoding = application/zip), so a v3 upload is stored but invisible to every retrieval path. A green job producing nothing retrievable.

  bvandeusen/upload-artifact is our pull mirror of forgejo/upload-artifact (the Forgejo project's fork, one commit on upstream v5.0.0 disabling that check). Mirrored so CI depends on a commit we hold; pinned by SHA because the mirror auto-syncs and a moved upstream tag would otherwise change what runs.

  Both desktop upload steps also set if-no-files-found: error and carry no continue-on-error. They previously had both defaults inverted, which is how 110 unreachable artifacts accumulated on this repo without anyone noticing — the upload could fail or match nothing and the run still went green. Scribe issues 2255 / 2270 have the full teardown.

  Download: GET /api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/artifacts for the id (global run id, not the repo-scoped run number), then …/actions/artifacts/{id}/zip. Note the workstation has no unzip — use python3 -m zipfile -e.

Desktop (Tauri) lane — separate workflow

The Tauri desktop client (desktop/) builds in its own workflow, .forgejo/workflows/desktop.yml, NOT in ci.yml — it's a heavy Rust + AppImage build (~20–40 min) that should only run on desktop/** changes, not on every backend/frontend push.

• Image: git.fabledsword.com/bvandeusen/ci-tauri:1.97 (Rust + Node + WebKitGTK 4.1 + Tauri v2 Linux deps + tauri-cli). Selected via container.image; runs-on: python-ci is only a scheduling label.
• Steps: build the shared frontend (embedded by generate_context!) → cargo tauri icon app-icon.png (platform icon set from the committed 1024px source) → cargo fmt --check → cargo clippy -D warnings → cargo test → cargo tauri build (produces .deb + .AppImage) → de-bundle the AppImage's graphics libs → verify the .deb → repackage for pacman.
• APPIMAGE_EXTRACT_AND_RUN=1 is set: AppImage tooling FUSE-mounts by default and CI containers have no /dev/fuse.
• Packaging tools used from the image (none installed at job time, rule 5): dpkg-deb / dpkg-query / apt-cache and dpkg-shlibdeps (from dpkg-dev, pulled in by build-essential) for desktop/packaging/deb/verify.sh; tar + a compressor for desktop/packaging/arch/package-prebuilt.sh. Both scripts degrade gracefully rather than hard-failing on an absent optional tool: bsdtar (libarchive-tools) is used for the pacman package's .MTREE when present and skipped when not, compression falls back zstd → xz → gzip, and the .deb clean-container install test runs only if a docker CLI is available. Run 2872 confirmed all three optional tools are ABSENT today, so the current build takes every fallback: the pacman package ships as .pkg.tar.xz with no .MTREE, and the .deb clean-container install test is skipped. All three are functional outcomes — pacman installs an .xz package fine, and only pacman -Qkk file verification needs .MTREE. Adding libarchive-tools + zstd + a docker CLI to ci-tauri would upgrade these paths; none of them block a green build.
• libssl-dev + pkg-config are load-bearing (both already in ci-tauri). Since M10.6 the desktop crate depends on reqwest with the native-tls backend, which on Linux compiles against OpenSSL. Do NOT drop either package from ci-tauri in a future slim-down — the Rust build fails at openssl-sys. (They're part of Tauri's own documented Linux prerequisites, so they should stay regardless.)
• libssl3 is covered transitively, on purpose — don't "fix" it. Since M10.6 dpkg-shlibdeps lists libssl3 among the binary's needs, but the .deb declares only libwebkit2gtk-4.1-0 + libgtk-3-0. verify.sh passes it because webkit's own recursive dependency closure includes OpenSSL, so apt installs it either way. Declaring it explicitly would be worse: the package name is release-dependent (libssl3 on bookworm, libssl3t64 after the 64-bit-time_t transition in trixie/Ubuntu 24.04), so a hardcoded name freezes the package to the build distro. Leaning on webkit's closure adapts. If webkit ever stops pulling OpenSSL, verify.sh fails the build loudly — that guard is what makes the indirection safe.
• Not verifiable in CI: the runner is Debian, so the pacman package cannot be pacman -U-tested here. That step logs .PKGINFO + the full file listing so the package is auditable from the run log; a real Arch install is the operator's confirm.

Windows lane — second job, second image

desktop.yml also runs a windows job that cross-compiles the NSIS installer.

• Image: git.fabledsword.com/bvandeusen/ci-tauri-win:1.97 (Rust + Node + cargo-xwin + LLVM/lld + NSIS). A separate image from ci-tauri per CI-Runner's docs/process.md fork rule — the MSVC CRT/SDK cache alone is >1 GB. Its pins are held in lockstep with ci-tauri; bump them together, since both lanes compile the same source.
• Why cross-compile: there is no Windows build host, and a Windows container cannot run on a Linux host (containers share the host kernel). cargo-xwin, lld-link and makensis are Linux programs that emit Windows PE output.
• NSIS only. .msi requires WiX v3, a Windows program — per Tauri, ".msi installers can only be created on Windows."
• Separate job on purpose: a Windows failure must not block the Linux artifacts, which are the primary product today.
• Weakest verification of any lane. Tauri documents this path as "not as straight forward as compiling on Windows directly and is not tested as much", to be used "only as a last resort" — and a Linux runner cannot execute a Windows binary. Green means it built. A real Windows machine check is mandatory before trusting a release.
• Unsigned. Installers will trip SmartScreen until a code-signing certificate exists; that is a purchasing decision, not a CI one.
• TLS backend is chosen for this lane's sake. The desktop crate pins reqwest to native-tls, which on x86_64-pc-windows-msvc resolves to schannel — pure-Rust bindings to the OS TLS stack. That keeps C/assembly out of the cross-compile entirely. Switching to rustls would pull in ring/aws-lc-rs and their assembler, which is exactly the class of dependency that broke this lane before (libsqlite3-sys → llvm-lib). Treat a TLS-backend change as a change to this lane, not just a dependency bump.
• No Postgres lane (unchanged): the desktop app's local store + sync behavior is verified on the operator's machine, not in CI.

Formatting the Rust lane before pushing

cargo fmt --check runs in CI and had failed on four consecutive desktop pushes by itself, each costing a full cycle to learn a whitespace nit. There is no Rust toolchain on the workstation (rule 10), but the CI image is pullable, and running a formatter is neither a test run nor a local stack:

docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
  -v "$PWD/desktop/src-tauri:/w" -w /w \
  git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo fmt --check

Drop --check to apply. --user keeps the container from leaving root-owned files behind; CARGO_HOME points somewhere writable for that user.

Don't infer formatting from existing code. Several lines in local/store.rs exceed 100 characters and survive only because rustfmt cannot break a string literal — copying that shape caused one of the four failures.

The desktop lockfile

desktop/src-tauri/Cargo.lock is committed, per Cargo's own guidance for binary crates. Without it every CI run re-resolved the graph, which meant a released .deb/.AppImage/.exe couldn't be rebuilt from its tag, a build could break with no repo change, and Renovate had nothing to bump (issue 2102).

Enforced by --locked on each job's first cargo invocation — cargo clippy --locked on Linux, a dedicated cargo fetch --locked --target x86_64-pc-windows-msvc step on Windows. If the manifest and the lockfile disagree, the run fails there instead of silently re-resolving; everything after it in the same job then compiles the recorded versions, so the flag isn't repeated on the bundle build. The Windows step exists separately because that job's only crate-graph command is the cross-compile itself, and drift is cheaper to learn in the first thirty seconds than thirty minutes in.

To regenerate it after a dependency change — same reasoning as cargo fmt above, and resolution is neither a test run nor a build:

docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
  -v "$PWD/desktop/src-tauri:/w" -w /w \
  git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo generate-lockfile

Resolving inside the CI image rather than against some other cargo is what keeps the lockfile format and the picked versions identical to what CI would have chosen. Commit the result in the same change as the Cargo.toml edit — a manifest change pushed without it fails the gate.

Pushing: dev is both a branch and a tag

git push origin dev fails in this repo:

error: src refspec dev matches more than one

The rolling update channel is a release on a fixed tag named dev (the tag never moves — Fabled-Git has no /releases/latest/download/<asset> route, so the updater needs a permanent URL). Once that tag is fetched locally, the short name dev resolves to both refs/heads/dev and refs/tags/dev. Fully qualify it:

git push origin refs/heads/dev:refs/heads/dev

Shell scripts have no CI lane

Nothing lints desktop/packaging/*.sh, and a broken installer or publish script fails at the moment a user runs it, not in a build. Check them before pushing — install.sh is POSIX sh, the rest are bash:

dash -n desktop/packaging/install.sh          # or: sh -n
bash -n desktop/packaging/publish-release.sh

Where a script resolves URLs from the Fabled-Git API, exercise the resolution against the live instance (plain curl reads, no install) rather than trusting the regex by eye. Both channel paths in install.sh were verified that way.

Hand-assembled JSON: parse it before you push it. publish-release.sh builds its request bodies as shell strings, and quoting context decides what survives into the JSON — a \ inside an unquoted heredoc loses its backslash to the shell, the same \ inside a single-quoted variable does not, and reaches Fabled-Git as an illegal escape (HTTP 422, one wasted build). sh -n cannot see this. Extract the body block and parse it for every branch it can take:

sed -n '/^# The install command printed/,/^JSON$/p' desktop/packaging/publish-release.sh > /tmp/body.sh
echo ')' >> /tmp/body.sh
bash -c 'GITHUB_SERVER_URL=https://git.fabledsword.com GITHUB_REPOSITORY=o/r \
  TAG=dev RELEASE_PRERELEASE=true; . /tmp/body.sh; printf "%s" "$BODY" | python3 -m json.tool >/dev/null'