thoughtsync/ci-requirements.md

CI Requirements — ThoughtSync

Spec lives in docs/process.md in the CI-Runner repo.

Runtime image

git.fabledsword.com/bvandeusen/ci-python:3.14

Selected via container.image (not a runs-on label) on all four jobs in .forgejo/workflows/ci.yml: typecheck (Vue/TS), lint (ruff), test (pytest), build (docker buildx).

Image deps used

• python 3.12+ (the runtime Dockerfile targets python:3.12-slim; tests run on the image's 3.14 — both >=3.12, so results stay representative)
• node 24 — npm ci + vue-tsc in the typecheck job, and the frontend builder stage inside the production Dockerfile. (Also required by the JS-based actions/checkout action — a Node-less runner fails every job at checkout.)
• ruff — lint job runs ruff check src/ with zero install overhead
• uv — test job creates the venv (uv venv /opt/venv) and installs the package with dev deps
• docker CLI + buildx — build job pushes the dev/release image to the Fabled-Git registry

Per-job tool installs

Nothing installed at job time beyond what the image provides — all four jobs run entirely on ci-python:3.14.

Notes

• No actions/cache. Deliberately omitted for npm/uv: it's a GitHub-fetched JS action and on a cold runner concurrent jobs race fetching it. We lean on the pinned ci-python image's pre-installed toolchain instead; npm ci / uv pip install cold cost is a non-blocker.

• Build gates on typecheck + lint only. The test job runs in parallel for visibility but does not block the dev image push. DB-backed / integration tests run against the dev image manually — ThoughtSync's unit tests are DB-free (no Postgres service lane in CI yet).

• dev push -> :dev + :<sha>; v* tag -> :latest + :<version> + :<sha> (family rule 46).

• The production runtime Dockerfile tracks python:3.12 so test results stay representative of the deployed image.

• Artifacts — use the mirrored upload action, never actions/upload-artifact.

  uses: https://git.fabledsword.com/bvandeusen/upload-artifact@cb8afe72b42edc798abfb8fcb556cf660d894245
  

  Upstream's actions/upload-artifact@v4 cannot work against this instance and no server-side change will help: its isGhes() rejects any hostname that isn't github.com / *.ghe.com / *.localhost and throws before it opens a connection, so the server is never asked what it supports. @v3 is worse — it reports success, and Gitea then serves artifacts back only through the v4 API (content_encoding = application/zip), so a v3 upload is stored but invisible to every retrieval path. A green job producing nothing retrievable.

  bvandeusen/upload-artifact is our pull mirror of forgejo/upload-artifact (the Forgejo project's fork, one commit on upstream v5.0.0 disabling that check). Mirrored so CI depends on a commit we hold; pinned by SHA because the mirror auto-syncs and a moved upstream tag would otherwise change what runs.

  Both desktop upload steps also set if-no-files-found: error and carry no continue-on-error. They previously had both defaults inverted, which is how 110 unreachable artifacts accumulated on this repo without anyone noticing — the upload could fail or match nothing and the run still went green. Scribe issues 2255 / 2270 have the full teardown.

  Download: GET /api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/artifacts for the id (global run id, not the repo-scoped run number), then …/actions/artifacts/{id}/zip. Note the workstation has no unzip — use python3 -m zipfile -e.

The integration lane

Added 2026-08-23. Before it, alembic upgrade head ran for the first time when the operator's container started — 26 revisions, none of them ever executed by CI — and the schema the migrations build had never been checked against the models that read it. M13 dropped three columns and rebuilt a STORED GENERATED column with nothing watching.

Copied from FabledScribe's integration job, which had already solved the awkward parts. Three of them are family rules for a reason:

• Job key integration, no name: (rule 80). act_runner derives the service container's name from the truncated job DISPLAY name, and the discovery step filters docker ps by it. A spaced or underscored name breaks the filter.
• Service hostnames are not routable on this runner (rule 79), so the step resolves the Postgres container's bridge IP with docker ps --filter + docker inspect and builds THOUGHTSYNC_DATABASE_URL from it. postgres:5432 will not connect.
• run: is busybox sh (rule 81) — no /dev/tcp — so the readiness wait is a small Python heredoc. Its terminator must dedent to column 0 after YAML strips the block indent; check with yaml.safe_load and print the run string if you edit it.

postgres:16-alpine, matching the production compose, so the schema is proven against the Postgres it will actually meet. The schema comes from real migrations, never metadata.create_all (rule 82): testing a schema no deployment has ever seen proves nothing, and that alembic upgrade head step IS the migration test — a broken revision fails the job there, before it can fail a container start.

Tests are marked integration (registered in pyproject.toml); the unit lane runs -m "not integration" and stays DB-free. Data resets with TRUNCATE ... CASCADE BEFORE each test rather than after, so a failure leaves its rows behind to look at.

Like test, it runs for visibility and does not gate the build.

There is no local way to run it — that would mean standing up Postgres on the workstation, which rule 12 reserves for an explicit request. This lane is verified in CI.

Desktop (Tauri) lane — separate workflow

The Tauri desktop client (desktop/) builds in its own workflow, .forgejo/workflows/desktop.yml, NOT in ci.yml — it's a heavy Rust + AppImage build (~20–40 min) that should only run on desktop/** changes, not on every backend/frontend push.

• Image: git.fabledsword.com/bvandeusen/ci-tauri:1.97 (Rust + Node + WebKitGTK 4.1 + Tauri v2 Linux deps + tauri-cli). Selected via container.image; runs-on: python-ci is only a scheduling label.
• Steps: build the shared frontend (embedded by generate_context!) → cargo tauri icon app-icon.png (platform icon set from the committed 1024px source) → cargo clippy --workspace -D warnings → cargo test --workspace → cargo fmt --all --check → cargo tauri build (produces .deb + .AppImage) → de-bundle the AppImage's graphics libs → verify the .deb → repackage for pacman.
• The three analyzer steps run from the REPO ROOT with --workspace, not from desktop/src-tauri. Scoping them to the desktop package was correct while it was the only Rust here; after the core was extracted it silently stopped being — the core's 89 tests stopped running, and a fourth crate would not be linted at all. The dependency crates still COMPILE either way, which is exactly why the gap is invisible from a green run. If you add a workspace member, check that it appears in the cargo test output before believing the lane covers it.
• APPIMAGE_EXTRACT_AND_RUN=1 is set: AppImage tooling FUSE-mounts by default and CI containers have no /dev/fuse.
• Packaging tools used from the image (none installed at job time, rule 5): dpkg-deb / dpkg-query / apt-cache and dpkg-shlibdeps (from dpkg-dev, pulled in by build-essential) for desktop/packaging/deb/verify.sh; tar + a compressor for desktop/packaging/arch/package-prebuilt.sh. Both scripts degrade gracefully rather than hard-failing on an absent optional tool: bsdtar (libarchive-tools) is used for the pacman package's .MTREE when present and skipped when not, compression falls back zstd → xz → gzip, and the .deb clean-container install test runs only if a docker CLI is available. Run 2872 confirmed all three optional tools are ABSENT today, so the current build takes every fallback: the pacman package ships as .pkg.tar.xz with no .MTREE, and the .deb clean-container install test is skipped. All three are functional outcomes — pacman installs an .xz package fine, and only pacman -Qkk file verification needs .MTREE. Adding libarchive-tools + zstd + a docker CLI to ci-tauri would upgrade these paths; none of them block a green build.
• libssl-dev + pkg-config are load-bearing (both already in ci-tauri). Since M10.6 the desktop crate depends on reqwest with the native-tls backend, which on Linux compiles against OpenSSL. Do NOT drop either package from ci-tauri in a future slim-down — the Rust build fails at openssl-sys. (They're part of Tauri's own documented Linux prerequisites, so they should stay regardless.)
• libssl3 is covered transitively, on purpose — don't "fix" it. Since M10.6 dpkg-shlibdeps lists libssl3 among the binary's needs, but the .deb declares only libwebkit2gtk-4.1-0 + libgtk-3-0. verify.sh passes it because webkit's own recursive dependency closure includes OpenSSL, so apt installs it either way. Declaring it explicitly would be worse: the package name is release-dependent (libssl3 on bookworm, libssl3t64 after the 64-bit-time_t transition in trixie/Ubuntu 24.04), so a hardcoded name freezes the package to the build distro. Leaning on webkit's closure adapts. If webkit ever stops pulling OpenSSL, verify.sh fails the build loudly — that guard is what makes the indirection safe.
• Not verifiable in CI: the runner is Debian, so the pacman package cannot be pacman -U-tested here. That step logs .PKGINFO + the full file listing so the package is auditable from the run log; a real Arch install is the operator's confirm.

Windows lane — second job, second image

desktop.yml also runs a windows job that cross-compiles the NSIS installer.

• Image: git.fabledsword.com/bvandeusen/ci-tauri-win:1.97 (Rust + Node + cargo-xwin + LLVM/lld + NSIS). A separate image from ci-tauri per CI-Runner's docs/process.md fork rule — the MSVC CRT/SDK cache alone is >1 GB. Its pins are held in lockstep with ci-tauri; bump them together, since both lanes compile the same source.
• Why cross-compile: there is no Windows build host, and a Windows container cannot run on a Linux host (containers share the host kernel). cargo-xwin, lld-link and makensis are Linux programs that emit Windows PE output.
• NSIS only. .msi requires WiX v3, a Windows program — per Tauri, ".msi installers can only be created on Windows."
• Separate job on purpose: a Windows failure must not block the Linux artifacts, which are the primary product today.
• Weakest verification of any lane. Tauri documents this path as "not as straight forward as compiling on Windows directly and is not tested as much", to be used "only as a last resort" — and a Linux runner cannot execute a Windows binary. Green means it built. A real Windows machine check is mandatory before trusting a release.
• Unsigned. Installers will trip SmartScreen until a code-signing certificate exists; that is a purchasing decision, not a CI one.
• TLS backend is chosen for this lane's sake. The desktop crate pins reqwest to native-tls, which on x86_64-pc-windows-msvc resolves to schannel — pure-Rust bindings to the OS TLS stack. That keeps C/assembly out of the cross-compile entirely. Switching to rustls would pull in ring/aws-lc-rs and their assembler, which is exactly the class of dependency that broke this lane before (libsqlite3-sys → llvm-lib). Treat a TLS-backend change as a change to this lane, not just a dependency bump.
• No Postgres lane (unchanged): the desktop app's local store + sync behavior is verified on the operator's machine, not in CI.

Android lane — being rebuilt (M12)

The Tauri-mobile Android lane is gone. Android is a native Kotlin/Compose client over the shared thoughtsync-core crate instead — see Scribe note 2730 for the decision and milestone M12 for the arc.

The image it will run on already exists: ci-rust-android:1.97, repurposed from ci-tauri-android rather than deleted (CI-runner dc802f2, Scribe #2732). tauri-cli is out and cargo-ndk is in; the NDK binutils symlinks and the PATH append stayed, because they were never Tauri problems — NDK r23 removed the triple-prefixed binutils that autotools, and so vendored OpenSSL, invokes by bare name. It also carries ktlint + detekt so the Kotlin analyzer lane needs no second image, and JDK 25 (which requires Gradle 9.1+ in this repo's wrapper — the old JDK 17 pin existed only because Tauri generated a Gradle 8.x project).

The Rust pin is in LOCKSTEP with ci-tauri and ci-tauri-win. All three build thoughtsync-core from one workspace Cargo.lock under --locked, so a mismatched Rust minor across the lanes would mean divergent resolution for no reason. Bump the three together or not at all.

Checking the Kotlin lane before pushing

Same authorisation and same reasoning as the Rust section below — analyzers, run in the CI image, with the workflow's exact arguments. From android/:

IMG=git.fabledsword.com/bvandeusen/ci-rust-android:1.97
DOCK="docker run --rm --user $(id -u):$(id -g) -e HOME=/tmp -v $PWD:/w -w /w"

$DOCK $IMG ktlint "app/src/main/**/*.kt"
$DOCK $IMG detekt --build-upon-default-config --config config/detekt.yml \
    --input app/src/main/java

HOME=/tmp because both tools want a writable home for their caches and --user has taken the image's away.

Neither of these can see a missing import. They parse Kotlin without resolving symbols, so a file that cannot possibly compile passes both. That is not a gap to work around — it is what these tools are — but it means a clean local run says nothing about whether the code builds. It cost a red CI run on 750d11d, where android.os.Build was lost in a file split and both analyzers were happy.

So there are two more local checks, each covering one blind spot:

python3 android/tools/check-symbols.py
python3 android/tools/check-strings.py

check-symbols.py flags any capitalised identifier that is neither imported, declared in the same package, a type parameter, nor implicitly available — and members of this package's own object declarations, so that Foo.bar() fails here when Foo has no bar. That second case exists because moving a function between two objects and forgetting to paste it into the second cost a red run (785ebdb): the call site was correctly qualified and every other gate passed. Not a type checker — compileDebugKotlin in CI remains the only real one, and it is also the ONLY lane that type-checks at all, since there is no Android SDK on the workstation.

check-strings.py covers resources, where the compiler is no help either: R is generated, so R.string.whatever type-checks whether or not the string exists. It catches a missing name, stringResource used on a plural or the reverse, and a format string that takes more arguments than the call passes — the last of which renders %2$s as literal text rather than failing.

Run all four before a push that touches Kotlin.

A caution worth keeping, because it bit twice: a checker of this shape is itself easy to get vacuously right. The first version stripped line comments with re.sub(r'//.*', src, flags=re.S), and DOTALL makes //.* swallow each file from its first comment to EOF — so it reported everything clean by examining almost nothing. Test a checker against a known-bad tree before trusting a green from it. check-symbols.py is verified by deleting the Build import from a copy of the source; check-strings.py by introducing one of each of its three fault kinds. Its own first version counted Kotlin's trailing commas as arguments and reported three correct call sites as broken — the opposite failure, and the one that teaches you to ignore the tool.

A fourth Kotlin check: read the artifact, don't recall the API

Compose comes from a BOM (compose-bom in libs.versions.toml), so no file in this repo states which material3 a build actually gets. Guessing its API and finding out from CI costs eight minutes a try. Resolve and read it instead:

# androidx is on Google's Maven, NOT Maven Central — repo1 returns 404
BOM=https://dl.google.com/dl/android/maven2/androidx/compose/compose-bom
curl -sS $BOM/2026.05.01/compose-bom-2026.05.01.pom | grep -A3 'material3</artifactId>'

M3=https://dl.google.com/dl/android/maven2/androidx/compose/material3/material3-android
curl -sS -o m3-src.jar $M3/1.4.0/material3-android-1.4.0-sources.jar

The sources jar answers what javap cannot: default arguments, parameter names, and whether a declaration carries @ExperimentalMaterial3Api. That last one is not optional trivia — an unnecessary @OptIn is itself a Kotlin warning, so guessing "safely" breaks the build's zero-warning record just as surely as omitting a required one breaks the build.

Same technique for any dependency. It is how work-runtime-ktx was found to be an empty 6 KB stub as of 2.11, with CoroutineWorker and PeriodicWorkRequestBuilder moved into work-runtime itself.

Checking the Rust lane before pushing

There is no Rust toolchain on the workstation (rule 10) and the desktop lane is verified entirely in CI — but the CI image is pullable, so the three analyzer steps can be run against it locally first. The operator authorised this on 2026-08-18 for fmt, clippy and test; it is not licence to run the bundle build or stand up anything.

Run all three, in this order, before any push that touches Rust:

IMG=git.fabledsword.com/bvandeusen/ci-tauri:1.97
DOCK="docker run --rm --user $(id -u):$(id -g) -e CARGO_HOME=/tmp/cargo -v $PWD:/w -w /w"

$DOCK $IMG cargo fmt --all --check
$DOCK $IMG cargo clippy --locked --workspace --all-targets -- -D warnings
$DOCK $IMG cargo test --locked --workspace

Drop --check from the first to apply it. --user keeps the container from leaving root-owned files behind; CARGO_HOME points somewhere writable for that user. Commands are IDENTICAL to the workflow's, deliberately — a local check that differs from CI is worse than none.

This reproduces CI exactly, not approximately. On the 2026-08-18 run the local test binary hashes (thoughtsync_core-bbaae79723888ad1, thoughtsync_desktop_lib-9d162263f8d0aca3, thoughtsync_ffi-fc557b96dc795e27) matched CI run 3931's byte for byte. Same image, same lockfile, same units.

target/ persists on the host between runs, so after the first cold build these take seconds (~30s for clippy). It is gitignored and reaches ~1.4 GB; delete it whenever the space is wanted.

Run these on every Rust-touching push, not just the ones that feel risky. Four consecutive failures across M13's removals — a private fn deleted along with the pub fn above it, an orphaned #[serde] attribute left where a field was removed, and a test pinning a protocol version literal — were all caught by these three commands in under a minute each, after CI had already found them the slow way. A removal is exactly the kind of change that looks safe and isn't: nothing in the Python or TypeScript lanes compiles Rust, so a break can travel several commits before the first lane that does gets to it.

Don't infer formatting from existing code. Several lines in local/store.rs exceed 100 characters and survive only because rustfmt cannot break a string literal — copying that shape caused one of four consecutive fmt-only CI failures, which is what this whole section exists to prevent.

Checking the frontend lane before pushing

Same technique, same authorisation, same reason — and it covers a gap the Rust gate cannot: vue-tsc --noEmit type-checks only the SCRIPT block, so a malformed TEMPLATE passes the typecheck lane and fails vite build in a different workflow. npm run build runs both, which is exactly what the desktop lanes run.

docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD:/w" -w /w/frontend \
  git.fabledsword.com/bvandeusen/ci-python:3.14 sh -c "npm ci --silent && npm run build"

The typecheck lane uses the ci-python image too — it is the node the frontend jobs already run on, not a separate one. Delete frontend/node_modules and frontend/dist afterwards; both are gitignored, but neither belongs in a working tree that never builds locally otherwise.

The desktop lockfile

Cargo.lock is committed at the workspace root, per Cargo's own guidance for binary crates. Without it every CI run re-resolved the graph, which meant a released .deb/.AppImage/.exe couldn't be rebuilt from its tag, a build could break with no repo change, and Renovate had nothing to bump (issue 2102).

Enforced by --locked on each job's first cargo invocation — cargo clippy --locked on Linux, a dedicated cargo fetch --locked --target x86_64-pc-windows-msvc step on Windows. If the manifest and the lockfile disagree, the run fails there instead of silently re-resolving; everything after it in the same job then compiles the recorded versions, so the flag isn't repeated on the bundle build. The Windows step exists separately because that job's only crate-graph command is the cross-compile itself, and drift is cheaper to learn in the first thirty seconds than thirty minutes in.

To regenerate it after a dependency change — same reasoning as cargo fmt above, and resolution is neither a test run nor a build:

docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
  -v "$PWD:/w" -w /w \
  git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo fetch

cargo fetch, not cargo generate-lockfile. Both update the lockfile, but generate-lockfile re-resolves the whole graph from scratch and will happily bump crates that have nothing to do with your change — turning a two-line manifest edit into a few-hundred-line lockfile diff nobody can review. cargo fetch performs the minimal resolution: existing pins are preserved, only the new entries are added. Verify it stayed additive before committing (git diff Cargo.lock | grep '^-' should show nothing but re-ordered dependency lists).

Resolving inside the CI image rather than against some other cargo is what keeps the lockfile format and the picked versions identical to what CI would have chosen. Commit the result in the same change as the Cargo.toml edit — a manifest change pushed without it fails the gate.

Pushing: dev is both a branch and a tag

git push origin dev fails in this repo:

error: src refspec dev matches more than one

The rolling update channel is a release on a fixed tag named dev (the tag never moves — Fabled-Git has no /releases/latest/download/<asset> route, so the updater needs a permanent URL). Once that tag is fetched locally, the short name dev resolves to both refs/heads/dev and refs/tags/dev. Fully qualify it:

git push origin refs/heads/dev:refs/heads/dev

Shell scripts have no CI lane

Nothing lints desktop/packaging/*.sh, and a broken installer or publish script fails at the moment a user runs it, not in a build. Check them before pushing — install.sh is POSIX sh, the rest are bash:

dash -n desktop/packaging/install.sh          # or: sh -n
bash -n desktop/packaging/publish-release.sh

Where a script resolves URLs from the Fabled-Git API, exercise the resolution against the live instance (plain curl reads, no install) rather than trusting the regex by eye. Both channel paths in install.sh were verified that way.

Hand-assembled JSON: parse it before you push it. publish-release.sh builds its request bodies as shell strings, and quoting context decides what survives into the JSON — a \ inside an unquoted heredoc loses its backslash to the shell, the same \ inside a single-quoted variable does not, and reaches Fabled-Git as an illegal escape (HTTP 422, one wasted build). sh -n cannot see this. Extract the body block and parse it for every branch it can take:

sed -n '/^# The install command printed/,/^JSON$/p' desktop/packaging/publish-release.sh > /tmp/body.sh
echo ')' >> /tmp/body.sh
bash -c 'GITHUB_SERVER_URL=https://git.fabledsword.com GITHUB_REPOSITORY=o/r \
  TAG=dev RELEASE_PRERELEASE=true; . /tmp/body.sh; printf "%s" "$BODY" | python3 -m json.tool >/dev/null'