Files
inkwell/ci-requirements.md
T
bvandeusenandClaude Opus 5.5 f16224b07c
Android / Build, or is the channel already serving this? (push) Successful in 3s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Build now, or wait for Android? (push) Successful in 2s
Desktop (Tauri) / Build, or is the channel already serving this? (push) Successful in 2s
CI & Build / Web typecheck and unit tests (push) Successful in 12s
CI & Build / Python tests (push) Successful in 23s
Android / Core and FFI clippy and tests (push) Successful in 51s
CI & Build / integration (push) Successful in 1m49s
CI & Build / Build & push image (push) Skipped
Desktop (Tauri) / Web tests, clippy, Rust tests and rustfmt (push) Successful in 2m20s
Desktop (Tauri) / Windows installer (cross-compiled) (push) Successful in 3m6s
Desktop (Tauri) / Tauri desktop (Linux) (push) Successful in 3m46s
Desktop (Tauri) / Update manifest (push) Successful in 3s
Android / Kotlin + Rust (APK) (push) Successful in 8m31s
Android / Build the server image (push) Successful in 1s
Root, CI, docs and packaging describe the project to a stranger (#6225)
#6225 "Comment standard sweep: root, config, CI workflows, docs and
packaging", milestone 546 "Bring the repo to the comment standard (rule 32)
and compose defaults (rule 198)".

- README restructured in newcomer order: what it is, quick start,
  configuration, setup pitfalls, updating and backups, features,
  troubleshooting, development, licence. Its quick-start compose carries
  no restart policy.
- Dockerfile and .env.example trimmed to one or two lines per setting.
- Workflows, packaging scripts and docs lose issue, run, note, rule and
  milestone references and their history narration. Comments that still
  described path filters and "step 6" as future work now state what the
  workflows do.
- ci-requirements.md rewritten against the current workflows. It said the
  build gated on typecheck and lint only, the integration lane did not
  gate, a v* tag published :latest, and ci-tauri lacked bsdtar and zstd;
  none of those is true now (the last checked against run 9630's log).
- Echo strings in guard-forward.sh and version.sh drop their note
  citations; every string a test asserts on is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:44:02 -04:00

18 KiB
Raw Blame History

CI Requirements — Inkwell

Spec lives in docs/process.md in the CI-Runner repo.

Runtime images

Image Used by
git.fabledsword.com/bvandeusen/ci-python:3.14 every ci.yml job; the decide jobs in desktop.yml and android.yml
git.fabledsword.com/bvandeusen/ci-tauri:1.97 desktop.yml verify, build, manifest; android.yml rust
git.fabledsword.com/bvandeusen/ci-tauri-win:1.97 desktop.yml windows
git.fabledsword.com/bvandeusen/ci-rust-android:1.97 android.yml build, server-image
postgres:16-alpine service container for ci.yml integration

Each is selected via container.image; runs-on is only a scheduling label.

The Rust pin is in LOCKSTEP across ci-tauri, ci-tauri-win and ci-rust-android. All three build inkwell-core from one workspace Cargo.lock under --locked, so a mismatched Rust minor would mean divergent resolution for no reason. Bump them together or not at all.

Image deps used

ci-python

  • python 3.14 for the tests. The runtime Dockerfile targets python:3.12-slim; both are >=3.12, so results stay representative.
  • node 24 — npm ci, vue-tsc and the web unit tests. Also required by the JS-based actions/checkout: a Node-less runner fails every job at checkout. The Dockerfile's own frontend stage uses node:22-alpine.
  • ruff — the lint job runs ruff check src/ with no install.
  • uv — the test jobs create the venv (uv venv /opt/venv) and install the package with dev deps.
  • docker CLI + buildx — the build job pushes the image to the Fabled-Git registry.
  • curl — packaging/fetch-clients.sh and packaging/guard-forward.sh. The guard refuses to run without it, because every lookup would read as "nothing published".

ci-tauri — Rust, Node, WebKitGTK 4.1, the Tauri v2 Linux deps and tauri-cli.

  • libssl-dev + pkg-config are load-bearing. inkwell-core, which the desktop crate builds in, uses reqwest with the native-tls backend, which on Linux compiles against OpenSSL. Dropping either fails the build at openssl-sys. They are also Tauri's documented Linux prerequisites. This is why android.yml's host-target rust job runs here and not on ci-rust-android, whose OpenSSL is vendored for the Android targets alone.
  • dpkg-deb, dpkg-query, apt-cache and dpkg-shlibdeps (from dpkg-dev, pulled in by build-essential) for desktop/packaging/deb/verify.sh; tar and a compressor for desktop/packaging/arch/package-prebuilt.sh.
  • Optional, used when present: bsdtar (libarchive-tools) for the pacman package's .MTREE, zstd (compression falls back zstd → xz → gzip), and a docker CLI for the .deb clean-container install test. ci-tauri has the first two, so the package ships as .pkg.tar.zst with an .MTREE. It has no docker CLI, so the install test is skipped; adding one would enable it, and its absence does not block a green build.

ci-tauri-win — Rust, Node, cargo-xwin, LLVM/lld and NSIS. A separate image per CI-Runner's fork rule: the MSVC CRT/SDK cache alone is >1 GB.

ci-rust-android — Rust, the four Android ABIs, cargo-ndk, SDK/NDK, JDK 25, ktlint and detekt. The NDK binutils symlinks and PATH append are there because NDK r23 removed the triple-prefixed binutils that autotools, and so vendored OpenSSL, invokes by bare name. JDK 25 requires Gradle 9.1+ in the wrapper. The analyzers are the image's pinned CLIs rather than Gradle plugins, so there is one version of each.

Per-job tool installs

None. Every job runs entirely on its image.

The lanes

ci.yml — on every push to dev and main, and when dispatched. gate, typecheck, lint, test and integration run in parallel, and build needs all five, so nothing publishes on red. dev publishes :dev; main publishes :latest and :<sha>. The image always builds: it is ~15 seconds, and building every push picks up python:3.12-slim updates.

gate stands the build down when the push touched the Android client's file set. The Android lane then dispatches ci.yml once it has published the APK, so the image bakes in the client from the same commit.

desktop.yml — on every push to dev and main.

  • decide reads the file set in packaging/version.sh and skips everything when the channel already serves this source. packaging/guard-forward.sh runs here, so it also covers the skip path.
  • verify builds the frontend, runs the web unit tests, then cargo clippy --locked --workspace, cargo test --workspace and cargo fmt --all --check from the repo root. Run from desktop/src-tauri instead, cargo would scope to the desktop package and the core's tests would silently not run. If you add a workspace member, check that it appears in the cargo test output.
  • build (Linux): cargo tauri build for .deb and AppImage, de-bundles the AppImage's graphics libs and re-signs it, verifies the .deb, repackages for pacman, and publishes to the channel. APPIMAGE_EXTRACT_AND_RUN=1 is set because CI containers have no /dev/fuse.
  • windows: cross-compiles the NSIS installer. There is no Windows build host, and a Windows container cannot run on a Linux host. .msi needs WiX v3, a Windows program, so NSIS only.
  • manifest writes latest.json after both, so it describes both platforms.

android.yml — on every push to dev and main. decide, then rust (the core's clippy and tests, which gate the APK), then build (native library and bindings, ktlint, detekt, host-JVM unit tests, assemble, check the signing certificate, publish), then server-image, which dispatches ci.yml.

release.yml — on a v* tag. Builds nothing; it writes the changelog for the release.

Notes

  • No actions/cache for npm or uv. It is a GitHub-fetched JS action, and on a cold runner concurrent jobs race fetching it. The pinned images' toolchains carry the load; cold npm ci / uv pip install cost is a non-blocker.

  • Artifacts: stock actions/upload-artifact@v7, never @v3. Stock works on this forge because gitea/runner 3.x edits the action's client-side isGhes() refusal out of its bundle (verified for upload-artifact v4–v7 and download-artifact v4–v8). @v3 reports success, but Gitea serves artifacts back only through the v4 API (content_encoding = application/zip), so a v3 upload is stored and invisible to every retrieval path. Upload steps set if-no-files-found: error and carry no continue-on-error, so a failed or empty upload fails the run.

    Download: GET /api/v1/repos/{owner}/{repo}/actions/runs/{run_id}/artifacts for the id (the global run id, not the repo-scoped run number), then …/actions/artifacts/{id}/zip. Without unzip, use python3 -m zipfile -e.

  • libssl3 is covered transitively, on purpose — don't "fix" it. dpkg-shlibdeps lists libssl3 among the binary's needs, but the .deb declares only libwebkit2gtk-4.1-0 + libgtk-3-0. verify.sh passes it because webkit's own dependency closure includes OpenSSL, so apt installs it either way. Declaring it would be worse: the name is release-dependent (libssl3 on bookworm, libssl3t64 after the 64-bit time_t transition in trixie/Ubuntu 24.04). If webkit ever stops pulling OpenSSL, verify.sh fails the build.

  • The Windows TLS backend is chosen for that lane's sake. native-tls on x86_64-pc-windows-msvc resolves to schannel, pure-Rust bindings to the OS TLS stack, which keeps C and assembly out of the cross-compile. rustls would pull in ring/aws-lc-rs and their assembler, the class of dependency that breaks this lane (libsqlite3-sys → llvm-lib). Treat a TLS-backend change as a change to the Windows lane.

  • What CI cannot verify. The runner is Debian, so the pacman package is not pacman -U-tested; the step logs .PKGINFO and the file listing instead. A Linux runner cannot launch a Windows binary or an APK, so green means built: a real Windows machine and a real phone are the check before trusting a release. Windows installers are unsigned and trip SmartScreen.

The integration lane

Real Postgres, so the schema the migrations build is checked against the models that read it before any deployment runs alembic upgrade head.

  • Job key integration, no name:. act_runner derives the service container's name from the truncated job DISPLAY name, and the discovery step filters docker ps by it. A spaced or underscored name breaks the filter.
  • Service hostnames are not routable on this runner, so the step resolves the Postgres container's bridge IP and builds INKWELL_DATABASE_URL from it. postgres:5432 will not connect. The runner's docker daemon is shared, so the lookup is scoped to this task's own containers.
  • run: is busybox sh — no /dev/tcp — so the readiness wait is a small Python heredoc. Its terminator must dedent to column 0 after YAML strips the block indent; check with yaml.safe_load and print the run string if you edit it.
  • Real migrations, never metadata.create_all. The alembic upgrade head step IS the migration test: a broken revision fails the job there, before it can fail a container start.

Tests are marked integration (registered in pyproject.toml); the unit lane runs -m "not integration" and stays DB-free. Data resets with TRUNCATE ... CASCADE BEFORE each test rather than after, so a failure leaves its rows behind to look at.

Checking lanes before pushing

The analyzers and tests below can run locally against the CI images, with the workflow's exact arguments, because a local check that differs from CI is worse than none. They reproduce CI exactly: same image, same lockfile, same units. Nothing else — no bundle builds, no services — is meant to run locally.

Rust

Run all three, in this order, before any push that touches Rust:

IMG=git.fabledsword.com/bvandeusen/ci-tauri:1.97
DOCK="docker run --rm --user $(id -u):$(id -g) -e CARGO_HOME=/tmp/cargo -v $PWD:/w -w /w"

$DOCK $IMG cargo fmt --all --check
$DOCK $IMG cargo clippy --locked --workspace --all-targets -- -D warnings
$DOCK $IMG cargo test --locked --workspace

Drop --check from the first to apply it. --user keeps the container from leaving root-owned files behind; CARGO_HOME points somewhere writable for that user. target/ persists between runs, so after the first cold build these take seconds; it is gitignored and reaches ~1.4 GB.

Run them on every Rust-touching push, not just the risky-looking ones. A removal looks safe and often isn't — a private fn deleted along with the pub fn above it, an orphaned #[serde] attribute — and nothing in the Python or TypeScript lanes compiles Rust, so a break can travel several commits before a lane that does gets to it.

Don't infer formatting from existing code. Some lines in local/store.rs exceed 100 characters and survive only because rustfmt cannot break a string literal.

Kotlin

From android/:

IMG=git.fabledsword.com/bvandeusen/ci-rust-android:1.97
DOCK="docker run --rm --user $(id -u):$(id -g) -e HOME=/tmp -v $PWD:/w -w /w"

$DOCK $IMG ktlint "app/src/main/**/*.kt"
$DOCK $IMG detekt --build-upon-default-config --config config/detekt.yml \
    --input app/src/main/java

HOME=/tmp because both tools want a writable home for their caches and --user has taken the image's away.

Neither can see a missing import. They parse Kotlin without resolving symbols, so a file that cannot compile passes both. Two more checks cover the blind spots:

python3 android/tools/check-symbols.py
python3 android/tools/check-strings.py

check-symbols.py flags any capitalised identifier that is neither imported, declared in the same package, a type parameter, nor implicitly available — and members of this package's own object declarations, so Foo.bar() fails when Foo has no bar. It is not a type checker: compileDebugKotlin in CI is the only one.

check-strings.py covers resources, where the compiler is no help: R is generated, so R.string.whatever type-checks whether or not the string exists. It catches a missing name, stringResource used on a plural or the reverse, and a format string that takes more arguments than the call passes — which renders %2$s as literal text rather than failing.

Run all four before a push that touches Kotlin.

A checker of this shape is easy to get vacuously right: re.sub(r'//.*', ..., flags=re.S) makes //.* swallow each file from its first comment to EOF and report everything clean. Test a checker against a known-bad tree before trusting a green from it. check-symbols.py is verified by deleting an import from a copy of the source; check-strings.py by introducing one of each of its three fault kinds.

Read the artifact, don't recall the API. Compose comes from a BOM (compose-bom in libs.versions.toml), so no file here states which material3 a build gets, and guessing costs eight minutes of CI a try. Resolve and read it:

# androidx is on Google's Maven, NOT Maven Central — repo1 returns 404
BOM=https://dl.google.com/dl/android/maven2/androidx/compose/compose-bom
curl -sS $BOM/2026.05.01/compose-bom-2026.05.01.pom | grep -A3 'material3</artifactId>'

M3=https://dl.google.com/dl/android/maven2/androidx/compose/material3/material3-android
curl -sS -o m3-src.jar $M3/1.4.0/material3-android-1.4.0-sources.jar

The sources jar answers what javap cannot: default arguments, parameter names, and whether a declaration carries @ExperimentalMaterial3Api. An unnecessary @OptIn is itself a Kotlin warning, so guessing "safely" breaks the zero-warning build as surely as omitting a required one. The same works for any dependency — it is how work-runtime-ktx shows up as an empty 6 KB stub as of 2.11, with CoroutineWorker and PeriodicWorkRequestBuilder in work-runtime itself.

Frontend

vue-tsc --noEmit type-checks only the SCRIPT block, so a malformed TEMPLATE passes the typecheck lane and fails vite build in the desktop lanes. npm run build runs both:

docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "$PWD:/w" -w /w/frontend \
  git.fabledsword.com/bvandeusen/ci-python:3.14 sh -c "npm ci --silent && npm run build"

Delete frontend/node_modules and frontend/dist afterwards.

The Cargo lockfile

Cargo.lock is committed at the workspace root, per Cargo's guidance for binary crates, so a build is reproducible from its commit and can't break with no repo change, and Renovate has something to bump.

--locked on each job's first cargo invocation enforces it — cargo clippy --locked on Linux, a dedicated cargo fetch --locked --target x86_64-pc-windows-msvc step on Windows. If the manifest and the lockfile disagree, the run fails there; everything after it in the same job compiles the recorded versions. The Windows step exists because that job's only other crate-graph command is the cross-compile, and drift is cheaper to learn in thirty seconds than thirty minutes in.

To regenerate it after a dependency change:

docker run --rm --user "$(id -u):$(id -g)" -e CARGO_HOME=/tmp/cargo \
  -v "$PWD:/w" -w /w \
  git.fabledsword.com/bvandeusen/ci-tauri:1.97 cargo fetch

cargo fetch, not cargo generate-lockfile. generate-lockfile re-resolves the whole graph and will bump crates unrelated to your change, turning a two-line manifest edit into a few-hundred-line lockfile diff. cargo fetch keeps existing pins and adds only new entries. Check it stayed additive (git diff Cargo.lock | grep '^-' should show nothing but re-ordered dependency lists), and commit it with the Cargo.toml edit — a manifest change pushed without it fails the gate.

Channel releases: dev-rolling and stable

The two update channels are releases on fixed tags, because Fabled-Git has no /releases/latest/download/<asset> route and the updater needs a permanent URL. The channel is called dev everywhere a person sees it; its release tag is dev-rolling, because a tag named dev would shadow the branch and make git push origin dev fail with error: src refspec dev matches more than one. packaging/channel-tag.sh is the one mapping CI reads; desktop/src-tauri/src/update.rs and desktop/packaging/install.sh carry their own copy because neither can run it — change all three together. tests/test_channel_tag.py and the update.rs tests fail if a channel tag is spelled like a branch.

A desktop app installed before 2026-09-10 reads a dev tag that no longer exists and gets no further updates; reinstall it with install.sh.

Shell scripts have no CI lane

Nothing lints desktop/packaging/*.sh, and a broken installer or publish script fails when a user runs it, not in a build. Check them before pushing — install.sh is POSIX sh, the rest are bash:

dash -n desktop/packaging/install.sh          # or: sh -n
bash -n desktop/packaging/publish-release.sh

Where a script resolves URLs from the Fabled-Git API, exercise the resolution against the live instance (plain reads, no install) rather than trusting the regex by eye.

Hand-assembled JSON: parse it before you push it. publish-release.sh builds its request bodies as shell strings, and quoting context decides what survives into the JSON — a \ inside an unquoted heredoc loses its backslash to the shell, the same \ inside a single-quoted variable does not, and reaches Fabled-Git as an illegal escape (HTTP 422). sh -n cannot see this. Extract the body block and parse it for every branch it can take:

sed -n '/^# The install command printed/,/^JSON$/p' desktop/packaging/publish-release.sh > /tmp/body.sh
echo ')' >> /tmp/body.sh
bash -c 'GITHUB_SERVER_URL=https://git.fabledsword.com GITHUB_REPOSITORY=o/r \
  TAG=dev RELEASE_PRERELEASE=true; . /tmp/body.sh; printf "%s" "$BODY" | python3 -m json.tool >/dev/null'