12 Commits

Include renames

Author	SHA1	Message	Date
Bryan Van Deusen	ff6e99eb62	image: bake every client in, not just the phone ... ~104 MB on top of ~85 MB, almost all of it the AppImage. That is what the product being complete costs (rule 23): a self-hoster gets a working app for their machine from the server holding their notes, with no account on a forge that is private. The AppImage is not optional within that — it is the only bundle that can replace itself in place, so a server without one cannot serve in-app updates to anybody. `packaging/fetch-clients.sh` replaces the inline fetch and writes the fixed names and sidecars `client_dist.py` reads. It never fails: a platform with nothing published means the server advertises nothing for it and the UI hides that download, and eight fetches must not become eight ways to redden a green lane. THE VERSION IS FETCHED, NOT DERIVED, and this is the part that would have been wrong the easy way. The obvious shortcut is `version.sh display desktop` in the image job — it has the checkout. But this commit may not be the commit the channel is serving: a push touching only `src/` does not rebuild the desktop, so the channel still holds an older build and a locally-derived version would describe those bytes with this commit's number. `client_dist.py`'s size check could not catch it, because size IS measured from the real file — it would sail through and lie about the version alone. So `write-manifest.sh` now publishes `thoughtsync-desktop.json` beside `latest.json`, from the same two values in the same breath, and only size/sha256 are measured at bake time. Which needed the prune's keep-list, or the sidecar would have been uploaded and deleted again in the same run — a fixed name is self-limiting, which is exactly why that list exists. `version_code` is NOT uniformly an integer, and coercing it was a leftover from the days when Android was the only platform. Android's must stay a JSON number: `ClientRelease` in core declares it `i64` and a string fails to deserialize on every phone in the field. The desktop's is Tauri's semver key `1.0.<minutes>` — the value its updater actually compares — and `int()` would have rejected every desktop sidecar CI writes. The table now says which is which, and tests pin both directions. Also retires the comment above the fetch step, which claimed the APK came from "always the rolling dev release" and mentioned `:<version>` images. M314 step 3 made the channel conditional in the code directly below it, and step 6 removed version-shaped image tags entirely. Verified against the live dev channel before pushing: the Android half resolves and exits 0, the desktop half degrades with a warning because the sidecar does not exist yet, and all five constructed bundle filenames return 200.	2026-08-30 13:11:04 -04:00
bvandeusenandClaude Opus 5	fa43c2f4e9	ci: a docs-only merge to main produced no image, so no :<sha> for that commit ... Rule 145 promises every push to `main` publishes a `:<sha>`, so any production commit is addressable without a release ceremony. `ci.yml`'s `paths:` filter quietly broke that: a commit touching only docs never triggered the lane, so that commit had no image and no sha tag. Pre-existing — the filter has always been there — but it is rule 145's guarantee and step 6 is where the tag set is being made to match the rule, so it is this step's to close. Confirmed live on a0c789b: a docs-only push produced two runs, both client lanes skipping correctly, and NO image at all. The server image now always builds. It is the cheap one — ~15 seconds against 6 and 9 minutes for the clients, which is exactly why they skip and it does not — and always building is what keeps `python:3.12-slim` fresh on something that can face the internet. That is also why §4's base-image tension does not bite this project: the artifact it would apply to is the one that never skips. #3146 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 00:32:27 -04:00
bvandeusenandClaude Opus 5	22a9a279b1	ci: one definition of what ships decides both the version and whether to build ... Step 6 of M314. Two changes that only make sense together. ## The image tag set rule 145 mandates dev push -> :dev main push -> :latest + :<sha> a v* tag -> nothing; the trigger is gone `:<sha>` was going out on EVERY branch — a rollback target nobody has ever pulled, accumulating forever, for a channel whose entire contract is that it moves. It is on main only now, where rollback matters and where gated merges (rule 2) make it dozens per year rather than one per push. No version-shaped image tag in any lane. Verified the way rule 145 asks — by looking for a CONSUMER, not for whether one is imaginable: `docker-compose.yml` is parameterised for a pin and the docs describe the option, but no compose file, deploy script or CI job reads one. ## Skip-if-exists, adapted, because §4 assumes a registry §5 removed Note 3127 §4 says to ask the registry whether that exact version exists. There is no `:<version>` tag to ask about any more. What there IS, for both clients, is a channel that publishes the version it serves — and that answers the same question: if the channel already serves what this source derives, the artifact would be byte-identical. So the `paths:` filters are gone from the desktop and Android lanes, replaced by a `decide` job reading the real file set. That duplication is not theoretical: `packaging/` was added to the sets and not to the filters, so the commit that fixed a derivation bug never ran on the two lanes it fixed (85ead4d). One definition, one reader. The cost is that both workflows now start on every push rather than a matching one — a ~15s container for a decision, against a lane that cannot silently fail to run. ## The server always builds, deliberately Its image is ~15 seconds against 6 and 9 minutes for the clients, so there is little to save. And always building is strictly BETTER for something that can face the internet: it picks up `python:3.12-slim` base updates on every push. That also dissolves §4's base-image tension for this project rather than deciding it — the artifact most exposed to base staleness is the one that never skips. Resolving a base digest at derive time was the alternative and it is forbidden: §7's corollary bars an external lookup, because two lanes would then derive different values for one source. ## The guard runs on the skip path It moved into `decide`, ahead of the decision. §6.3 is explicit that skipping because "this version already exists" is indistinguishable from "we derived a stale value that happens to match" unless something checks. It also now runs once per lane instead of once per job. ## Two defects found while wiring this `ci.yml`'s gate greps a path list that MUST match Android's file set, and `packaging/` was missing from it. A packaging-only push would have had the Android lane build and dispatch while the gate ALSO let the image through — two images for one commit, and on main a second push of the same `:<sha>` with different bytes. Rule 145's exact prohibition. `guard-forward.sh` ends every fetch in `|| true`, so a runner image without curl would have read as "nothing published yet" and passed without checking anything. Missing curl is now fatal. #3146 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 00:19:16 -04:00
bvandeusenandClaude Opus 5	6e891357ff	ci: the deriver is in the file sets but was not in the path filters ... `85ead4d` changed `packaging/version.sh` — the script that decides what every artifact claims to be — and the desktop and Android lanes did not run at all. Only CI & Build fired, and only because it happens to watch `tests/**`. So the fix in that commit is unverified on exactly the two lanes whose bug it was fixing. `version.sh` lists `packaging` in all three file sets; the workflows' `paths:` filters did not. Two places holding one decision, with one of them updated — the failure this subsystem keeps producing (#2181-2183, and again in step 3 where `install.sh` still expected stable's bundles on a versioned release). The script's own header already warned about this: "a change here that is not mirrored there means a lane that does not fire — check both." Written, then not followed, in the same commit. Step 6 removes the duplication for real by replacing these filters with skip-if-exists. This is the stopgap until then, and it says so at each site. #3144 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-28 21:11:26 -04:00
bvandeusenandClaude Opus 5	c5044339a1	versioning: each artifact derives from its own files, with the clock picked per value ... Step 4 of M314. `desktop/packaging/build-version.sh` was one generator feeding the desktop bundles AND the Android APK off `GITHUB_RUN_NUMBER`, so a Kotlin-only commit re-versioned the desktop and a Rust-only commit re-versioned the phone. Note 3127 §3 cites this repo as its example of that failure. It is replaced by `packaging/version.sh` — one definition of HOW to derive, three file sets, and the sets in one place. Lives at the repo root rather than under desktop/, because it serves three artifacts now and a shared thing filed under one consumer ends up owned by it. ## Two values, and the clock chosen per value (§2) desktop key 1.0.<minutes since 2020-01-01> commit time desktop display 2026.08.28.0900 commit time (#3181 shows it) android versionName commit time android versionCode <minutes since 2020> BUILD time server version 2026.08.28.0900 commit time, no ordering key Every human-readable version in the repo is now one shape. The two exceptions are not version names at all — they are bare monotonic integers a comparator reads and nobody quotes. The desktop needs a separate key because Tauri parses `latest.json` with the semver crate and `2026.08.28.0900` fails it twice (four segments, and `08` is a leading zero). `1.0.` and not `0.0.`: the minor has to clear the installed `0.2.466` line or every dev user is stranded on "up to date" permanently. Android's code comes from BUILD time while the desktop's key comes from COMMIT time, deliberately. Android hard-fails a downgrade with INSTALL_FAILED_VERSION_DOWNGRADE and leaves a channel you cannot get out of, so its key must be monotonic by construction; the desktop merely declines to offer an update, which a guard can catch. ## The bug this found in itself The shallow-clone guard `exit 1`-ed inside a function called as `$(...)` — which ends the SUBSHELL, not the script. `display` still failed, but only because `date` then choked on the empty string. `key` printed the error to stderr, emitted `1.0.-26297280`, and exited ZERO. That is precisely the failure the guard exists to prevent: a too-low version on a green lane, and too-low is the direction you cannot recover from. It resolves into a global in the parent shell now. The test is parametrized over both requests, because one path was covered and the other was broken in exactly the way the covered one was meant to rule out. ## Also `fetch-depth: 0` on every job that derives — four of them, and only ci.yml's gate had it. Depth-1 is silently wrong rather than loudly broken (§6.1). The file sets include each artifact's BUILD RECIPE (its workflow, and `packaging/`). A workflow file is not shipped, but change a Gradle flag and the bytes change while the source does not — and once step 6 skips a build whose version already exists, that serves the OLD artifact on a green run. The base images are deliberately NOT resolved at derive time: that is an external lookup, which §7's corollary forbids. `Dockerfile` is already in the server's set, so pinning `FROM` by digest in step 6 puts the base inside the set for free. `build-version.sh` is deleted, its last consumer (the pacman packager) moved over, and the one finding worth keeping out of its header — why not a `-dev.N` prerelease — is preserved in the successor. #3144 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-28 20:51:39 -04:00
bvandeusenandClaude Opus 5	c268ae4f23	ci: main publishes, so a tag stops being required — and :latest stops shipping a dev client ... Step 3 of M314. Note 3127 §0's diagnostic is "is `main` publishing sufficient for a user to receive the build" — and here it was not. The desktop and Android lanes BUILT on main and published nothing: `Publish release` was gated on `refs/tags/v*`, the channel publishes on `refs/heads/dev`, the manifest job on dev-or-tag. So the stable channel moved only when somebody cut a tag, which made a `v*` tag load-bearing rather than the optional bookmark the model wants. Both channels are rolling fixed-tag releases now. `dev` from dev, `stable` from main, same machinery — `publish-release.sh` already took RELEASE_TAG, `write-manifest.sh` already pruned, and both already PATCHed a stale description on 409 (#2182). This is wiring, not new mechanism. ## The defect this carried `ci.yml`'s "Fetch the Android client to bake in" read `releases/download/dev` UNCONDITIONALLY, on every branch. Every image baked in the dev APK — `:latest` included — so a stable server served a dev-channel client to anyone who downloaded it from there. That has nothing to do with versioning; it is fixed here because this is the step that finally gives `stable` an APK to point at. It also means Android needs no channel machinery of its own. The APK is served FROM the image, so the channel is already a property of which image you run — note 3127 §7's "nothing to hand off" shape, arrived at here by accident. One branch-conditional line, not a second channel in `client_dist.py` as this milestone first assumed. ## The break this nearly shipped `install.sh --channel stable` read the version out of `stable/latest.json` and then fetched `releases/tags/v<version>` for the bundles — correct while stable was a manifest-only pointer, and broken the moment stable holds its own. Stable is the DEFAULT channel, so `curl … | sh` would have failed for everyone between this commit and the first merge to main. Both channels are one lookup now: fetch the fixed-tag release, install what is on it. A transitional fallback covers the window where `stable` still has no bundles, marked for deletion in step 7 — without it the default channel is broken for however long it takes to merge, and that window is gated on an operator request rather than on this lane. ## The two writers problem `stable`'s manifest was written by tag builds. It is written by main now, and the tag path stops writing it — two writers for one channel is a race with no winner worth having. A `v*` tag still writes its own versioned manifest; its build consequence goes entirely in step 7. Also corrected: `update.rs`'s header still described stable as following `v*` tags. Nothing in that file moved — it only ever read `<channel>/latest.json` — but the comment was a lie, and it is the file somebody reads to understand the feed. #3143 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-28 18:22:31 -04:00
bvandeusenandClaude Opus 5	77c5422951	ci: a failing lane must not publish an image ... build gated on lint + typecheck only, so run 4293 failed its test lane and pushed :dev and :09b5f87 regardless — the deployed server was running a build whose tests were red. The comment justified this by saying DB-backed testing happened manually against the dev image rather than on every push. That was true when it was written and stopped being true at 6f21db8, which added the integration lane. The reason went away; the exception didn't. Gate on test and integration too. A :<sha> image is the rollback unit for its commit (family rule 46) — one publishable from a failing run is not something you can roll back to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 16:52:59 -04:00
bvandeusen	6f21db85a1	ci: an integration lane, so the migrations are finally run by something ... 26 Alembic revisions and none had ever been executed by CI. `alembic upgrade head` ran for the first time when the operator's container started, and the schema the migrations build had never been checked against the models that read it. M13 dropped three columns and rebuilt a STORED GENERATED column with nothing watching but a server boot. Copied from FabledScribe's `integration` job, which had already solved the parts that are easy to get wrong — and which are family rules precisely because they were: a separator-free job key with no `name:` (act_runner derives the service container name from the truncated display name, and the discovery step filters `docker ps` by it), bridge-IP resolution because service hostnames aren't routable on this runner, and a Python readiness wait because `run:` is busybox sh with no `/dev/tcp`. `postgres:16-alpine` to match the production compose. The schema is built by real migrations, never metadata.create_all — that step IS the migration test. Six tests, each pinning something that has only ever been checked by hand: - an ORM insert against the migrated schema, which is the model/migration agreement nothing has verified until now; - `notes.title`, `notes.kind` and `note_revisions.title` are actually gone, and `note_links` with them — a silently no-op migration shows up here; - the rebuilt `search_vector` indexes both the name and the body, which matters because 0026 had to DROP and recreate a generated column rather than alter it; - a note keeps its body AND its items, the shape step 2 made normal; - `_apply_note_items` leaves items alone when a change doesn't mention them — the data-loss path step 2 removed, pinned so its return would be caught; - a note with no body is still named by its first item, the hole that made removing the title unsafe until checklists stopped being their own kind. Runs for visibility; does not gate the build, matching `test` and Scribe. No local equivalent: running it means standing up Postgres on the workstation, which rule 12 reserves for an explicit request. Documented in ci-requirements alongside the Rust, Kotlin and frontend gates.	2026-08-23 00:24:57 -04:00
bvandeusen	0cf77336d4	ci: build the server image after the Android lane, not alongside it ... Baking the newest client into every image left two holes, both raised by the operator. **An Android-only push never rebuilt the image.** `ci.yml` does not trigger on `android/**`, so a new APK could be published and no image would ever pick it up until some unrelated server change came along. **A push touching both raced.** Both workflows start at once; the image build would fetch the PREVIOUS client and there would be no second build to correct it — `:<sha>` is the immutable rollback unit (rule 46), so rebuilding it with different content would make it neither immutable nor a rollback unit. Ordering now runs the other way: the Android lane finishes, then calls the image build. `ci.yml` gains a `gate` job that stands down on any push touching the Android app, and `android.yml` dispatches `ci.yml` when it is done. One image per commit, containing the client from that commit. Cases: - **server only** — ci builds immediately; the newest published client is already the right one. - **Android only** — ci does not trigger at all; the Android lane dispatches it afterwards. - **both** — ci's push run stands down, the Android lane dispatches it. Exactly one image. - **tag** — always builds. The Android lane does not run on tags, so waiting for a call that never comes would mean a release tag with no image. The dispatch is `always()`, so a FAILED Android build still lets the server image through with the previous client. The alternative is a broken Android lane silently blocking server delivery, which is a worse failure than a slightly old APK. Two details that would each have made this quietly wrong: The gate diffs the whole PUSHED RANGE (`event.before..HEAD`, full fetch), not `HEAD^..HEAD`. A three-commit push whose Android change sat in the first would otherwise have looked Android-free and raced anyway — silently, which is the worst version of this bug. The dispatch is `curl -fsS`, not `|| true`. If that call ever stops working the symptom is server images silently never being built for Android pushes, which nobody would notice until wondering why the app stopped updating. The gate's path list has to match android.yml's trigger, and two places holding one decision is the recurring failure in this repo (issues 2181-2183). It is a `git diff` rather than a config precisely so the decision is visible in the log, and both sides carry a comment pointing at the other.	2026-08-20 21:46:05 -04:00
bvandeusen	010e9a2f85	server: bake the newest Android client into every image (operator call) ... Reverses the placement decision made an hour ago. That one put the APK only on the data volume, reasoning that ~55 MiB should not be charged to installs that never touch Android. The operator's call is that ending the manual copy is worth the megabytes, and it is their deployment. CI now fetches the newest published client into the build context immediately before the image build, so `:dev`, `:latest` and `:<version>` all ship one and a `docker compose pull` delivers a new server and a new client together. **Always the rolling `dev` release — the newest build there is.** A versioned image therefore carries the newest client rather than one pinned to that version. Deliberate: the two negotiate a sync protocol version before linking, so a mismatch is caught by the handshake, and pinning would buy nothing the handshake does not already provide. **Fetched by the JOB, never by the Dockerfile.** The release is private, and a token used inside a build ends up in the context or a layer. **It cannot fail the image build.** No release yet, a network blip, a first-ever build — all of them log a warning and produce an image with no client, which is a state the server already supports. Half a pair is cleaned up rather than shipped: a sidecar without its APK is worse than neither, because the server would be describing something it cannot serve. **The volume still wins.** `DATA_DIR/client/` is checked first and the baked copy second, so an operator who deliberately drops a build in gets that build — and a BROKEN drop-in falls through to the image's copy rather than taking the feature offline, which is what makes the copy-order advice survivable instead of load-bearing. Three tests cover the precedence, including that last case. The baked copy lives inside the package, not under DATA_DIR: that path is a volume mount, and anything the image wrote there would disappear behind it the moment one is attached. `client/.keep` is tracked so `COPY client/` cannot fail on a tree where the CI step never ran; the artifacts themselves are gitignored, since a 55 MiB binary does not belong in git history and is re-fetched on every build anyway.	2026-08-20 21:19:47 -04:00
bvandeusenandClaude Opus 4.8	05d5249e20	M0 CI: derive registry username from github.repository_owner ... The registry login needed a REGISTRY_USER secret that couldn't be set/read on this fresh repo (harness blocks secret writes; the value never surfaced via the API). The username is the repo owner and is public (it's in the image path), so derive it from github.repository_owner instead of a secret. REGISTRY_TOKEN remains the actual credential. Removes an entire class of setup friction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRgehjoz7Yv8LkUfADxACm	2026-07-19 13:32:37 -04:00
bvandeusenandClaude Opus 4.8	af8aa25464	M0: Docker + Fabled-Git CI (typecheck + lint + test + build) ... - 2-stage Dockerfile (node:22 build Vue -> python:3.12-slim runtime); CMD runs `alembic upgrade head` then hypercorn (rule 82). - .forgejo/workflows/ci.yml on ci-python:3.14: typecheck (vue-tsc) + lint (ruff check src/) + test (uv venv + pytest, DB-free) + gated buildx push with the rule-46 tag scheme (:dev/:latest/:<sha>). No local integration lane yet. - ci-requirements.md (rule 39); docker-compose for a local app+Postgres stack. - Real README with layout, dev instructions, and the milestone arc. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRgehjoz7Yv8LkUfADxACm	2026-07-19 13:17:38 -04:00