100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusen	63e0a423d7	ci: a weekly base-image refresh on the channel tags (milestone 326 step 4) ... Skip-if-exists is keyed on our own source, so an artifact whose source stops moving stops picking up base-image updates. `agent/` last changed 2026-07-17; every push since has correctly declined to rebuild it, which also means it will serve that day's nvidia/cuda layers indefinitely. A `schedule:` trigger, Sunday 06:00 UTC, away from CI-runner's Monday security sweep so the two are never diagnosing each other. #3154's blocking open question is dissolved rather than answered. It was written when the identity was a `r-<revision>` TAG, and asked how the next ordinary push could avoid repointing :latest back off the refresh. Milestone 318 replaced that tag with a LABEL, and #3183 made the repoint step exclude its source tag so the label stays readable. Excluding the source is what also keeps a refresh from being undone: on the next main push the reuse check hits, :latest is not rewritten, and the new :c-<sha> is written FROM the refreshed :latest. To be verified by digest, not by this argument. Four decisions, each commented where it lives: * It builds `main`, not the branch that triggered it. Forgejo registers a cron from the default branch — `dev` here — so a scheduled run arrives with github.ref on dev, and a refresh of :dev would be refreshing the one channel that is rebuilt constantly anyway. The ref is decided once in a top-level `env: BUILD_REF` that all four checkouts take. Deriving it per job would let the halves disagree: sign-extension would derive dev's extension version while build-web bundled main's, and the release download would 404 on a version that exists perfectly well. * It publishes only the channel tag. :c-<sha> for main's HEAD already names the bytes that commit built; re-pushing it over refreshed layers would break the one tag rule 145 makes immutable, and it is the rollback unit — so the breakage would surface on the day somebody needed it. The repoint step needs no schedule case: the tag list is the channel tag alone, SOURCE is the only entry, it is excluded as always, and the step correctly does nothing. * It bypasses reuse by construction, since it rebuilds the same source and fc.revision always matches. Checked in the reuse step beside force_build, so one decision still drives both the build and the repoint. * `pull: true`, on the scheduled path only, is the actual mechanism. A moved base tag changes the FROM layer's cache key and everything above it rebuilds; an unmoved one is satisfied by the registry cache and the refresh is a ~13s no-op that republishes nothing. That no-op is the point — :latest should change when there is something new in it, not every Sunday. The known lag, left deliberately: an apt package update while the base tag stands still is not caught, and closing it needs no-cache: true, which buys weekly churn for it.	2026-08-29 22:53:48 -04:00
bvandeusenandClaude Opus 5	5e72076298	ci: registry-backed layer cache for all three images (milestone 326 step 2) ... `cache-from`/`cache-to` on `<image>:buildcache`, `mode=max`, on all three build steps. Closes the half of the driver change that step 1 left open. Step 1 measured worse, not better, and that was expected but is worth stating with numbers. Run 4896, first builds after moving to `docker-container`: build-web 3m44s (cold baseline 2m23s) build-ml 3m49s (cold baseline 3m20s) build-agent 11m12s (cold baseline 9m26s) The container driver gets a FRESH buildkit instance per job, so it has no local layer store to fall back on — where the old docker driver at least reused whatever the runner's dockerd happened to hold. That is why a registry cache is the only cache this driver can have, and why step 1 on its own is a regression rather than a win. `mode=max` so intermediate stages cache too. The agent's two ~150s pip layers and web's frontend-builder stage are the entire cost, and a min-mode cache would drop exactly those. A `:buildcache` tag is not the withdrawn tag scheme returning. Rule 145 narrowed against names NOTHING reads; this one is read by every build that runs, is one moving ref per image rather than one per build, holds cache blobs rather than a shippable artifact, and is overwritten in place rather than accumulating. Closer to `:dev` than to the `:2026.8.28` tags 318 deleted — and said so in the workflow, so it is not "cleaned up" by a later reader. Expect the next build to be slower again, once: it is still cold AND now pays the cache export. The measurement that matters is the one after that. Scribe #3114. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 18:56:13 -04:00
bvandeusenandClaude Opus 5	e21c9fdd34	ci: a force_build escape hatch for the path skip-if-exists hides (326 step 3) ... `workflow_dispatch` with a `force_build` boolean, honoured inside each of the three reuse steps. It exists because skip-if-exists made its own build path untestable. `agent/` has not changed since 2026-07-17, so the agent build has correctly declined to run on every push since — which also means #3190, whose whole symptom lives on that path, cannot be reproduced on demand. Editing build.yml does not force a build either, and that is deliberate: the workflow is not shipped bytes, so it is in no artifact's path set, and putting it in one would re-version every artifact for a comment change. That is also why this lands before step 2 rather than after. Step 1 moved the builds onto a container driver and turned attestations off; the claim that `fc.revision` still reads back cannot be checked until something actually builds under that driver. Run 4887 confirmed only the cheaper half — `Set up buildx` succeeded on all three jobs, so the buildkit sibling container does start against the mounted socket. Details worth keeping: * FORCE is checked in the reuse step, not in the build step's `if:`. The repoint step keys off `hit` too, and a force that bypassed only the build would leave the two disagreeing about what had happened. * `github.event.inputs`, not the `inputs` context — release.yml already uses that form and it is the one this runner is known to evaluate. Read through env rather than interpolated into the run block, same as release.yml's TAG. * One input, not one per artifact. Three booleans is an interface nobody remembers. Scribe #3252, #3249. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 18:38:29 -04:00
bvandeusenandClaude Opus 5	6b3ec98fa8	ci: build on a real buildx driver, attestations off (milestone 326 step 1) ... Adds `docker/setup-buildx-action@v3` to build-web, build-ml and build-agent, and sets `provenance: false` / `sbom: false` on all three build-push steps. Two open issues share one root, which is why this is one change: * #3114 — the agent rebuilds a ~6.3 GB CUDA + torch image whenever the runner's local cache is cold, 9m26s against 7s warm. The default `docker` driver cannot export a registry cache at all, so the fix is unavailable until the driver moves. The cache itself is step 2, deliberately not here. * #3190 — build-agent goes red AFTER a successful push, `No such image` from the local daemon. The leading candidate is the docker driver resolving image metadata against a local store a registry-direct push never filled. The attestation flags are the load-bearing part. On the default driver they were no-ops; on the container driver, build-push-action@v5 defaults provenance to TRUE when pushing. Provenance attaches an attestation manifest, that makes the pushed tag a manifest INDEX, and `.Image.Config.Labels` does not resolve through an index — so the `fc.revision` label the reuse check reads off the channel tag would come back `<none>` on every push. Every image would rebuild forever, every lane would stay green, and the only symptom would be the bill. Same failure as #3183, through a different door; note #3127 §4 records the same shape for `platforms:`. Unverified until CI says otherwise: these jobs run INSIDE a container against a mounted docker socket, so the buildkit container is a sibling of the job container rather than a child. That works over a socket mount and has never been tried on this runner. The gate is the SECOND dev push, not this one. The images currently published were built by the old driver, so one `reuse: NOTE ... no readable fc.revision` is expected now; what must not appear is a second one. Scribe #3249. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 18:36:35 -04:00
bvandeusenandClaude Opus 5	1a941e900b	test: encode the extension's AMO rendering exception (milestone 318 step 8) ... Missed in 2e01242. This module pinned zero-padded `YYYY.MM.DD.HHMM` for all four artifacts, which is the family shape and was right until the extension acquired a documented reason not to use it. Both assertions failed exactly as written, on the value they were written to catch. Rather than exempt the extension, the exception is pinned to the constraint that justifies it: * `test_version_is_zero_padded_calver` now covers the three padded artifacts. * A new sibling covers the unpadded one against **AMO's own grammar** — `2026.08.29.0201` fails it, so a regression to padding fires immediately. Matching only `YYYY.M.D.HHMM` would not: on a date with no leading zeros the two renderings are the same string, so a padding regression would sit unseen until the first single-digit month, and surface as a burned AMO version rather than a red lane. * `test_version_and_revision_describe_the_same_commit` compares NUMBERS, per rule 148's own definition of comparison — so one assertion covers both renderings and says the real thing: whatever the padding, the extension must denote exactly the value its own commit stamps. Exact-string equality is still asserted for everything not in AMO_UNPADDED, so the exception cannot quietly spread. Scribe #3138. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 13:46:18 -04:00
bvandeusenandClaude Opus 5	2e01242381	feat(extension): derive the version as unpadded CalVer (milestone 318 step 8) ... `1.0.<minutes since 2020>` -> `YYYY.M.D.HHMM` UTC, from the commit time of the newest change to a packaged extension file. Same clock and same commit as before; readable instead of opaque, and the same value the rest of the family derives. The hold on this step was two questions about AMO, and Mozilla's own docs answer both: ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$ 1. four all-numeric segments -> ACCEPTED ({0,3} more after the first). 2. leading zeros -> REJECTED. A segment is the single digit `0` or starts 1-9, so `08` and `0201` are refused. MDN says it in prose too: "Non-zero numbers must not include a leading zero." So the documented fallback applies, extension only: the same numbers rendered without the family's zero-padding. `2026.08.29.0201` and `2026.8.29.201` are one value in two renderings — rule 148 defines comparison as numeric per segment, under which they are equal — so nothing already published is reordered, and left-padding each segment recovers the family string exactly. HHMM stays one segment because AMO allows at most four. The transition is safe in the other direction too: 2026 > 1, so every CalVer outranks every published 1.0.x. build.yml's downgrade guard confirms it. Also in scope: * MAJOR.MINOR is gone. `cmd_major_minor`, `cmd_patch` and VERSION_EPOCH go with it, the committed version in manifest.json / package.json is now wholly inert, and ci.yml's MAJOR.MINOR-agreement check is retired rather than left running beside a fact that stopped existing (rule 22). * ci.yml's `extension-version` lane now asserts Mozilla's regex verbatim instead of a loose `^[0-9]+(\.[0-9]+)*$` — which would have passed the padded shape. It also asserts YYYY.M.D.HHMM, because AMO would accept a regression to `1.0.<minutes>` while that orders below everything signed since. Checking here is the point: AMO 409s on re-signing, so a version it rejects is burned and cannot be reused. * `artifacts.sh version extension` delegates to packaging.sh, so the two cannot answer differently. The direction matches the existing one — artifacts.sh already asks packaging.sh for the extension's path set. #3156 is what makes this commit safe to make: packaging.sh is in web's path set, so the web revision moves with the extension version and build-web rebuilds instead of republishing an image bundling the previous XPI. Scribe #3138. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 13:43:30 -04:00
bvandeusen	41f2bec3af	fix(ci): the build pushes one tag; the rest are written registry-side (#3190) ... buildx on this runner pushes the first tag to the registry and then re-pushes the remaining ones through the DOCKER driver, reading them out of a local image store that a registry-direct build never populated: #27 pushing …/fabledcurator:latest DONE 15.8s #28 pushing …/fabledcurator:c-0e15c44 with docker #28 ERROR: tag does not exist: …:c-0e15c44 It is intermittent — build-ml made the identical two-tag push seconds later in the same run and succeeded — and the consequence is worse than the red job suggests. `:latest` had already published, so production was correct while the immutable rollback tag rule 145 requires of every main push simply did not exist. Nothing else would ever have noticed: a missing :c-<sha> has no consumer that fails, so it surfaces at the moment somebody needs to roll back, which is the worst time to learn a rollback target was never written. So the build now pushes exactly one ref — the channel's — and the existing repoint step, which already excluded the source tag and already ran on every reuse, now runs on the build path too and owns every other tag. `imagetools create` is a registry-side manifest copy: no local daemon, nothing that can be absent. This adds no new code path; it puts the build case onto the one that was already proven. Chosen over the alternative of asserting each tag resolves after the build, which would have made the failure loud without making it rarer. The cost, accepted: `imagetools create` wraps its source in an index, so :c-<sha> is an index rather than a plain image and fc.revision does not resolve through it. Nothing reads that label off :c-<sha> — the reuse check only ever inspects the CHANNEL tag — and the index names the same manifest, so a pull is byte-identical. The reuse path already produced :c-<sha> this way; this only makes it uniform. `build_tags` goes with it — the tag list now has exactly one consumer.	2026-08-29 00:38:26 -04:00
bvandeusen	d38585ed94	docs: true up the tag scheme against what the pipeline publishes (318 step 9) ... A doc describing a tag scheme the pipeline stopped using is worse than no doc — it is a confident wrong answer, and #3159 named it as the drift that had just bitten rule 148. build.yml (each edit ×3, once per build job): * The derived-values block claimed three values including a `tag` subcommand that no longer exists, and said "on dev the date tag is computed nowhere else". There are two values and no date tag. * The BASE-IMAGE FRESHNESS paragraph argued from a pin that no longer exists — "a date tag has to keep serving the bytes it served (fabledcurator:2026.7.17 still resolves to July's image)". Milestone 318 removed the argument's premise rather than answering it: with no version tags there is no immutable name a refresh could contradict, and rule 145 already allows a moving tag to be republished. Deleted the argument, kept a line saying why there is nothing to argue about (#3159's phrasing: the workflow becomes obviously correct instead of carefully correct). * The repoint comment said ":c-<sha> and the date pin". There is no date pin. * "its set is too narrow — the direction that serves stale bytes on a pin" now names the failure that actually exists: the reuse check hits and the channel serves a web image bundling the previous XPI (#3156). README gains a "Versions and tags" section — the three tags, why there is no fourth, and the fact that with no registry name carrying it, the instance's own report is the only answer to "which build is this?". Also fixes two stale claims: three workflows (four since release.yml) and "AMO signing runs on main only", which has been false since milestone 271 step 6 signed on both channels. ci-requirements.md was missing milestones 313 and 318 entirely: no artifacts.sh, no label-keyed reuse, no FC_VERSION. Added those, plus the #3156/#3202 membership test (can changing this file change the published bytes — not is it copied in), and corrected packaging.sh's "two consumers" to three now that artifacts.sh reads it. docker-compose.yml needed nothing — it names `:dev` directly and never offered a pin target. Note #3127 and rule 145's relation notes are updated in Scribe.	2026-08-29 00:09:52 -04:00
bvandeusen	a3071a7549	fix(ci): the file that decides FC_VERSION now moves it (#3202) ... `scripts/artifacts.sh` decides both values the web image carries — the `fc.revision` label the reuse check compares and the `FC_VERSION` baked into the image — and was in no artifact's path set. So a change to `cmd_version` alone left every revision untouched, the reuse check hit, the build was skipped, and the published image went on reporting the OLD version format, indefinitely, until some unrelated commit forced a rebuild. Nothing goes red; the footer just shows a well-formed string of the wrong shape. Milestone 318 step 5 is the worked instance: b3989d0 -> rev=fb2c4d5b80be ver=2026.8.28.1249 5771fd5 -> rev=fb2c4d5b80be ver=2026.08.28.1249 bce894b -> rev=bce894ba2499 ver=2026.08.28.2208 Same revision across the zero-pad commit, so web's build was skipped. It cost nothing only by timing: FC_VERSION did not exist until step 6 landed one commit later. Web only, and that is the interesting part. Every artifact stamps a revision, but only web also stamps a version. A revision-only artifact needs no entry here, because changing how a revision is COMPUTED changes the derived value, which then disagrees with the label on the published image and forces a rebuild — the mechanism self-corrects, since it compares against a string stamped into a real artifact. Nothing compares a version to anything. That asymmetry is why this was invisible and is now written down in both files. Named as a file rather than `scripts`: release_notes.py sits beside it and only reads derived values, so it decides nothing and must not re-version web. This is #3156 one level up — packaging.sh excluded from the version it derives — so the guard is generalised rather than duplicated: one DERIVERS table naming each deriver and the artifacts whose identity it decides. The too-wide test gains a note saying where the line is, since "copied into no image" no longer settles it on its own.	2026-08-28 22:01:58 -04:00
bvandeusen	b6b9fd8287	ci: a release publishes a changelog, not an image (318 step 7) ... Step 2 took the build consequence away from a `v*` tag — `main` has already built and published the commit by the time anyone tags it, and rebuilding would re-push `:c-<sha>`, which rule 145 forbids even when the source matches. That left the tag with nothing to do at all. This is the job it has instead. Step 6 put the derived version in the Settings footer, so an operator can say WHICH build they are running; this says what is in it that was not in the one they ran last month. Both halves of one question (note #3127 §5). The previous release is found by walking ANCESTRY, not by sorting a list. That is load-bearing here specifically: rule 148 moved the tag shape from `v26.05.22.0` to `v2026.08.28.2208`, and lexicographically `v2026...` sorts BEFORE `v26...` — the third character is `0` against `6`. A sorted implementation would reach back past every new-shape tag to the newest old-shape one and publish months of commits as "changes since", looking entirely correct while doing it. `git describe --exclude` is immune to the shape change, and reachability is the more honest question anyway. The publisher GETs and PATCHes rather than POSTing and recovering the id from a 409 — note #3127 §6.7, which is ThoughtSync #2182's bug. A `v*` tag is created once so the conflict path is rare, but "rare" is how that one survived to be found somewhere else. Cross-checks are reported on the release, not enforced. The tag is already pushed by the time this runs, so failing would leave the operator with a tag, no release, and a red lane to explain it — while the release is still the useful object. It says so at the top when the tag names a version the web image does not report, or when the commit is not on `main` and the `:c-` rollback refs it lists were never published. Nothing runs on a schedule and nothing auto-tags on merge. Release tags are bookmarks (note #3127 §0); FC went twelve weeks without one and nothing was wrong. Also here: - `scripts/` joins the ruff lane. release_notes.py runs only on a tag push, so a syntax error there would otherwise surface at the one moment nobody wants to be debugging a workflow. - version.spec.js reads the workflow directory instead of listing three files by hand. Its own comment says the assertion should survive consumers coming and going; the hardcoded list was the part that could not, and release.yml would have joined the directory without joining the check. Tests build a synthetic history spanning the tag-shape change rather than leaning on this repo's tags, so the span assertion holds whether or not a checkout brought the tags along — a span test that quietly skips is worse than one that fails.	2026-08-28 20:57:25 -04:00
bvandeusen	bce894ba24	feat(settings): the instance reports which build it is (318 step 6) ... A dim line at the foot of Settings: `FabledCurator 2026.08.28.1249 · dev`. This is no longer a convenience. Milestone 318 stopped publishing version image tags, so an instance's own report is the ONLY answer to "which build is this?" — there is no registry name left to check it against. Note #3127 §5 says it directly: a wrong answer here has no second source to contradict it. Three states, kept distinct because collapsing any two of them lies: not asked yet render nothing asked, no version render "unknown" asked, has a version render it A blank footer reads as "no version", which is a different claim from "I cannot say". And a failed health call deliberately does NOT mark the build loaded — a network blip says nothing about the image, and presenting it as "unknown" would look like a defective build. Carried on /api/health rather than a new route: it answers at the same cost (two module constants, no I/O) and TopNav already fetches it app-wide, so a separate endpoint would mean a second request for two strings. Both fields are OMITTED when unset rather than sent empty. Absence already means "cannot say" — an image predating the field says exactly that by not having the key — so a second spelling would make every reader special-case it. The pre-existing test asserting the body is EXACTLY {"status": "ok"} is what keeps a well-meaning `or ""` default from creeping in. FC_CHANNEL now has one definition. It was read from the environment in extension.py and would have been read again here; the new build_info module holds both, and extension.py binds it as a module-level name so existing tests monkeypatch it exactly as before. Separate from config.py on purpose: those are operator settings meant to be changed, these describe the artifact. Channel sits beside the version, never inside it (rule 149), asserted from both ends. A `-dev` suffix would read as a 0 segment to the extension's parseInt comparator and make every dev build compare equal — #2993 exactly. Not hidden, per the operator and §7: the JS bundle and asset hashes fingerprint the build anyway, and "I'm on 2026.08.28.1249" is the single most useful line in a bug report.	2026-08-28 18:08:31 -04:00
bvandeusen	5771fd5770	build: zero-pad the derived version to YYYY.MM.DD.HHMM (318 step 5) ... `2026.8.28.1249` becomes `2026.08.28.1249`. Note #3127 §1 and rule 148 both specify the padded form. The old reasoning was that each segment should read as a plain integer, and it never held — comparison strips leading zeros on parse anyway, which the same paragraph said. What stripping actually bought was this project emitting `2026.8.28.1432` while a sibling emitted `2026.08.28.1432`: two shapes one character apart, which is the hard kind of difference to notice. Two obviously different formats would be safer than two nearly identical ones, and identical is safer still. Nothing already published is reordered: comparison is numeric per dot-segment, so `08` and `8` are equal. strip0 goes, and with it three of the four git calls per version — git's format-local takes the whole format string, and splitting it into pieces only ever existed to strip the padding between them. It also fixes a real edge the old helper mangled. A commit at 03:22 UTC derived `322` for its HHMM field, silently turning a four-digit field into three; it now derives `0322`. Verified against a real commit rather than reasoned about. Checked before relying on it, since step 8 feeds this to Firefox: the extension's comparator is `parseInt(n, 10)` with an explicit radix, so `08` reads as 8 and there is no octal hazard (rule 150). Two tests added. One pins the padded shape — the only thing keeping the family's projects emitting one string is an assertion that they do. The other asserts version and revision describe the same commit: they are derived independently, and a divergence would mean an instance naming one commit while carrying another's bytes, which is unfalsifiable from outside because both values still look well-formed.	2026-08-28 16:35:26 -04:00
bvandeusen	b3989d0224	fix(ci): the repoint was destroying the label it depends on ... Reuse worked exactly once per artifact, then every later push rebuilt at full price. Nothing failed and nothing went red — the savings simply evaporated. `imagetools create` wraps its source manifest in an INDEX. The repoint step passed the channel tag as both source and target, so after any reuse :dev stopped being a plain image, `.Image.Config.Labels` no longer resolved through it, and fc.revision read as absent on the next push. Observed across three runs rather than reasoned about: run 4749 read fc.revision=a7e626a67a79 off fabledcurator-ml:dev and skipped the build; run 4751 read <none> off the same tag and rebuilt. The only thing to touch it in between was 4749's own repoint. The agent hit in 4751 precisely because its :dev had last been written by a real build, not by a repoint — which is the control case. Milestone 313's r-<rev> design was immune without anyone noticing why: the source (the identity tag) was never one of the targets. Step 3 made the channel tag both, and inherited a bug the earlier shape had avoided by accident. Fix: exclude the source from the target list, so the channel tag is only ever written by a real build and stays a plain readable image. On dev that leaves nothing to do, which is correct — the hit already established that :dev points at the right content. On main it leaves :c-<sha>, which rule 145 requires of every main push whether or not a build ran. Also added a note the reuse step prints when a channel tag exists but carries no readable label. That is expected exactly once per artifact during the migration; if it appears on every push, the tag is being index-wrapped again and reuse is dead. This class of failure — correct behaviour, quietly worth less than it reads — is the third one this milestone has turned up, and it is the one that does not announce itself.	2026-08-28 14:55:49 -04:00
bvandeusen	cd0b0ff04a	ci: publish :latest + :c-<sha> on main, :dev on dev — nothing else (318 step 4) ... The narrowing itself. Rule 145, narrowed 2026-08-28 once it was verified that nothing pins: "a third name for the same thing is upkeep for a model we do not run." Gone: :2026.8.28 the per-artifact date tag from milestone 313 step 3. It shipped, it was verified on both branches, and its premise is gone. This is a withdrawal, not a correction — nothing about it was wrong. :main a second moving name for whatever :latest already pointed at, justified by nothing but symmetry with :dev. Kept, and note the inversion: :c-<sha> on main. Milestone 313 step 6 planned to retire it on "haven't used, not important" — amended rule 145 makes it the rollback unit and, with version tags gone, the only immutable identifier left. #3139 is cancelled as superseded rather than quietly dropped. No retention job is built. One already runs (#3157): a nightly Gitea package_cleanup_rule with keep_count=10. Two consequences worth writing down. Its keep_pattern protects any dotted-numeric tag, so the date tags this commit stops minting would have been exempt from retention and accumulated without bound — silently, in the direction of unbounded growth. That trap is now dissolved rather than needing the pattern change #3139 planned. And :c-<sha> IS prunable under that pattern, so the honest statement is that it is the rollback unit for roughly the last 10 builds, not for every main commit ever. cmd_tag goes with its last consumer rather than being left as a dead subcommand for a later step to notice. The report step logs version and revision, which is the same information — the date is the first three fields of the version. Docs needed no change: every consumer already tracks a moving tag (docker-compose.yml on :dev, agent/README.md on :latest), which is the evidence the whole stance rests on.	2026-08-28 14:45:11 -04:00
bvandeusen	454eb3f973	fix(tests): artifact identity tests talk to artifacts.sh, not to a sibling ... Run 4746 failed lint and pytest on the same cause: `from test_artifact_paths import ROOT, declared_paths`. No other test module in this repo imports another, so that was a new convention introduced for no gain — and the wrong one, since `tests/` is a package and the bare name does not resolve. Everything now goes through `artifacts.sh`, which is the interface build.yml actually calls. The tests exercise the contract rather than a Python re-implementation of it, and the duplicate `declared_paths` helper is gone rather than copied. Two real defects found while fixing it: The newest commit is now computed by committer TIME, matching what artifacts.sh means. It was `git log -1`, whose default order is reverse-chronological only within topological constraints — so on a merged history it can name a different commit than the newest timestamp does. Both agree on this repo today (verified across all four artifacts), which is exactly what makes it a flake waiting for the branch shape that separates them. The third test asserted the same invariant as the first in different words. Removed rather than left as apparent coverage.	2026-08-28 14:42:42 -04:00
bvandeusen	7e065fed70	ci: key the reuse check on an image label, not a tag (318 step 3) ... The shadow (dee93fa, run 4732) answered the gate: `imagetools inspect --format` reads `.Image.Config.Labels` against this registry on buildx v0.36.1. So the reuse check now asks the moving channel tag whether the image it already points at carries this commit's `fc.revision`, and the r-<rev> identity tags stop being published. Three things this removes rather than manages: A name minted per build that one thing read. Rule 145's narrowing is aimed exactly there — "a third name for the same thing is upkeep for a model we do not run." The -main/-dev qualifier, and the CHANNELLED list behind it. Which tag you inspect IS the channel, so the distinction has nowhere to live. cmd_identity goes with it. A silent expiry nobody wrote down. r-<rev> matches no branch of the registry's keep_pattern (#3157), so identity tags were prunable past the newest 10 — a pruned one costs a rebuild, in the safe direction and entirely invisibly. A label rides inside a tag that has to exist anyway. It also dissolves #3154 instead of deferring it: a scheduled base refresh rebuilds :latest with the same revision label, the next unrelated push sees a match and skips, and the refreshed base survives. Under the tag scheme that push repointed :latest back to the older base. The measured detail that shapes the code: a missing label returns an EMPTY STRING and exits 0. Branching on the exit code would read "no label yet" as success and skip a build that was needed. So it compares values, and every uncertain case — absent label, unreachable tag, older image — lands as empty, never equals a 12-char revision, and falls through to a build. Reading the specific key matters too. The map carries the base image's labels, and org.opencontainers.image.version sits right beside ours reading 24.04 on the agent — a plausible-looking wrong answer. Expect every artifact to rebuild once on this push: nothing carries a label yet and it cannot be backfilled, since the reuse path copies a manifest and config labels are not manifest annotations. One rebuild per artifact, ever, self-healing after. test_artifact_identity.py is rewritten around what is now load-bearing. The CHANNELLED drift test had nothing left to guard; in its place the revision is asked of git directly, so the file fails if the derivation ever stops being "the commit this artifact's own shipped files last changed in".	2026-08-28 14:37:36 -04:00
bvandeusen	dee93faa37	ci: shadow the label-based reuse key before trusting it (318 step 3) ... Additive and inert. Every build now stamps `fc.revision` as an image label, and a diagnostic step tries to read it back off the channel tag. Nothing downstream reads either yet — the reuse check still keys on the r-<rev> tag. The question step 3 has to answer before committing: can the reuse check be keyed on a LABEL read off the channel tag instead of on a separate tag? If it can, nothing is minted per build, the -main/-dev identity split disappears with CHANNELLED, and #3154's base-refresh clobber stops existing rather than needing a workaround. It also closes a defect found in what already shipped: r-<rev> tags do not match the registry's keep_pattern (#3157), so they are prunable, and a pruned identity tag silently costs a rebuild. A label cannot be pruned — it rides inside a tag that must exist. Three probes rather than one, because `--format` templating over .Image varies by buildx version and one round trip per guess is a poor trade. Two questions, answered at different times, and conflating them would abandon a working design: 1. Do the template mechanics work here? Answered on the first run — if the probes print a labels map, even an empty one, `--format` resolves on this buildx and this registry. That is the real gate. 2. Does a value round-trip? Only once an artifact genuinely rebuilds. This push touches build.yml, which is in no path set, so all three artifacts reuse and stamp nothing. Expect no values this run. That second point is the migration path rather than a defect: under the real mechanism "no label" reads as a MISS, so each artifact rebuilds once and is stamped from then on — one extra build per artifact, ever, in the safe direction. `imagetools create` cannot backfill it, since it copies a manifest and config labels are not manifest annotations.	2026-08-28 14:00:09 -04:00
bvandeusen	d9aa5aa832	ci: a release names a build, it does not make one (milestone 318 step 2) ... Removes the `v*` image-build trigger. A release tag names a commit `main` already built and published; rebuilding it produced the same source under the same names and RE-PUSHED `:c-<sha>` — which rule 145 forbids even when the bytes match, because image configs carry timestamps and "same source" does not mean "same manifest". The tag build was publishing nothing new and violating an immutability rule to do it. Three constructs go with it, all of which existed only to manage that build: The 10-minute XPI polling loop. A release cut fired the tag build and the main build together; the tag build skipped sign-extension and raced straight to the download, losing every time (operator-flagged 2026-05-27 after v26.05.27.0). Polling was the fix for a build that should not have run. It is now a single fetch, and a 404 means the two jobs derived different versions for one commit — which should fail loudly rather than be slept through. The IS_TAG_PUSH carve-out from milestone 313 step 4, which stopped a tag build from claiming an identity tag it would have re-pointed at fresh bytes. build-web's `always() && (success || skipped)` gate, which existed to let a SKIPPED sign-extension through on a tag push while still blocking a failed one. sign-extension now always runs, so a plain `needs` gives exactly the wanted behaviour. Its own branch condition goes too: main and dev are the only triggers, so a gate naming both matched everything, and a condition that is always true reads as though some path avoids it. Releases still happen — rule 148 is untouched and tags are still cut on explicit request per rule 2. They stop building images and start carrying a changelog (step 7). Net 115 lines deleted, 73 added, most of that comments explaining races that can no longer occur. Nothing in the repo referenced the tag build, so no doc changes were needed; the broader doc pass is step 9.	2026-08-28 13:49:45 -04:00
bvandeusen	fb2c4d5b80	fix(extension): packaging.sh decides the version, so it must move the version ... `packaging.sh pathspec` excluded `extension/scripts/**` — the same list web-ext ignores. But the two lists answer different questions, and this is the one place they disagree. packaging.sh is not packaged into the XPI. It does decide the version string, and build.yml stamps that string into the manifest.json that IS packaged. Changing how the version is computed therefore changes the shipped bytes, and the derivation has to see it. Harmless while every push rebuilt the web image. Step 4 made the rebuild conditional on the derived revision moving, which turns it into a silent failure: a packaging.sh change yields a NEW version, so sign-extension misses its ext-<version> cache and signs — while build-web sees an unmoved revision, reuses the published image and ships the OLD XPI. One orphaned AMO signature, and an instance serving code the registry calls current. Found while checking the ground under step 5, which changes the version format and is exactly the commit that would have hit it. Split the list rather than widening the shared one: NOT_VERSION_RELEVANT drives the pathspec, NOT_PACKAGED_TRACKED still drives web-ext's ignore list, and scripts/ stays out of the XPI. The two directions are not symmetric, which is why the version list is the narrower one — too wide costs a re-sign and a rebuild for a change that ships nothing new, too narrow serves stale bytes and says nothing. No version churn: the last packaging.sh commit predates the current extension revision, so the derived version is unchanged at 1.0.3500147 and web's revision stays a7e626a67a. Both suites now pin the disagreement from their own side, because the tempting fix for either half is to make the lists one again — version.spec.js asserts the pathspec does NOT exclude scripts while the ignore list still does, and test_artifact_paths.py asserts packaging.sh is inside the extension and web path sets.	2026-08-28 08:49:06 -04:00
bvandeusen	609bc82acc	ci: reuse the published image instead of rebuilding it (milestone 313 step 4) ... Before building, each job asks the registry whether this artifact's content is already published. On a hit it skips the build entirely and repoints the channel and date tags at the existing manifest with `imagetools create` — registry-side, no layer transfer, seconds. This is the step that stops a push touching only `agent/` from rebuilding web and ml, and stops a merge to main rebuilding what dev already built. The question is asked with a new `artifacts.sh identity`, not with the date tag: the date tag is day-precise and last-one-wins, so two different builds share it and it cannot answer "is this content published?". The commit sha would move on every push and never hit, which is the redundant rebuild being removed. The revision does both jobs — content-unique, and stable across pushes that did not touch the artifact. Identity is channel-qualified for web and only for web, because web is the only image that takes a build-arg: FC_CHANNEL is baked in and reported by /api/extension/manifest, so its dev and main builds of one revision are genuinely different images. ml and agent take none, which is what lets a merge reuse dev's build rather than rebuilding the agent's CUDA image to produce bytes that already exist. tests/test_artifact_identity.py reads the Dockerfiles and fails if that list drifts from the ARG declarations, in either direction — collapsing the channels ships an instance that reports the wrong one, and splitting them needlessly rebuilds every merge. Failure direction is deliberate: an inspect that errors for any reason reads as a miss and the build runs. Only a real 200 skips one. A tag-push never claims the identity. It rebuilds a revision main already published, and image configs are not bit-reproducible, so re-pushing r-<rev> would point an immutable tag at fresh bytes — rule 145's exact prohibition. It publishes only its own :v... label and otherwise reuses. Base-image freshness, decided rather than left implicit: an artifact whose source stops moving stops picking up base updates under its pinned tag. That is what a pin means, and rule 145 already says the refresh belongs on the moving tag instead. Filed as #3154 rather than folded in here, because the naive version regresses :latest on the next unrelated push. ci.yml's backend lane gains fetch-depth: 0 — the new tests derive real revisions, and on a depth-1 clone that derivation returns the tip sha or fails, so the lane would go green while asserting nothing. The three build jobs' shadow steps are renamed and re-commented: those values stopped being informational at step 3, and a step captioned "nothing reads this" beside steps that do is worse than no caption.	2026-08-28 08:24:58 -04:00
bvandeusen	7a20c55441	ci: publish a per-artifact date tag on main builds (milestone 313 step 3) ... Each main build now also publishes <image>:YYYY.M.D — the date of the commit that artifact's shipped files last changed in. Purely additive; :main, :latest, :c-<sha> and the dev path are untouched, so reverting this commit reverts the behaviour. main push -> :main, :latest, :c-<sha>, :2026.8.27 dev push -> :dev Per artifact, so an image whose files did not change keeps the tag it already had. On this commit the agent reads 2026.7.17 while web and ml read 2026.8.27 — six weeks apart, from one push. Step 4 turns that into not rebuilding it. Day precision, and a second main build the same day replaces the first. Operator's call, and the reasoning is theirs: same-day work is not something worth pinning. A rollback goes to a day, not to the fourth merge of a Tuesday afternoon. It also makes retention mean "the last N days" rather than "the last N pushes". CALVER is computed inside the main branch rather than at the top of the step, and hard-fails when empty. There is no `set -e` here, so an unconditional assignment that failed would have left it empty and published the tag `fabledcurator:` — an invalid name, from a step that still reported success. It is also simply unused on the dev and tag paths. Fixed a stale comment while in this block rather than leaving it for step 7: it claimed release tags carry "no `.N` per family release-posture rule". Rule 148 was amended on 2026-08-24 to REQUIRE the suffix, after the ban caused a same-day tag to be retargeted and a release deleted to make room (note 2813). FC's own tags already carry suffixes; only the comment was asserting the superseded rule. Verified before pushing: the derivation holds across 200 commits of real history — a derived revision always touches its own path set, the version never decreases along any parent->child edge, and web tracks all three extension-only commits in the log. That last one is the direction that would serve stale bytes on a pin.	2026-08-27 21:56:52 -04:00
bvandeusen	0c43fa3eb2	ci: shadow the per-artifact derived versions (milestone 313 step 2) ... Every build job now logs the tag, version and revision its artifact would get. Nothing reads them; no `set -e`, and each derivation falls back to UNAVAILABLE, so a broken script cannot fail a build. Same discipline as milestone 271 step 2, which is what made that cutover safe to do in one commit. Also fixes a landmine the plan named but had not checked: build-ml and build-agent were checking out at depth 1. Both now use fetch-depth: 0. That mattered more than it looks. A depth-1 clone sees one commit, so `git log HEAD -- <shipped paths>` either returns that commit's timestamp — plausible, and wrong — or returns nothing. For build-ml on this push it would have returned today's date, because HEAD touches backend/, and nothing downstream would have questioned it. For build-agent it would have returned nothing at all, since no single commit here touches agent/, and artifacts.sh exits non-zero rather than guessing. One direction is silent and one is loud; only the loud one was ever going to get noticed. What to read from the shadow lines over the next few pushes, in order of how badly each would bite: * a push touching the extension must move BOTH the extension and web, because build-web bakes the XPI in. If web does not move, its path set is too narrow and a pinned web image will serve an extension it does not name. * a push touching only agent/ must leave web and ml still. If they move, their sets are too wide and they will rebuild for changes they do not ship. * a docs-only push must move nothing. * dev and main must derive the same values for the same source. Step 3 only lets these values name a tag once those hold.	2026-08-27 21:41:14 -04:00
bvandeusen	cf06c81db9	build: one definition per artifact of what it ships (milestone 313 step 1) ... scripts/artifacts.sh generalises what packaging.sh established for the extension: four published artifacts, four path sets, four independent versions derived from the newest commit touching each set. Measured on this commit, and this is the point of the whole thing: web tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a ml tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a agent tag=2026.7.17 version=2026.7.17.1657 rev=57e5243 extension tag=2026.8.27 version=2026.8.27.1547 rev=a7e626a The agent is six weeks behind because agent/fc_agent has not changed since 57e5243. Today it rebuilds and re-tags on every push regardless; from step 4 it will not. Three outputs, because they answer different questions and conflating them is how this goes wrong: tag YYYY.M.D the published image tag. Day precision, per the operator: same-day work is not worth pinning, so a second build that day replaces the first. version YYYY.M.D.HHMM the ordering key. The extension needs this and cannot use `tag`: Firefox compares it to decide whether an update exists, so two same-day builds must be distinguishable or the second hits the ext-<version> cache and ships stale bytes. That is issue #2397's failure mode exactly. revision <sha> content identity. Because `tag` is only day-precise, "does this tag already exist" cannot decide whether a build can be skipped — two different builds legitimately share a tag. Step 4 keys on this instead. Path sets read from the Dockerfiles rather than guessed. Notable calls: - web includes the extension's packaged set, because build.yml bakes the signed XPI into frontend/public/extension/ before the docker build. Miss that and :latest serves a NEW extension under an unchanged web version. - web excludes frontend/test: vite builds from src/, index.html and public/, so a spec change lands in the builder layer but never in dist. - agent is agent/Dockerfile + agent/requirements.txt + agent/fc_agent, NOT agent/. README.md, ruff.toml and docker-compose.yml sit in that directory and never reach the image. - every set includes its own Dockerfile and requirements: a base-image bump changes the artifact as surely as a source edit does. - the extension's set is read from packaging.sh, not restated. One definition, per #2397. tests/test_artifact_paths.py guards both directions of being wrong, since both are silent. Too narrow — a COPY'd file missing from the set — means the version does not move when the content does, and a pin serves stale bytes. Too wide means re-versioning for a change the artifact does not ship. The test parses each Dockerfile's COPY lines and compares them against the declaration, so adding a COPY without updating the set fails the lane. No workflow reads any of this yet. Step 2 shadows it.	2026-08-27 21:36:17 -04:00
bvandeusen	0db38cc111	ci: log in to the registry with the docker CLI, not docker/login-action ... build-ml failed at the login step twice on a7e626a, five seconds in, with MODULE_NOT_FOUND on the action's own dist/index.js. Not the token — the secret resolved to *** and the action never ran far enough to use it. The cause is a race in act_runner's shared action cache, not corruption. A remote action is cached at one /root/.cache/act/<hash> per runner, and build-web, build-ml and build-agent all start in the same second and all want docker/login-action@v3. One job re-clones that directory — emptying and repopulating it — while another walks it to copy into its container, and the walker lstat()s a file that just vanished. The two failures named DIFFERENT missing files, eslint.config.mjs then jest.config.ts, which is what rules out a bad cache entry and points at the race: a dangling entry would name the same file every time. Re-running does not help, because the re-run starts the three jobs simultaneously again. It reproduced immediately. Dropping the action removes FC from that race for this step. Logging in is one command, the docker CLI is already in the CI image per ci-requirements.md, and the same reasoning as family rule 5 applies: a marketplace action buys nothing when the tool is baked into the image the workflow already selected. Password on stdin, never as an argument — an argument lands in the process table and draws docker's own deprecation warning. This narrows the exposure rather than closing it. All three jobs also share docker/build-push-action@v5 and can race on it the same way; that one has not lost yet, and replacing it means hand-rolling buildx invocation including the build-args and provenance handling, which is a bigger change than this failure justifies. Recorded on #3118. Live consequence being cleared: fabledcurator-ml:dev was left a commit behind fabledcurator:dev, which is the stale-pairing trap the trigger comment on 239b1ed warns about.	2026-08-27 12:08:46 -04:00
bvandeusen	a7e626a67a	feat(extension): report the channel beside the version (step 7) ... Closes the half of the ask the signing work didn't: a way to tell a dev build from a main one. FC_CHANNEL is baked into the web image at build time and /api/extension/manifest reports it as its own key, next to version — the popup banner, the toolbar tooltip and the Settings card all name it. Beside the version, never inside it. A `1.0.3499884-dev` suffix is the obvious shortcut and it is the exact failure this design comes from: versionIsNewer parses each dotted segment with parseInt, so a suffixed segment reads as 0, every dev build compares equal to every other, and "no update available" stops being distinguishable from "I cannot read this version". The comparator already degrades rather than discarding (rule 150), which is a reason not to NEED the suffix, not a licence to add one. Two tests hold the line — one backend, asserting version and channel are separate keys; one frontend, asserting the rendered version text stays the bare derived number. Optional on the read side, and absent rather than defaulted. An image built before this field says nothing by not having the key; an image built without a channel now says nothing the same way, so there is one absence to handle instead of a second spelling of "unknown". Every reader drops the label entirely when it is missing and reads exactly as it did before. Reported verbatim rather than validated against {dev, main}: if an image declares something else, showing what it claims helps whoever is debugging more than dropping it would. FC_CHANNEL is declared LAST in the Dockerfile. An ARG invalidates every layer below it, and this is the one value that differs between the dev and main builds of identical source — earlier, and the two channels could never share a cached pip install. A tag push counts as main: a vYY.MM.DD tag is cut from main, so that image is a main-channel artifact wearing an immutable name. No channel switcher, deliberately. background.js:34 already records that Firefox's static update_url cannot apply, because every FC instance is a different host — so the extension asks its configured backend, and the channel IS the instance it points at. Switching is repointing apiUrl and reinstalling from that host. A separate setting would contradict each server build shipping its own extension. This commit touches packaged extension files, so it moves the derived version and will sign a new one via AMO — the first push to exercise the extension-changed path from dev end to end.	2026-08-27 11:47:30 -04:00
bvandeusen	fe48e77821	ci(extension): retire the manual-bump guard, true up the docs (step 5) ... The guard asked whether a packaged extension file changed without the version moving. Since step 4 nobody moves the version by hand, so it was checking a fact that had stopped existing — and it was not merely dead weight: it would have failed the lane on every real extension change, demanding a bump that decides nothing. Removed rather than left running beside the new mechanism (rule 22). What replaces it is thinner and true. The extension-version lane now asserts the derivation resolves on this commit, that the derived value is the plain dotted-numeric shape AMO accepts, and that MAJOR.MINOR agrees between manifest.json and package.json. MAJOR.MINOR is the one part still hand-set, and packaging.sh reads it from manifest.json ALONE, so a divergence ships a version package.json disagrees with. The lane keeps fetch-depth: 0 — checking that the derivation survives a real checkout is half its remaining value. Deliberately not checked there: that the derived value beats what is already signed. That guard belongs in build.yml, where it compares against the real ext-* releases. Comparing against origin/main in a lane would be wrong, because dev legitimately derives a LOWER value whenever main is ahead on the extension, and a lane that fails for being behind is a lane people learn to ignore. packaging.sh is down to two consumers from three. version.spec.js's "ci.yml derives its pathspec" test would have gone red on that, so it is rewritten to assert the property rather than the consumer: no workflow inlines an :(exclude)extension/ literal, across all three. That keeps the #2397 anti-regression value while surviving consumers coming and going. A second test pins build.yml to packaging.sh version and fails if it goes back to grepping the committed value — which is not a style regression but the #3092 bug itself. build.yml joins extension.yml's trigger paths, since the suite now asserts against it. The lockstep test narrows from the whole version string to MAJOR.MINOR. The committed patch numbers are inert now; asserting on them would fail for a difference that changes nothing. Docs. extension/README.md's Release section described extension.yml signing on main and committing the XPI into frontend/public/ — untrue since 2026-05-25, and it told the reader to hand-bump both files, which is now exactly the wrong instruction. Rewritten, with a Versioning section that says plainly that editing the patch number does nothing and why the key is commit time rather than a count. ci-requirements.md drops the third packaging.sh consumer and names every job that needs full history. Root README no longer claims the extension is signed on main only.	2026-08-27 11:29:44 -04:00
bvandeusen	9eb946b21b	ci(extension): sign on dev too, and bundle the XPI into :dev (step 6) ... The step the milestone exists for. sign-extension ungates from main-only to main-or-dev, and build-web downloads the XPI on dev as well, so a dev push produces an image carrying the extension that is being developed rather than requiring a merge to try one. Not two signatures. The version is the commit TIME of the newest packaged extension change, so dev and main derive the SAME number for the same source. A dev push that changes the extension signs it; the merge to main finds the ext-<version> release already there, hits the cache, and bundles the byte-identical XPI into :latest with no second AMO call. One signature per extension CHANGE, shared by both channels. That property is what makes two channels affordable at all, and it is why step 4 had to land first: ungating this while the version was still the hand-set 1.0.11 would have found the existing ext-1.0.11 release, skipped AMO, and bundled main's stale XPI into :dev — a dev channel confidently serving old code. Tags stay excluded. The tag path deliberately skips signing and polls for the release instead (the 2026-05-27 race). The ext-<version> release's target_commitish moves from the literal "main" to $GITHUB_SHA. Either branch can create that release now, and tagging a dev-signed XPI against a main commit that need not even contain the source it was built from is a lie that costs nothing to avoid. Known, not addressed here: two concurrent builds that both derive the same unsigned version will both call AMO and the loser gets a 409. The window already existed between main and tag pushes; dev signing widens it. It fails loudly rather than shipping anything wrong, and the rollback trap cleans up the empty release. Filed separately. Also unchanged here: ci.yml's manual-bump guard is still in place and still false. It does not fire on this commit — nothing packaged changed — but it will fail the lane on the next extension change, demanding a bump that no longer decides anything. Step 5 next.	2026-08-27 10:56:58 -04:00
bvandeusen	5447a40e97	ci(extension): the derived version drives signing (milestone 271 step 4) ... Cutover. sign-extension no longer reads the version out of the repo — it runs packaging.sh version and stamps the result into manifest.json and package.json in the working tree before web-ext sees them. Never committed back: the commit carrying the bump would itself be a change to the extension and would move the version again. Shadow mode ends here, in both build.yml and ci.yml. It had one job — validate the formula at zero cost before a real AMO version was burned — and CI confirmed it on 239b1ed: shadow: manual=1.0.11 derived=1.0.3499884. build-web re-derives rather than being handed the value, so it gains fetch-depth: 0. It was the outstanding landmine: a depth-1 clone derives a WRONG, too-low version rather than failing, and would then 404 fetching a release that exists under its real name. sign-extension and extension-version already had full history. New guard, and it stays permanently: refuse to sign when the derived version is strictly OLDER than the highest ext-* release already signed. Firefox rejects a downgrade and AMO never releases a burned version, so backwards is unrecoverable — it strands every install that took the higher one. Strictly older, not older-or-equal: equality is the ordinary case, an unchanged extension deriving the same version it did last build, which is exactly what makes the ext-<version> cache hit and holds AMO to one call per extension CHANGE rather than per push. The release list is paginated because ext-* shares it with the v* tags, and the bound fails rather than calling the highest it happened to see the highest there is. First derived value is 1.0.3499884 against a highest-signed ext-1.0.10, so the backfill direction is right by six orders of magnitude. 1.0.11 sits in the repo and was never signed; nothing is stranded by skipping past it. Still main-only. Step 6 ungates sign-extension to dev, which is what actually puts an XPI on :dev. Note for step 5: ci.yml's manual-bump guard is now false. It still demands a hand bump when a packaged file changes, and that bump no longer decides anything — the derived value overwrites it at build time. Harmless but pointless, and it should be retired before the next extension change.	2026-08-27 10:45:17 -04:00
bvandeusen	239b1ed8d9	ci: build :dev images again so the dev channel can carry a build ... build.yml triggered on main and tags only. The 2026-05-26 comment gave the reason: "operator tests from :latest after merge-to-main, not from the dev branch image. Saves one full docker build per dev push." That trade has since been named as a fault. Family rule 147 — main IS production, test on :dev, never by shipping — and rule 146 — a rolling channel refreshes itself, and a channel that can only be refreshed by shipping is not a channel. 146's note on 147 describes this exact shape: the pressure to test by shipping does not come from carelessness, it comes from :dev being unable to carry the build. Two live consequences, not hypotheticals: - docker-compose.yml pins fabledcurator:dev, an image nothing has published since May. The registry-image path of the documented quick-start could not have worked. - trying an extension change required merging to main, because sign-extension is gated to main and :dev did not exist to carry an XPI. Shipping was the only way to test. All three images build on dev. Deliberate: a :dev web image paired with a stale :dev ml or agent is a worse trap than no dev channel, because the mismatch surfaces as a runtime failure rather than a missing tag. The cost the 2026-05-26 note was avoiding is real and is now paid on every dev push — layer reuse should keep ml's cost to the COPY layers, but if it bites, narrowing is a `paths:` filter away. :dev only. The dev path never writes :c-<sha>: that is the rollback unit (rule 145), and a rolling tag may legitimately carry newer contents than the :c-<sha> of the same commit. This does NOT yet put an XPI on :dev — sign-extension is still gated to main, and ungating it has to wait for the derived version to control publishing, or dev would sign the hand-set 1.0.11, hit the existing cache and ship main's stale XPI. That is the next step.	2026-08-27 09:26:58 -04:00
bvandeusen	cd5444e3ae	ci(extension): derive the version from commit TIME, not commit count (#3092) ... Rule 149: an artifact's ordering key must be time-derived, never a commit count. packaging.sh's cmd_patch was a count. Why that matters here rather than in the abstract. A count is per-branch: dev and main count different histories of the SAME code. Today only main signs, so nothing has ordered the two against each other and the fault is invisible. The moment dev also publishes an extension, the two versions order by which branch accumulated more commits rather than by which is newer — and a squash-merge makes it permanent, because main gains one commit where dev gained five. dev then climbs away from main and a dev install can never cross back. That is Roundtable's 2026-08-24 incident (Scribe #2993) in a different repo: their versionCode was the branch's commit count, and it produced a channel you could enter and not leave. Measured on this repo today the old formula gives main=23, dev=24 — one apart, which is exactly how the inversion stays invisible until it strands somebody. New formula: minutes since 2020-01-01 of the LATEST commit touching a packaged extension file. Same anchor and unit Roundtable settled on. Commit time, not build time, and the difference is load-bearing: - stable while the extension is unchanged, so the ext-<version> signature cache still hits and AMO is called once per extension CHANGE rather than once per push. Build-time minutes would re-sign on every push and never let two channels share a signature. - after a merge, main sees the same commit and derives the same number, so :latest reuses the signature :dev already produced for byte-identical code. Same code, same version, one signing. - monotonic: max() over a set that only gains members. Verified across all 24 extension-touching commits, zero non-monotonic steps. - reproducible from any checkout. Derives 1.0.3499884 on dev, 1.0.3465860 on main — both far above the last hand-set 1.0.11, so milestone 271's backfill guard is satisfied by construction rather than by an offset. Still shadow-only: nothing reads the derived value yet. Both shadow steps log it, and ci.yml's runs on dev too, so both channels' numbers are visible — that is the pair that has to stay ordered. Prior shadow observations describe the OLD formula and prove nothing about this one, so the window restarts; ci.yml says so at the step. New requirement recorded in ci-requirements.md: a depth-1 clone derives a wrong, too-low value rather than failing, so fetch-depth: 0 is load-bearing wherever packaging.sh version is called. Refs #3092, milestone 271	2026-08-27 09:26:58 -04:00
bvandeusen	5a0e1bbd03	perf(ml): batch the auto-apply sweeps' image_tag inserts (#3072) ... Item 1 of #3072. Both sweeps issued a single-row pg_insert(image_tag) from inside their per-image loop. Steady state that is nothing; a first sweep over a back-catalogue is one round-trip per applied tag, tens of thousands of them. Each chunk now collects its rows and writes them in one statement. The ticket suggested one insert per chunk PER TAG. A single multi-row VALUES carries every tag at once, so it is one statement per chunk full stop — and the sweeps already accumulate across all heads before they commit, so nothing had to be restructured to allow it. Not a new helper: wip_title.apply_wip_image_tags was already doing the chunked ON CONFLICT DO NOTHING insert, so that shape is extracted to services/image_tag_apply.insert_image_tags and all three writers share it. The extraction deliberately leaves wip_title's pre-SELECT behind rather than pulling it into the shared function — the sweeps don't need it (their `skip` sets already exclude applied and rejected images) and it exists only to produce an accurate count, which the sweeps also compute themselves. So the shared primitive returns nothing: psycopg reports rowcount -1 for a multi-row ON CONFLICT DO NOTHING insert, and a count taken from the statement would be a lie rather than an approximation. Ordering note for the system-tag sweep: tag rows are now written after that chunk's PresentationReview rows rather than interleaved before them. Safe — PresentationReview FKs to image_record and tag, not to image_tag. Chunk size stays 5000: 5000 rows x 3 bound params = 15000, inside Postgres' 65535-parameter ceiling with room to spare. tests/test_image_tag_apply.py covers the primitive directly, since it is now the single place three writers can be wrong at once — most importantly that a re-run never restamps a hand-applied tag's source, which would silently poison head training (it excludes the auto sources). Left alone: _insert_presentation_review is still per-row, and the retract path still deletes per-row. Both operate on sets that are small by construction, unlike the apply path. Refs #3072	2026-08-27 07:51:23 -04:00
bvandeusen	1ac448d881	refactor: four small cleanups from the review pass (#3072) ... Items 2-5 of #3072. Item 1 (the per-row sweep inserts) is separate. 2. .fc-bad was not merely duplicated — it is .fc-weak under a second name. Both local definitions were `color: rgb(var(--v-theme-error))`, identical to the global .fc-weak, and GpuAgentCard was already using .fc-weak to colour exactly what GpuActivityPanel coloured .fc-bad (an errored count, red when non-zero). So rather than promoting a synonym to app.css, both call sites now use .fc-weak and the local defs are gone. app.css's status-colour comment records why there is no .fc-bad, next to the existing note on why .fc-ok is deliberately NOT global. 3. GalleryItem.vue's obsidian literals now use --v-theme-background, which IS obsidian (vuetify-theme.js maps background -> surfaces. obsidian). Preferred over --fc-chrome-rgb: same value, but that variable is named for the nav fade, not for the palette entry. The ticket said these were the only three real uses in the tree. They are not — GalleryItem itself had two more in the artist-label gradient (fixed here, so the file is now consistent), and ~13 more live in SeriesView, SeriesReaderView, ImageViewer, ArtistHeader, ExploreView and GalleryFilterBar. Those are a separate sweep, filed rather than folded in here. 4. The attachment download path had two hand-formatted copies. One definition now, `attachment_download_url`, next to the model both serializers already import. The test pins it by MATCHING the built path against the app's real URL map rather than comparing to a literal — a string-equality test would still pass after someone renamed the route, which is the drift the helper exists to prevent. 5. Extension API key now compares with hmac.compare_digest. Compared as BYTES, not str: compare_digest's str form raises TypeError on non-ASCII, and this value comes straight from an attacker-controlled header, so the str form would turn a junk key into a 500 instead of a 403. Low stakes either way — the API is unauthenticated-by-design on a LAN — but it costs nothing. Refs #3072	2026-08-27 07:48:18 -04:00
bvandeusen	bfc5135f19	docs: true up README — status, the five pieces, CI (#3070) ... README.md was last touched in 4aff9c5 (2026-05-14) and several of its most visible lines had gone false: - "Pre-v1. Not yet functional." — FC has been continuously deployed for months. Replaced with what main/dev actually mean for what's running. - "Node 22 pre-installed" — ci-requirements.md and extension.yml both say node 24, and frontend/package.json requires >=24. - "Runner label python-ci — a runner with Python 3.14, ruff and Node 22 pre-installed ... The runner image (runner-base:python-ci) is built from CI-Runner/CI-python/" — describes runs-on as selecting the toolchain. It doesn't: runs-on is a scheduling label, and every job names its own container.image (ci-python:3.14, or node:24-bookworm-slim for the extension lane). - "Both ci.yml and build.yml use this label" — there are three workflows. The CI section now points at ci-requirements.md rather than restating it, so the two can't drift apart again; that file is current and is the one the CI-runner process expects. Added a "What's in here" table for the five deployable pieces — the extension, the GPU agent and the ML image were unmentioned, three of the five. Also corrected the RELEASE_TOKEN write:release scope, which is no longer "for future release-cutting workflows": it backs the ext-<version> releases that cache the signed XPI. Refs #3070	2026-08-27 07:39:49 -04:00
bvandeusenandClaude Opus 5	89155478a8	test(refetch): cover the Layer-2 auto-refetch remediation (#3071) ... refetch_service was the only module under backend/app/services/ with no test file — and not an inert one: it runs unattended off the recovery sweep and deletes a file from disk before asking a downloader to replace it. The frontend cites it by name as the reason the Import tab could be retired ("imports heal themselves"). The ticket described it as having zero direct coverage. That is true of the module, but not of the code: test_api_import_admin.py already drives the happy path end-to-end through the refetch route — file deleted, task flagged, one dispatch, second attempt a no-op. These 17 tests therefore target what the route tests cannot reach rather than restating them: * every branch of resolve_refetch_source — disabled Source, a `sidecar:<platform>:<slug>` synthetic anchor, a platform mismatch, the gallery-dl `NN_` numbering-prefix sidecar, the lowest-id pick among several candidates, and each of the five ways it declines (no sidecar, unreadable JSON, non-object JSON, no platform, no artist folder / no matching Artist row). * that the file SURVIVES when nothing re-pollable resolves. This is the assertion the module exists for: `no_source` is the common case on a filesystem-only library, where the file on disk is the operator's only copy. The route-level no_source test cannot catch a regression here — its path never existed, so an unconditional unlink would pass it. * that the `refetched` bound is checked BEFORE the unlink, so a second sweep leaves the re-downloaded file alone rather than deleting it again. * that an unlink failure is logged and stepped over, not raised — a raise would abort the whole sweep for every other poison-pill row in the batch. Exercised with a real IsADirectoryError rather than a patched pathlib. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-27 07:33:14 -04:00
bvandeusenandClaude Opus 5	516521e7b0	refactor(platforms): drop migration 0088 — no deviantart rows exist (#3069) ... Operator confirms the instance has never used DeviantArt, so there is nothing for 0088 to quiesce. The migration only ever had two jobs — disable leftover `source` rows and delete a stale `credential` row — and both were guards against data that does not exist here. Removing it rather than keeping a no-op: a migration that runs on every deploy to touch zero rows is a permanent cost paid for a hypothetical, and it would read to a future reader as evidence that DeviantArt sources once existed. `platform` has no CHECK constraint, so retiring the key needs no schema change of its own. alembic head returns to 0087. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-27 07:26:02 -04:00
bvandeusenandClaude Opus 5	ddf896078c	refactor(platforms): retire deviantart end-to-end (#3069) ... Executes the 2026-07-05 product decision (FC downloaders = art-dedicated services only), which removed Twitter/X and Bluesky but left deviantart fully wired for seven weeks — the half-retired state rule 22 exists to prevent. Removed: the PlatformInfo module and its registry entry, the gallery-dl extractor block, extension_service's artist-page pattern, the extension's PLATFORMS + PLATFORM_ARTIST_PATTERNS entries, its manifest host permission and content-script match, the frontend icon/colour/label, and the operator- facing "supported platforms" list that still advertised it. Two judgment calls, both recorded in migration 0088: * existing `source` rows are DISABLED, not deleted. The row is the only record of the artist's DeviantArt URL. Disabling is also required for correctness rather than tidiness: with the platform unregistered the download path falls through to gallery-dl, which carries its OWN deviantart extractor, so an enabled row would have kept downloading from a dropped platform. * the `credential` row IS deleted — a live session cookie for a site FC will never call again. Adds the invariant whose absence is why manifest.json drifted in the first place: nothing tied its domain lists back to the platform table. The extension suite now asserts both directions, plus that no host permission belongs to an unclaimed domain (`*://*/*` exempted — FC is self-hosted at an operator-chosen URL the extension cannot enumerate). Extension version 1.0.10 -> 1.0.11: ci.yml's guard hard-fails a packaged extension change without a bump. No release is cut — build.yml's sign-extension job only runs on main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-27 07:24:08 -04:00
bvandeusenandClaude Opus 5	2e0f8f8c61	feat(cleanup): reclaim orphaned attachments — rows and store blobs (#3068) ... PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or artist left the row behind rather than taking it. Nothing ever pruned those rows, and nothing in the repo had ever unlinked a file under the attachment store — so both rows and bytes accumulated permanently, invisible to every existing diagnostic. Why a disk->DB reconciliation rather than a row sweep: the store is sha-addressed and idempotent, so ONE blob backs MANY rows. Deleting a row does not free its blob, and since the artist cascade (#3066) now deletes its attachment rows outright, a freed blob has no DB pointer left to find it by. Walking the store and asking "does any row still reference this sha?" catches orphans from every cause, including ones no future delete path will think to report. Preview and apply share `_orphan_attachment_conditions` (rule 93). The dry-run derives its surviving-sha set by NEGATING that same predicate, so it is honest about blobs the delete would free rather than counting them as still-referenced — the one place this was easy to get backwards, so it has its own parity test. Guards, each with a reason: - A blob is written before its row commits, so a just-stored file legitimately has no referencing row. Files under 6h are never judged — same guard and reasoning as ORPHAN_TEMP_MIN_AGE_HOURS. - `.partial` staging files belong to cleanup_orphaned_temp_files; skipped rather than raced. - The sha is parsed as the first 64 chars, not via Path.stem: store() takes the extension from the source filename, and a URL-encoded basename yields a multi-dot suffix that would make stem eat part of the sha. - A 900s walk budget reports partial=True instead of running to the task's hard limit (rule 89). - TASK_STUCK_THRESHOLD_MINUTES override at 30 (= time_limit 25 + 5). Without it a healthy 20-minute walk is phantom-flagged 'RecoverySweep' at the bare 5-min default — the #883 failure class; its invariant test is mirrored here. Defaults to the safe preview at both the task and the route, unlike the other maintenance triggers: this apply unlinks files. Operator-triggered only, never on a beat. Ships with its UI (rule 27): AttachmentReclaimCard in Cleanup → Duplicates & leftovers, built on the existing useMaintenanceTask/MaintenanceTile shapes, so a run survives navigating away. Surfaces files_failed and partial explicitly, since both change what the numbers mean. Also promotes humanBytes to utils/bytes.js — it was byte-identical in VideoDedupCard and GatedPurgeCard and this card would have been the third copy. The three divergent `formatBytes` helpers are deliberately left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 22:37:24 -04:00
bvandeusenandClaude Opus 5	2ce467e347	fix(cleanup): artist cascade preview counts posts and attachments (#3067) ... `project_artist_cascade` is documented as "a read-only projection of what delete_artist_cascade would touch" and drives the Tier-C confirm dialog, but it counted only images, sources, thumbs, import_tasks and bytes. It never counted posts or attachments, both of which the apply destroys. That is silent in the worst case. `Post.artist_id` is ondelete=CASCADE, so every post goes whether or not it carried an image — and FC has a large body-only post population (#1288 measured 694 pixiv posts with text and no images). Such an artist previewed as `images: 0`, reading as "empty, safe to remove", while the apply destroyed every captured body, description, external-link set and raw_metadata snapshot. The danger-zone card already promised "every image, source, post, and attachment" — the copy was honest and the numbers were not. Root of the drift: the preview re-derived its own predicates instead of sharing the apply's, the same shape as the 2026-06-08 fandom-tag deletion that rule 93 exists for. So rather than bolt on two counts, both halves now build from shared `_artist_{images,posts,attachments}_conditions` helpers, following the `_unused_tag_conditions` / `_bare_post_conditions` style already in the file. Sources keep no helper — the apply doesn't query them either, it gets them from the Artist.sources ORM cascade. The apply also now reports `posts_deleted` (counted before the delete, since the CASCADE leaves nothing to count after). Rule 93's second half asks for the apply to be tested, and parity is only assertable if both halves state the number. Adds a preview/apply parity test that runs both against one artist and asserts the three pairs agree AND that the rows actually went, plus a body-only-artist test covering the case that motivated this. The confirm dialog's counts grid renders every key, so posts and attachments surface there automatically; the prose summary line names posts explicitly, since that is the number that changes how an artist reads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 22:23:29 -04:00
bvandeusenandClaude Opus 5	39cf81aea6	fix(cleanup): clear an artist's attachments before the cascade delete (#3066) ... `delete_artist_cascade` could abort partway through, and it aborted after the irreversible half. Deleting an artist CASCADEs to Post (post.artist_id is ondelete=CASCADE), which SET NULLs post_attachment.post_id — and `uq_post_attachment_null_post_sha` is a partial UNIQUE on sha256 ALONE WHERE post_id IS NULL. So any two of that artist's attachments sharing a sha collapse onto one another and raise. That shape is ordinary, not corrupt: `_capture_attachment` deliberately writes one row per post over a single sha-addressed blob, so a creator who attaches the same pdf to two posts already has two such rows. A pre-existing filesystem-import row (post_id NULL) with the same sha collides on its own. The images and their on-disk files are deleted and committed in 500-row batches BEFORE the artist row is touched, so the failure landed after them: images gone, artist and posts alive, files unrecoverable. Fix: delete the artist's post_attachment rows explicitly first, matched by artist_id OR by the owning post's artist (artist_id is nullable, so neither arm alone covers every row). `_repoint_post_links` already guards the identical collision class in the reconcile path; this is its artist-cascade counterpart. Migration 0043 reasoned only about upgrade-time safety and never about this later SET NULL. The sha-addressed blobs are deliberately left on disk: one blob backs many rows, so unlinking needs a refcount pass, and this Tier-C op must not delete bytes its own preview never disclosed. Adds `attachments_deleted` to the summary, and two regression tests — the same sha on two posts, and an unrelated NULL-post row that must survive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 22:02:51 -04:00
bvandeusen	306de50f61	docs: Fabled-Git, not Forgejo, in ci-requirements ... The instance has run Gitea since the migration. Also fixes a dead rulebook pointer: the topic was renamed forgejo.md -> fabled-git.md, so the "CI philosophy" reference pointed at a file that no longer exists. Prose only — no workflow or path change. Scribe issue #2272.	2026-07-31 23:42:47 -04:00
bvandeusenandClaude Opus 4.8	57e52433d0	feat(agent): idle-unload GPU models to free VRAM when the queue is idle ... The SigLIP embedder + YOLO proposers load lazily then stay resident for the container's whole lifetime — a 24/7 agent with an empty queue squats on ~5GB of VRAM doing nothing (operator-observed: 4900MiB held at GPU-util 8% / P8). Sleep mode only sheds downloaders + poll cadence; even a UI Stop left the models loaded. Add a monitor thread that unloads the torch-owned models after cfg.idle_unload_seconds (env IDLE_UNLOAD_SECONDS, default 300; 0 disables) with the GPU genuinely idle (active==0, buffer drained, no job completed in the window), then torch.cuda.empty_cache() to hand the blocks back to the driver. They reload lazily on the next job via the existing _ensure_embedder / _proposers_for. Covers both sleep-mode idle and a full Stop. Surfaced in /status (models_loaded) and the agent UI pipe line; the VRAM meter drops too. Residual: imgutils CCIP/person ONNX sessions + the CUDA context stay resident (no clean unload API) — idle VRAM drops substantially, not to zero. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TbrA36zNczjVhrM6cWThQa	2026-07-17 12:57:31 -04:00
bvandeusenandClaude Opus 4.8	ec66ea5f83	refactor(ui): settings-card primitives + fix threshold clamp / card misgroup (#161) ... Tier-3 frontend DRY for the ML settings cards, plus the F-D2 clamp bug and the F-D3 card misgrouping. New primitives (components/common + composables): - <SettingToggleRow> — the accent-icon + .fc-section-h label + right-aligned switch row (HeadsCard x3, CropProposersCard). iconColor prop absorbs the on/off dim. - <SettingNumberField> — compact numeric field that CLAMPS to [min,max] on commit. This fixes F-D2: HeadsCard/CropProposersCard previously sent Number(raw) straight to the API, so an out-of-range threshold bounced off the 400 validator (only TranslationCard clamped). density prop for the grid cards. - useSettingSave(patchFn) — the busy + patch + toast + revert-on-failure flow each card hand-rolled (HeadsCard x6 handlers, CropProposersCard, MLBackfillCard, VideoEmbeddingCard). Returns ok/false for the optimistic-switch revert. Adopted in HeadsCard, CropProposersCard, MLBackfillCard (handler only — its plain labelled switch is a different affordance), VideoEmbeddingCard. F-D3: MLThresholdSliders.vue actually rendered a "Video embedding" (frame- sampling) card but sat under "Tagging → Suggestion thresholds". Renamed it VideoEmbeddingCard.vue and moved it to the "GPU agent & embeddings" section. Left deliberately (over-DRY guard): TranslationCard uses an inline error ALERT (not a toast), already clamps its confidence with a NaN fallback, and lives on the ImportStore — a genuinely different save pattern, so forcing it onto useSettingSave would change its UX. Behaviour-preserving refactor; CI has no Vue type-check so this needs a live UI pass (toggles persist + revert on failure, thresholds clamp on blur, video card now under Embeddings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 22:28:47 -04:00
bvandeusenandClaude Opus 4.8	e92570a31e	refactor(extension): DRY the web-root transform + cookie-export flow (#161) ... Behavior-preserving (extension JS has lint-only CI + your manual test): - api.webRoot() single-sources the baseUrl→web-root transform (strip trailing slash + /api) that was copy-pasted in background.js's self-update check and OPEN_ARTIST_PAGE, whose comments even cross-referenced each other. - exportPlatformCookies(key) shares the extract→verify→upload spine between EXPORT_COOKIES (single) and EXPORT_ALL_COOKIES; it returns a structured outcome so each caller keeps its own response/skip messages verbatim. - popup.js mutedNote(text) replaces the "centered muted note" div hand-rolled in the platform-loading, sources-loading, and empty-sources renderers. No version bump — no behavior change, so it rides the next real ext release rather than forcing an AMO re-sign + reinstall. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:57:14 -04:00
bvandeusenandClaude Opus 4.8	a2d1ed935d	refactor(ui): DRY the settings-card CSS tokens + fix unstyled headers (#161) ... - Promote .fc-section-h to a global token (app.css). It was copied identically into 4 cards, and TranslationCard used the class with NO local def — so its section headers rendered unstyled. Now fixed everywhere. - Promote .fc-good / .fc-weak status colours to globals; delete the local copies in the GPU/heads cards. (.fc-ok stays local — divergent: on-surface in HeadsCard vs success in QueuesTable. .fc-bad stays — different name.) - Delete 10 identical local .fc-muted redefinitions that crept back after the 2026-06-09 sweep; the global utility already covers them. - DbMaintenanceCard: opacity:0.6 muted text → the .fc-muted token (the exact anti-pattern that token's comment forbids). - HeadsCard: collapse byte-identical ratePct() into pct(). CSS-only + one template class swap; no logic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:51:23 -04:00
bvandeusenandClaude Opus 4.8	05df51b749	fix(ml): drop unnecessary quotes on MLSettings.load annotations (UP037) ... Python 3.14 evaluates annotations lazily, so the self-referential return type needs no forward-ref quotes — matches ImportSettings.load. Fixes the ruff lint lane on the DRY-pass push. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:49:30 -04:00
bvandeusenandClaude Opus 4.8	099e1e664c	refactor(patreon): DRY the campaigns-API request (#161) ... _lookup_via_api and resolve_display_name shared ~90% of their body (same endpoint, params, headers, error handling — differing only in which field they pluck from data[0]). Extract _campaigns_api_first(vanity, cookies_path) -> dict|None; callers pluck the campaign id vs the display name. Return-value behavior preserved (the display-name path additionally gains the helper's more granular warning logs). Covered by the existing test_patreon_resolver.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:41:34 -04:00
bvandeusenandClaude Opus 4.8	c87f8a1bb3	refactor(maintenance): DRY the recover_stalled head-run twins (#161) ... recover_stalled_head_training_runs and recover_stalled_head_auto_apply_runs were near-exact copies (coalesce-flip + keep-last-N prune, differing only in model + two constants). Extract _recover_stalled_runs(model, stall_minutes, keep_runs, label); the two tasks become thin wrappers. The other two recover tasks are deliberately NOT folded in (library-audit has no prune tail; backup uses a single started_at cutoff). test_recover_stalled_head_runs.py covers both wrappers (stalled→error, fresh survives) — previously untested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:41:34 -04:00
bvandeusenandClaude Opus 4.8	666b3a2ec8	refactor(ml): DRY pass — shared sweep helpers + table-driven settings (#161) ... Consolidate duplication accrued across the ML tagging + settings backend, behavior-preserving (over-DRY guard applied — the three auto-apply sweep BODIES stay separate; only their shared inner helpers are extracted). - _sigmoid / _conflict_scores / _insert_presentation_review (heads.py): the score→prob transform (6 inlined sites), the presentation conflict signal (2 sites), and the ring-loud PresentationReview insert (2 sites, single- sourced so the mode column can't drift on the shared composite PK). - _applied_or_rejected (training_data.py): the per-tag "applied ∪ rejected" skip-set, byte-identical at 3 sweep sites (heads.py x2, tasks/ml.py ccip). - ccip sweep divergence fixes: import ccip._FIGURE_KINDS + training_data._l2norm instead of local copies that silently drift when the canonical changes. - MLSettings.load / .load_sync classmethods (mirror ImportSettings); route all 8 scalar_one singleton reads through them (the session.get None-path stays). - GET serializers for MLSettings + ImportSettings are now table-driven off the same _EDITABLE tuples PATCH writes, so a new field can't be silently absent from GET (the split that historically dropped fields). - AUTO_APPLY_THRESHOLD_MIN/MAX constant single-sources the [0.5,0.999] operating range across the service clamp + the 5 API validators. - test_ml_dry_helpers.py pins _applied_or_rejected + _sigmoid. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 21:41:24 -04:00
bvandeusenandClaude Opus 4.8	d80a5255ed	feat(extension): in-app update prompt — popup banner + toolbar badge (#1489) ... The extension is installed per-instance from the operator's FC host, so Firefox's static update_url can't apply (each instance has a different host) and updates were fully manual. Add a self-hosted-friendly update surface that reuses the existing public GET /api/extension/manifest ({version, latest_url, sha256}): - lib/api.js: getExtensionManifest(). - background.js: checkForUpdateInfo() compares the instance's latest published version against runtime.getManifest().version (dotted-numeric compare so 1.0.10 > 1.0.9); CHECK_UPDATE message handler; refreshUpdateBadge() sets a toolbar badge via browser.action; a daily browser.alarms check plus on startup/installed. New 'alarms' permission (non-prompting). - popup: an 'Update available — vX' banner with an Update button that opens the signed XPI (web root, /api stripped like OPEN_ARTIST_PAGE) → Firefox's native install prompt. Never blocks the popup on a failed check. No backend changes (endpoint already exists). Bump 1.0.8→1.0.9 so this ships; from here on updates surface themselves instead of needing a manual reinstall. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 18:28:06 -04:00
bvandeusenandClaude Opus 4.8	69b5637bd6	fix(extension): show Add-to-FC button on subscribed Patreon creators (#1485) ... Symptom (operator-flagged): the extension injected the Add-as-source button on a Patreon creator you had NOT subscribed to, but it disappeared once you were subscribed — the opposite of when it's useful. Root cause (extension logic, not Patreon security): Patreon serves a creator under three URL shapes — bare patreon.com/Atole, patreon.com/c/Atole, and patreon.com/cw/Atole (the 'creator workspace' URL you land on once subscribed; documented in patreon_resolver._VANITY_RE). The button's artist-page gate (PLATFORM_ARTIST_PATTERNS.patreon in platforms.js) and its byte-mirror probe pattern (_PLATFORM_PATTERNS in extension_service._derive) only matched the bare single-segment form and explicitly excluded c/. So the subscribed-view URL failed the gate → no button. The ingestion resolver already handled all three; only these two gates were too narrow. Fix: both regexes now accept optional cw/ and c/ prefixes and drop the strict single-segment end-anchor, so a creator's inner page (/cw/Atole/posts, /Atole/membership) also matches — robust to whatever exact shape the subscribed view uses. Nav-page exclusions (home/search/messages/notifications/library/ settings/posts + post permalinks) preserved. New unit test covers all three prefixes, sub-paths, and nav-page rejection (both regexes validated identically). Bump extension 1.0.7→1.0.8 so a fresh signed XPI ships the fix (also exercises batch-5 web-ext-10's AMO sign path end-to-end on the main build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 17:50:07 -04:00
bvandeusenandClaude Opus 4.8	51749e05db	chore(deps): update web-ext 8→10 (batch 5) (#1450) ... Renovate dep-dashboard batch 5. web-ext is the extension's build/lint/sign CLI (devDependency only; no extension source changes). Verified against the 8→10 changelog + FC's actual usage: - All CLI flags we use survive unchanged: --source-dir, --no-config-discovery, --ignore-files, --overwrite-dest (build), --channel/--api-key/--api-secret (sign). No removed/renamed flags for lint/build/sign. - v9's one breaking change (.js config files rejected) does NOT apply: we pass --no-config-discovery on every command and ship no config file. - Node: web-ext 10 baselines Node 22. The lint job runs on node:24-bookworm-slim; the load-bearing AMO sign job runs on ci-python:3.14 which installs Node 24 (CI-runner NODE_MAJOR=24) — both satisfy it. Sign is cache-skipped this push (extension version unchanged at 1.0.7) but is verified compatible for the next version bump. - The bundled addons-linter jumps to 10.1.0 — the extension.yml lint job (web-ext lint over the MV3 manifest) is the CI verifier for any new manifest findings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 16:30:11 -04:00
bvandeusenandClaude Opus 4.8	50d6c42207	fix(ui): softer chrome-gradient falloff + segmented media toggle (#1478) ... Two operator-flagged polish items from the Vuetify-4 review: Gradient: reshape the nav + sub-header fade from a near-linear ramp to a hold-then-soft-drop profile — hold high opacity (0.92 → seam 0.68) through the bulk of the chrome, then ease to transparent over a small section at the bottom with an intermediate stop, so it tails off softly instead of running a straight line into a hard edge. Raising the shared --fc-chrome-seam also makes the tab strips more legible over scrolling content. Media toggle: FC's global VBtn { rounded: 'pill' } default made Vuetify 4 pill-round each SEGMENT of the All/Images/Videos v-btn-toggle individually, so the rounded ends collided at the joins. Square the inner segments and clip the group to one 8px outline — a proper segmented control. Both are colour/border-radius only — no control height changes, so the filter bar height and the nav offset (--fc-nav-h) are untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 15:50:33 -04:00
bvandeusenandClaude Opus 4.8	67c7ca8603	fix(ui): measure real nav height (--fc-nav-h), stop Explore breadcrumb tucking under nav (#1481) ... The app uses a plain sticky TopNav (no v-main), and the nav's height was hardcoded as 64px in ~6 places: the Explore + Subscriptions full-height workspaces (height: calc(100vh - 64px)) and every sticky sub-header pinned beneath the nav (top: 64px — Gallery filter bar, Browse/Series/Settings tabs). Vuetify 4's MD3 sizing changed the real nav height, so 64px was wrong: the Explore workspace was sized taller than the space below the nav, overflowed the viewport, and its breadcrumb tucked under the (taller) nav on 1080p. TopNav now measures its own height via ResizeObserver and publishes it as --fc-nav-h on documentElement (default 64px in app.css). Every consumer uses var(--fc-nav-h) instead of the magic number, so the layout self-corrects to the nav's real height and stays correct as it reflows (per-view teleported actions, mobile breakpoint). Also tightens the new chrome-gradient seam — sub-headers now pin at the nav's exact bottom. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 15:27:37 -04:00
bvandeusenandClaude Opus 4.8	eed42a260a	fix(ui): one continuous chrome gradient across nav + sticky sub-headers (#1478) ... The TopNav and each sticky sub-header pinned beneath it (Gallery's filter bar, the Browse/Series/Settings/Subscriptions tabs bars) each painted their OWN dark-to-transparent gradient (Gallery) or a solid surface band (the rest), so the fade read as happening twice — dark, fade out, then dark again — instead of one gradient flowing from the nav down through the sub-nav. Operator asked to treat the sub-nav as part of the nav with a single gradient. New shared .fc-chrome-continues primitive (app.css): the nav fades from opaque to a shared --fc-chrome-seam alpha (on views flagged meta.stickyChrome), and the sub-header continues from that exact seam alpha to transparent over its own height. Both reference the same var so the alphas meet at the 64px boundary — no re-darkening, no doubling. Percentage stops keep it spanning the filter bar's expanding refine panel; the primitive's blur keeps tabs/controls legible where the old solid bars had none. --fc-chrome-seam is the single tuning knob. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 14:42:07 -04:00
bvandeusenandClaude Opus 4.8	61b14e8f65	fix(ui): cleaner active-tab indicator (drop odd v4 slider) (#1480) ... Vuetify 4's MD3 v-tab slider underline rendered wider than the tab and floated below it (operator-flagged in the v4 review). The active tab's text is already accent-coloured, so drop the slider and mark the active tab with a subtle accent fill + rounded top — a clean highlight, app-wide across all tabbed views. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 14:27:02 -04:00
bvandeusenandClaude Opus 4.8	447bf73519	fix(ui): remove redundant app-shell top gap (sticky nav double-offset) ... Every view showed a large empty band at the top (operator-flagged during the Vuetify-4 review; pre-existing). Cause: .fc-content had padding-top:64px to clear the navbar, but TopNav is position:sticky and already reserves its own space in the v-app flex column — the 64px was a fixed-navbar leftover that double-counted the offset. Removed it; content now flows directly below the sticky nav (and the full-height calc(100vh-64px) views no longer overflow). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 14:09:15 -04:00
bvandeusenandClaude Opus 4.8	6104452d2e	feat(deps): vuetify 3→4 (batch 4 phase B, #1449) ... Vuetify 4 is a Material-Design-3 styling refresh with revert snippets, not a component-API overhaul. FC's exposure was small: - Bump vuetify ^4, vite-plugin-vuetify ^2.1.0, vue ^3.5, engines node>=24. - Restore the dropped global CSS reset (minimal reset from the upgrade guide) in Vuetify's own low-precedence reset layer, so FC's margin-zeroing assumptions hold. - v-row prop→utility: 'dense' → density=compact (×3), align=center → class=align-center. - v-snackbar: multi-line removed → min-height=68. - v-autocomplete #item slot: item→internalItem (item now aliases raw) in TagPicker + GalleryFilterBar (item.raw.* → internalItem.raw.*). ACCEPTED (cosmetic, operator reviews live per plan #158): MD3 typography (text-body-2 ×73), non-uppercase buttons (v4 dropped the uppercase default), MD3 elevation. CI verifies BUILD only — the LOOK is the live-review pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 13:35:01 -04:00
bvandeusenandClaude Opus 4.8	b59828635e	chore(deps): pinia 2→3 + vue-router 4→5 (batch 4 phase A, #1449) ... Both are Vue-3-compatible majors with no code impact for FC: - vue-router 5: no breaking changes when not using file-based routing (FC uses a plain createRouter in router.js). - pinia 3: drops Vue 2 + deprecated APIs; FC uses string-first setup-syntax defineStore + no custom pinia plugins, so nothing to change. Phase A of the Vuetify-4 UI framework migration (milestone #158); Vuetify 4 lands in phase B. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 13:25:16 -04:00
bvandeusenandClaude Opus 4.8	fac5ae6ce5	feat(explore): reach dial to escape dense clusters + anti-revisit (#1476) ... The Explore walk got stuck in dense signatures — neighbours all too similar, so forward-arrow couldn't escape and Random was the only exit. Root cause: MMR only diversifies WITHIN the nearest ~400 pool; in a dense cluster that whole pool is near-identical, so there's no escape route in it. - gallery_service.similar(reach=0.0, exclude_ids=None): reach>0 widens the pool (cap 400→1000) and _reach_sample strides across an outward-growing distance span so the set handed to MMR spans near→mid-far (guaranteed escape routes), not just the tight cluster. exclude_ids drops already-walked images. Gallery 'more like this' (reach=0) is unchanged. - api/gallery similar: parse reach + exclude_ids. - explore store: default reach 0.4 (auto-diversifies without touching the dial), pass the breadcrumb as exclude_ids, setReach action. - ExploreView: a Near↔Far reach slider in the trail. - tests: _reach_sample math (deeper ranks with higher reach, near kept); similar exclude_ids drops walked + reach path runs clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 12:06:22 -04:00
bvandeusenandClaude Opus 4.8	af0d39ed52	feat(wip): soft title tier — sketch/doodle vocab + ring-loud audit (#1474) ... Extends WIP title-tagging to lower-precision cues (sketch/doodle/scribble) safely. - wip_title.py: soft matcher (word-anchored; sketchbook/kadoodle don't trip it); WIP_TITLE_SOFT_SOURCE + soft SQL prefilter; apply_wip_image_tags takes a source arg. - training_data._AUTO_SOURCES += 'wip_title_soft' → the soft tier is PROVISIONAL and never trains the wip head (a finished "sketch" can't pollute it). Only the hard tier (wip_title) + manual train. - ImportSettings.wip_soft_title_tagging_enabled (OFF by default, opt-in). Migration 0087. - importer: hard tier wins, soft is the fallback (source wip_title_soft). - backfill: refactored into a shared _backfill_wip_tier; hard always, soft when enabled. - heads.soft_wip_conflict_audit + daily beat: score soft-tagged images against content heads, flag ring-loud ones (PresentationReview mode=process) for the review strip — the operator's "measure if they got falsely tagged" safety. - api settings toggle; ImportFiltersForm soft toggle. - tests: soft matcher pos/neg; soft source not a training positive; audit flags ring-loud + spares quiet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-13 10:14:38 -04:00
bvandeusenandClaude Opus 4.8	d9a14e890d	feat(system-tags): process auto-tag settings UI + mode-aware review strip (#1464) ... Frontend for the system-tag refactor (milestone #157 step 6). - HeadsCard: new 'Auto-tag work-in-progress' section (enable + tag-confidence + conflict knobs) for wip/editor process auto-apply, mirroring the chrome card; copy notes they stay VISIBLE and the head only learns from titles/manual (no runaway). Presentation copy narrowed to banner-only. - HiddenReviewStrip: mode-aware — chrome flags read 'hidden as X / Keep hidden / Un-hide'; process flags read 'auto-tagged X / Keep tag / Remove tag'. Same endpoints (the backend returns mode), different words. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 23:22:13 -04:00
bvandeusenandClaude Opus 4.8	ad2a5fc5fe	feat(system-tags): process vs chrome groups + WIP provisional auto-apply (#1464) ... Backend for the system-tag behavior refactor (milestone #157). editor screenshot moves from chrome (hidden) to the PROCESS group (shown, like wip); wip+editor gain provisional auto-apply so they stop needing endless manual identification — without a runaway loop. - tag.py: split PRESENTATION_SYSTEM_TAGS → CHROME_SYSTEM_TAGS (banner) + PROCESS_SYSTEM_TAGS (wip, editor screenshot). - heads.py: generalize presentation_auto_apply_sweep → system_tag_auto_apply_sweep (mode chrome|process). Same Guard 1 (skip human/confirmed) + Guard 2 (ring-loud conflict → PresentationReview). process mode uses source 'process_auto' and does NOT hide (hide is a gallery-query effect of group membership). - training_data._AUTO_SOURCES += 'process_auto' → the head never trains on its own auto-applied output; only wip_title/manual train it (the runaway break). - ml_settings: process_auto_apply_enabled (OFF, opt-in) + threshold + conflict threshold. presentation_review.mode ('chrome'|'process'). Migration 0086. - gallery_service: default-hide reads CHROME only (editor now shows); Explore neighbors exclude the whole PROCESS group. - tasks/ml + celery beat: scheduled_process_auto_apply (daily, opt-in); prune covers both modes. - api: ml_admin process_* CRUD+validation; hidden-review returns mode. - tests: rename chrome sweep calls; new test_process_auto_apply (apply, guards, mode flag, no-self-train); gallery test asserts editor now visible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 23:15:59 -04:00
bvandeusenandClaude Opus 4.8	0da0e47784	fix(wip-title): count inserts via pre-SELECT, not driver rowcount ... apply_wip_image_tags relied on result.rowcount, but psycopg reports -1 for a multi-row INSERT ... ON CONFLICT DO NOTHING (executemany path), so the return count (and the backfill's reported total) was wrong. Compute the count from a pre-SELECT of already-tagged ids within the same transaction; keep ON CONFLICT DO NOTHING as a race-safety belt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 12:22:25 -04:00
bvandeusenandClaude Opus 4.8	503c8854bc	style(importer): sort wip_title import into the local-import block ... Ruff I001 fixup for 5719387 — the new .wip_title import belongs after .thumbnailer (alphabetical), not after .archive_extractor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 12:16:11 -04:00
bvandeusenandClaude Opus 4.8	571938781a	feat(tagging): title-based WIP auto-tagging (#1458) ... Auto-apply the `wip` system tag to posts whose TITLE explicitly declares work-in-progress ("WIP" / "work in progress") — a deterministic, high-precision complement to the image-based ML `wip` head. WIP images are excluded from the Explore/gallery browse, so honouring the artist's own label keeps unfinished pieces out of the main browse. - services/wip_title.py: precision-first token-anchored matcher (swipe/wiped never trip it) + sync apply helpers (source='wip_title', ON CONFLICT DO NOTHING, chunked under the psycopg param ceiling). - importer: live hook on FRESH import only (never on deep-scan/supersede), so a manually-removed WIP tag is never re-applied by a routine re-scan. - maintenance.backfill_wip_title_tags: operator-triggered back-catalogue sweep (coarse SQL prefilter + regex confirm, keyset-paginated). Deliberately NOT a beat — a periodic re-run would silently undo manual removals. - ImportSettings.wip_title_tagging_enabled (default ON, migration 0085) gating the live hook; GET/PATCH + POST /settings/wip-title/scan. - Settings UI: toggle + "Scan existing posts" button. - Tests: pure matcher unit tests + integration (apply idempotency, backfill precision). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-12 12:12:19 -04:00
bvandeusen	0cf3a02797	Merge pull request 'chore(deps): update node.js to v24' (#220) from renovate/node-24.x into dev	2026-07-11 21:38:33 -04:00
bvandeusenandClaude Opus 4.8	c98db303d0	chore(frontend): drop dead vue-tsc devDep + check script ... Frontend is pure JS (no .ts/JSDoc); CI never ran vue-tsc. Removed the devDep and its orphaned `check` script instead of bumping to v3, and updated the ci.yml comment to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 21:31:36 -04:00
bvandeusen	2d0fca8729	Merge pull request 'chore(deps): update frontend build/test toolchain (major)' (#219) from renovate/major-frontend-buildtest-toolchain into dev	2026-07-11 21:30:29 -04:00
bvandeusenandClaude Opus 4.8	7f5e0603de	ci(renovate): pre-merge CI for renovate PRs + group FE toolchain ... ci.yml: add pull_request trigger (base dev) so renovate/* PRs get validated before merge; dev→main flow unchanged (base main), no duplicate runs. renovate.json: group vite/vitest/plugin-vue/happy-dom/@vue-test-utils into one PR (version-coupled). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 21:21:17 -04:00
bvandeusen	49f6765326	Merge pull request 'chore(deps): update dependency structlog to v26' (#218) from renovate/structlog-26.x into dev	2026-07-11 20:00:09 -04:00
bvandeusen	b23b19bf58	Merge pull request 'chore(deps): update dependency gdown to v6' (#217) from renovate/gdown-6.x into dev	2026-07-11 20:00:06 -04:00
bvandeusen	c17a6e6c40	Merge pull request 'chore(deps): update docker/dockerfile docker tag to v1.25' (#216) from renovate/docker-dockerfile-1.x into dev	2026-07-11 20:00:02 -04:00
bvandeusen	64053a5f57	Merge pull request 'chore(deps): update dependency pgvector to >=0.5,<0.6' (#215) from renovate/pgvector-0.x into dev	2026-07-11 19:59:58 -04:00
bvandeusen	5010de6178	Merge pull request 'chore(deps): update dependency nh3 to >=0.3,<0.4' (#213) from renovate/nh3-0.x into dev	2026-07-11 19:53:11 -04:00
bvandeusen	3063af6a74	Merge pull request 'chore(deps): update dependency cryptography to v49' (#214) from renovate/cryptography-49.x into dev	2026-07-11 19:53:07 -04:00
bvandeusen	279b0560d9	Merge pull request 'chore: Configure Renovate' (#211) from renovate/configure into dev	2026-07-11 18:45:52 -04:00
bvandeusenandClaude Opus 4.8	aea2701c28	feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155) ... The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh == cached, verified by probe) confidently mis-detects short ASCII English like "... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a re-translate reproduced it. Confidence alone can't separate the 0.86 collision (genuine German lands there too), and single-word mis-flags sit at a confident 1.0 no floor catches. Two operator-approved levers: - Acceptance floor is now a live Settings value (ImportSettings. translation_min_confidence, default 0.90; surfaced in the Translation card), so it's tunable without a redeploy. _accept takes the threshold as a parameter. - Per-post sticky override (Post.translation_override: auto/force/original). 'force' stores a translation even below the floor (rescue a skipped legit-foreign title); 'original' keeps the original and clears any stored translation (kill a confident mis-flag no floor catches). The sweep honors it on every run and _reset_translations skips 'original', so the choice survives a Re-translate-all. POST /api/posts/<id>/translation-override applies it immediately (translate now when the service is up, else queue for the sweep). UI: PostTranslationControl on the posts-feed card. Migration 0084 (both columns + a CHECK on the override). The feed + provenance serializers expose translation_override. With a stricter floor the rollback finally works: raise it -> Re-translate all -> the 0.86 mis-flags are rejected and restored to the original; force / keep-original handle the residual either way. Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor cases); sweep force/original + re-translate-skips-original; override endpoint (validation, original clears, force queues when disabled, feed exposes it); settings min_confidence default/save/validate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy	2026-07-10 22:38:25 -04:00
bvandeusenandClaude Opus 4.8	a444cf82d1	refactor(tags): unify suggestion source — one canonical DB-tag dropdown, drop dead raw/alias machinery (#154) ... Every tag suggestion is a canonical DB tag now (tagging-v2 #114: heads + CCIP score EXISTING concept tags). The pre-heads apparatus for model-predicted tags that didn't exist in the DB — creates_new_tag / raw_name / via_alias, the /suggestions/alias endpoint + add_alias_and_accept, AliasPickerDialog, and the store's aliasAccept/removeAlias — was dead and is removed. The type-to-add dropdown was TWO row sources (server autocomplete + the image's ML suggestions) merged with a dedup that dropped the %-bearing suggestion row when the debounced server hit landed — the operator's "confidence % flickers then vanishes". Now it's ONE list of DB-tag matches, each annotated with the model's confidence (join by canonical_tag_id) when the tag was scored for this image. No dedup, no flicker; picking a suggested tag still records acceptance via TagPanel.findPending. Single per-image fetch: score_image now reports above_threshold per row (computed vs the head's own suggest cut, separate from the inclusion floor), so the rail makes ONE min=0 request and derives the panel (above_threshold) and the dropdown (all, text-filtered) client-side — the two /suggestions calls collapse to one. Manual "Create 'X' as <kind>" (novel typed names) is unchanged; the alias table + tag-side alias admin + auto-apply alias matching are untouched. Tests: gate/serializer assertions updated (above_threshold; dropped dead-field + alias-endpoint checks); frontend spec seeds via the single load and covers the byCategory/aboveByCategory split. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy	2026-07-10 15:22:51 -04:00
bvandeusenandClaude Opus 4.8	6ab7fd5c7f	tune(translation): set latin acceptance floor to 0.80 (#1376) ... Calibrated against fresh probes once Interpreter returned real langdetect confidence: genuine German detected at 1.0, a correctly-detected but ambiguous latin string at 0.86. Set _MIN_LATIN_CONFIDENCE to 0.80 (below that band) so legitimate ambiguous non-English still translates while genuinely-unsure guesses are rejected. Real langdetect also fixed the original mis-flag at the source, so this floor is a safety net, not the primary fix. Pin 0.86-accepted in the gate test to guard against bumping the floor back up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy	2026-07-09 16:40:02 -04:00
bvandeusenandClaude Opus 4.8	7ddf94220d	feat(translation): accept/reject gate on Interpreter's detection confidence (#1376) ... Interpreter now returns a real per-detection confidence (source stays "auto"), so curator can reject the mis-detections it was blindly storing — e.g. a short English title mis-labelled as German and rewritten into the archive. The gate consumes ONLY Interpreter's own reported detection — curator does no language detection of its own (Scribe rule 133): a field is stored when the engine actually translated it AND either the detected language is CJK (script-detected, reliably high — ja/ko/zh trusted outright, incl. pure-kanji Japanese that lands as zh ~0.75) or the reported confidence clears a latin-script floor (_MIN_LATIN_CONFIDENCE = 0.90). A latin detection below the floor keeps the original and marks the post handled; a missing confidence fails open. The client already sent source="auto" and parsed confidence, so this is purely the gate + tests. Tests: pinned interpreter-client test now asserts source stays "auto"; new pure-unit gate tests (CJK trusted / latin floor / case-insensitive / fail-open) in the fast lane; end-to-end reject-low-latin, accept-high-latin, accept-low-cjk sweeps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy	2026-07-09 16:17:56 -04:00
bvandeusenandClaude Opus 4.8	ffdbdbaf07	feat(translation): 8h sweep + drain button + detection probe (#1376) ... Throughput: translate_posts now runs every 8h (was daily) as the steady-state cadence for newly-imported posts, and the Settings "Translate now" button runs it in drain mode (run-until-done, no reset) so one press clears the whole untranslated backlog instead of a single 300-post chunk. The interrupt/backoff re-enqueue now preserves the drain flag so a bulk drain resumes cleanly after an Interpreter restart. Misdetection groundwork: surface the detector's confidence from the Interpreter client (it was in the detectedLanguage payload but discarded) and add a read-only "Test translation" box — POST /settings/translation/ probe + TranslationCard UI — that shows detected language + confidence + engine + result for pasted text, without saving. Lets the operator see why a short/abbreviation-heavy English title gets mis-detected so the detection guard (min-length + confidence floor) can be tuned from real numbers. The guard itself follows once the mis-detected cases are probed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy	2026-07-08 20:13:49 -04:00
bvandeusenandClaude Opus 4.8	a017771621	feat(ui): thin kind-coloured border on tag chips so they don't wash out ... The tonal fill (esp. character = info) is intentionally faint and blends into the dark tag rail. Add a thin border in each chip's own kind colour via color-mix on currentColor (the tonal chip's themed foreground), defining the edge without changing the fill. Theme-aware in both light and dark. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-08 10:31:44 -04:00
bvandeusenandClaude Opus 4.8	25e555cab6	fix(ui): stop leading tag-chip icon clipping on the left edge ... The larger size=default chip widened Vuetify's negative start-margin on the leading kind-icon, placing it left of the .v-chip__content box whose overflow:hidden (the name-truncation guard) then clipped its left edge. Zero the icon's negative inline-start margin so it sits inside the clip box; the chip's 12px padding keeps a comfortable left inset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-08 10:29:28 -04:00
bvandeusenandClaude Opus 4.8	3c7ab44e74	feat(translation): per-field language detection for mixed-language posts ... _translate_one translated [title, description] in ONE Interpreter call and keyed the whole-post passthrough on the aggregate detected_lang (the FIRST item). So an English title + non-English description detected "en" and marked the post handled, leaving the description untranslated. Now each field is translated independently (its own detected_lang / passthrough) and the non-target field is stored on its own; translated_source_lang reflects the translated field's language. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 23:31:04 -04:00
bvandeusenandClaude Opus 4.8	06f98acf3e	feat(tagging): bolder auto-tag accept/reject + larger tag chips in the rail ... The in-pill ✓/✗ on unconfirmed auto-tags read muted — a faint colored icon on a tonal chip (worst on character tags, kind=info) that only lit up on hover. Make them solid green/red circles with a white glyph (22px, icon 15), mirroring the Suggestions rail's verdict buttons so accept/reject read identically. Also bump the applied-tag chips from size=small to default and the leading kind icon to match — bigger, clearer tags throughout the rail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 23:17:18 -04:00
bvandeusenandClaude Opus 4.8	4371ddb7e7	feat(translation): live re-translate progress in the Settings card ... translation/status now reports `active` (a translate/retranslate sweep is running, from the TaskRun table) and `last_run` (the most recent finished run's task + status). The Settings card polls live while a sweep runs, showing a spinner + "Translating… N remaining" that ticks down, and flags a last run that ended in error/timeout. No migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:20:32 -04:00
bvandeusenandClaude Opus 4.8	40cc11be5b	feat(deploy): container healthchecks + Swarm rolling-update auto-rollback ... web gets a /api/health liveness check; workers a lenient celery-ping check. A shared deploy policy (update_config order=start-first, failure_action=rollback, monitor 90s; rollback_config; restart_policy) means a bad image that never goes healthy is rolled back automatically instead of taking the service down. Ignored by plain `docker compose up` (deploy: is swarm-only), so the dev override is unaffected. Assumes prod deploys from this file via docker stack deploy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:15:33 -04:00
bvandeusenandClaude Opus 4.8	0b78264d62	feat(maintenance): daily janitor for orphaned .part/.partial staging files ... Downloads/imports stage into <name>.part / <name>.partial then os.replace() into place, so a kill mid-write leaves a discardable temp — never a corrupt final. cleanup_orphaned_temp_files sweeps ones left behind under the images root, only older than 6h so an in-flight download's staging file is never removed. Daily beat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:15:33 -04:00
bvandeusenandClaude Opus 4.8	9eae636047	feat(translation): pooled Interpreter session + manual sweep resumes after drain ... - interpreter_client: shared requests.Session with a connect-only retry (connect=2, no status retries — we map 429/5xx ourselves) so a proxy reload is smoothed and the keep-alive connection is pooled across the sweep. - translate_posts: on an interrupt (drain), re-enqueue after the Retry-After hint / default backoff instead of waiting for the daily beat; self-terminates via the health gate. Steady-state one-chunk-per-run on success is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:10:05 -04:00
bvandeusenandClaude Opus 4.8	f8105046dc	feat(explore): exclude WIP-tagged work from the Explore rabbit-hole ... Explore's neighbour grid (/api/gallery/similar → gallery_service.similar) now takes an Explore-only exclude_wip flag that drops `wip` system-tagged images from the candidates, alongside the banner/editor presentation tags. The gallery's own "similar" button is unchanged (keeps wip, #1274) — only the Explore store passes exclude_wip=1. The anchor itself may still be a WIP; only neighbours are filtered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 22:10:05 -04:00
bvandeusenandClaude Opus 4.8	d631ed023c	style(translation): use datetime.UTC alias (ruff UP017) ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 21:04:17 -04:00
bvandeusenandClaude Opus 4.8	c64261593d	feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience ... Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs mid-flight. Give in-flight work room to drain and make interrupted work resume cleanly instead of stalling. - docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s / scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown can actually drain before SIGKILL. - celery_app.py: task_reject_on_worker_lost=True — a task killed past the grace window is re-queued (safe: idempotent + chunked, recovery sweeps re-drive stragglers). - interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter behind a reverse proxy no longer raises an opaque HTTPError. - translation.py: thread retry_after out of _translate_batch; retranslate_posts resumes after the Retry-After hint (or 60s default, capped 900s) on an interrupt with _reset_done=True, self-terminating via the health gate. - tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff. No migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 21:01:00 -04:00
bvandeusenandClaude Opus 4.8	1f6d94f51d	feat(translation): re-translate on model change — artist-scoped + global re-run (#146) ... retranslate_posts resets the 5 translation columns to NULL for a scoped set of posts (all, or WHERE artist_id IN ids) then reuses the untranslated sweep to re-run them, chasing the tail until drained (run-until-done). Interpreter cache keys on engine_version so a changed model re-translates, an unchanged one is cache-fast. Reset only happens when the service is configured+healthy so translations are never wiped when they can't be rebuilt. New POST /settings/translation/retranslate (artist_id | all=true). UI: per-artist 'Re-translate posts' on the Artist Management tab + 'Re-translate all' in the Settings Translation card, both with confirm dialogs. No migration (reuses m143 columns). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 20:03:19 -04:00
bvandeusenandClaude Opus 4.8	6a255482ea	feat(translation): "Test connection" button — on-demand Interpreter health check (#143) ... New POST /api/settings/translation/test pings /v1/health for a GIVEN base URL (not the saved one), so the operator can verify a URL before enabling it. TranslationCard gains a Test-connection button that reports reachable/unreachable inline and updates the status dot. Endpoint test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 13:00:37 -04:00
bvandeusenandClaude Opus 4.8	af5aa21e45	test(translation): API endpoint tests for status + run (#143 step 6) ... /settings/translation/status defaults (off → no health call) + /run 400-when- unconfigured + 202-when-configured (monkeypatched .delay). requests is already a backend dep, so no requirements/ci-requirements change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:39:43 -04:00
bvandeusenandClaude Opus 4.8	83c1745fd0	feat(ui): translation-forward post text + Settings card (#143 step 5) ... PostCard + modal ProvenancePanel show the English title/description by default when a translation exists, with a per-card "show original (<lang>)" toggle — translated bodies render as plain text, originals keep their sanitized HTML. New TranslationCard in Settings → Ingestion & filters: enable switch, Interpreter base URL (generic placeholder, no default host), target language, a reachability indicator + untranslated-posts count + "Translate now". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:38:41 -04:00
bvandeusenandClaude Opus 4.8	ead60978e3	feat(translation): expose translated fields in post feed + provenance payloads (#143 step 4) ... post_feed_service (card + detail) and provenance_service._post_dict now include post_title_translated, description_translated (card-truncated / detail-uncapped) and translated_source_lang, keeping the originals for the toggle. Feed serialization test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:31:46 -04:00
bvandeusenandClaude Opus 4.8	7a4de7278d	feat(translation): backfill sweep + beat + manual trigger (#143 step 3) ... tasks/translation.py — translate_posts: picks untranslated posts (title OR description non-empty), per-post [title, description] batch via the Interpreter client, stores translations + detected lang + engine_version; passthrough / already-target posts are marked handled with no stored translation. 503 or a connection error interrupts (retry next cycle), 400 stops (fix config), per-post commit keeps progress; wall-clock bounded. Wired into celery (maintenance_long lane) + a daily beat. No-op unless enabled + base URL set + healthy. GET /settings/translation/status + POST .../run for the Settings card. Task tests (stubbed client, monkeypatched session). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:29:28 -04:00
bvandeusenandClaude Opus 4.8	7f8073c4c8	feat(translation): Interpreter client — batch translate + health (#143 step 2) ... services/interpreter_client.py: sync (requests) client for the LibreTranslate- compatible /v1/translate — no new dep, mirrors the platform clients. translate() maps translatedText[]↔texts (order + length), returns detected_lang + engine_version (aggregate = first item, fine for a per-post [title, description] batch); passthrough items come back unchanged in their slot. InterpreterUnavailable on 503 / connection error (retry later), InterpreterBadRequest on 400. health() checks /v1/health engines.llm. 10 unit tests with mocked HTTP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:23:49 -04:00
bvandeusenandClaude Opus 4.8	a3bc98a53c	feat(translation): Post translation columns + settings + migration (#143 step 1) ... Post gains post_title_translated / description_translated / translated_source_lang / translation_engine_version / translated_at — filled by the translate sweep so viewing is instant. ImportSettings gains translation_enabled (OFF by default), interpreter_base_url (EMPTY — no default host; the operator points it at their own Interpreter proxy behind a reverse proxy) and translation_target_lang (en), exposed + validated via /settings/import. Migration 0083. Settings defaults + patch + validation test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:20:31 -04:00