Compare commits

...
Author SHA1 Message Date
bvandeusen fb2c4d5b80 fix(extension): packaging.sh decides the version, so it must move the version
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
Build images / build-ml (push) Successful in 4s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 18s
extension / lint (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 30s
Build images / sign-extension (push) Successful in 1m41s
Build images / build-web (push) Successful in 11s
CI / integration (push) Successful in 3m42s
extension / lint (pull_request) Successful in 21s
`packaging.sh pathspec` excluded `extension/scripts/**` — the same list
web-ext ignores. But the two lists answer different questions, and this is
the one place they disagree.

packaging.sh is not packaged into the XPI. It does decide the version
string, and build.yml stamps that string into the manifest.json that IS
packaged. Changing how the version is computed therefore changes the shipped
bytes, and the derivation has to see it.

Harmless while every push rebuilt the web image. Step 4 made the rebuild
conditional on the derived revision moving, which turns it into a silent
failure: a packaging.sh change yields a NEW version, so sign-extension misses
its ext-<version> cache and signs — while build-web sees an unmoved revision,
reuses the published image and ships the OLD XPI. One orphaned AMO signature,
and an instance serving code the registry calls current. Found while checking
the ground under step 5, which changes the version format and is exactly the
commit that would have hit it.

Split the list rather than widening the shared one: NOT_VERSION_RELEVANT
drives the pathspec, NOT_PACKAGED_TRACKED still drives web-ext's ignore
list, and scripts/ stays out of the XPI. The two directions are not
symmetric, which is why the version list is the narrower one — too wide
costs a re-sign and a rebuild for a change that ships nothing new, too
narrow serves stale bytes and says nothing.

No version churn: the last packaging.sh commit predates the current
extension revision, so the derived version is unchanged at 1.0.3500147 and
web's revision stays a7e626a67a.

Both suites now pin the disagreement from their own side, because the
tempting fix for either half is to make the lists one again — version.spec.js
asserts the pathspec does NOT exclude scripts while the ignore list still
does, and test_artifact_paths.py asserts packaging.sh is inside the extension
and web path sets.
2026-08-28 08:49:06 -04:00
bvandeusen 609bc82acc ci: reuse the published image instead of rebuilding it (milestone 313 step 4)
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 7s
CI / frontend-build (push) Successful in 17s
extension / lint (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m43s
Before building, each job asks the registry whether this artifact's content
is already published. On a hit it skips the build entirely and repoints the
channel and date tags at the existing manifest with `imagetools create` —
registry-side, no layer transfer, seconds. This is the step that stops a
push touching only `agent/` from rebuilding web and ml, and stops a merge to
main rebuilding what dev already built.

The question is asked with a new `artifacts.sh identity`, not with the date
tag: the date tag is day-precise and last-one-wins, so two different builds
share it and it cannot answer "is this content published?". The commit sha
would move on every push and never hit, which is the redundant rebuild being
removed. The revision does both jobs — content-unique, and stable across
pushes that did not touch the artifact.

Identity is channel-qualified for web and only for web, because web is the
only image that takes a build-arg: FC_CHANNEL is baked in and reported by
/api/extension/manifest, so its dev and main builds of one revision are
genuinely different images. ml and agent take none, which is what lets a
merge reuse dev's build rather than rebuilding the agent's CUDA image to
produce bytes that already exist. tests/test_artifact_identity.py reads the
Dockerfiles and fails if that list drifts from the ARG declarations, in
either direction — collapsing the channels ships an instance that reports
the wrong one, and splitting them needlessly rebuilds every merge.

Failure direction is deliberate: an inspect that errors for any reason reads
as a miss and the build runs. Only a real 200 skips one.

A tag-push never claims the identity. It rebuilds a revision main already
published, and image configs are not bit-reproducible, so re-pushing
r-<rev> would point an immutable tag at fresh bytes — rule 145's exact
prohibition. It publishes only its own :v... label and otherwise reuses.

Base-image freshness, decided rather than left implicit: an artifact whose
source stops moving stops picking up base updates under its pinned tag. That
is what a pin means, and rule 145 already says the refresh belongs on the
moving tag instead. Filed as #3154 rather than folded in here, because the
naive version regresses :latest on the next unrelated push.

ci.yml's backend lane gains fetch-depth: 0 — the new tests derive real
revisions, and on a depth-1 clone that derivation returns the tip sha or
fails, so the lane would go green while asserting nothing.

The three build jobs' shadow steps are renamed and re-commented: those
values stopped being informational at step 3, and a step captioned "nothing
reads this" beside steps that do is worse than no caption.
2026-08-28 08:24:58 -04:00
bvandeusen 7a20c55441 ci: publish a per-artifact date tag on main builds (milestone 313 step 3)
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-ml (push) Successful in 6s
Build images / build-agent (push) Successful in 6s
Build images / build-web (push) Successful in 5s
extension / lint (push) Successful in 17s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m44s
Each main build now also publishes <image>:YYYY.M.D — the date of the
commit that artifact's shipped files last changed in. Purely additive;
:main, :latest, :c-<sha> and the dev path are untouched, so reverting this
commit reverts the behaviour.

    main push -> :main, :latest, :c-<sha>, :2026.8.27
    dev push  -> :dev

Per artifact, so an image whose files did not change keeps the tag it
already had. On this commit the agent reads 2026.7.17 while web and ml
read 2026.8.27 — six weeks apart, from one push. Step 4 turns that into
not rebuilding it.

Day precision, and a second main build the same day replaces the first.
Operator's call, and the reasoning is theirs: same-day work is not
something worth pinning. A rollback goes to a day, not to the fourth merge
of a Tuesday afternoon. It also makes retention mean "the last N days"
rather than "the last N pushes".

CALVER is computed inside the main branch rather than at the top of the
step, and hard-fails when empty. There is no `set -e` here, so an
unconditional assignment that failed would have left it empty and published
the tag `fabledcurator:` — an invalid name, from a step that still reported
success. It is also simply unused on the dev and tag paths.

Fixed a stale comment while in this block rather than leaving it for step
7: it claimed release tags carry "no `.N` per family release-posture rule".
Rule 148 was amended on 2026-08-24 to REQUIRE the suffix, after the ban
caused a same-day tag to be retargeted and a release deleted to make room
(note 2813). FC's own tags already carry suffixes; only the comment was
asserting the superseded rule.

Verified before pushing: the derivation holds across 200 commits of real
history — a derived revision always touches its own path set, the version
never decreases along any parent->child edge, and web tracks all three
extension-only commits in the log. That last one is the direction that
would serve stale bytes on a pin.
2026-08-27 21:56:52 -04:00
bvandeusen 0c43fa3eb2 ci: shadow the per-artifact derived versions (milestone 313 step 2)
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-ml (push) Successful in 7s
Build images / build-agent (push) Successful in 7s
Build images / build-web (push) Successful in 6s
CI / frontend-build (push) Successful in 21s
extension / lint (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m44s
Every build job now logs the tag, version and revision its artifact would
get. Nothing reads them; no `set -e`, and each derivation falls back to
UNAVAILABLE, so a broken script cannot fail a build. Same discipline as
milestone 271 step 2, which is what made that cutover safe to do in one
commit.

Also fixes a landmine the plan named but had not checked: build-ml and
build-agent were checking out at depth 1. Both now use fetch-depth: 0.

That mattered more than it looks. A depth-1 clone sees one commit, so
`git log HEAD -- <shipped paths>` either returns that commit's timestamp —
plausible, and wrong — or returns nothing. For build-ml on this push it
would have returned today's date, because HEAD touches backend/, and
nothing downstream would have questioned it. For build-agent it would have
returned nothing at all, since no single commit here touches agent/, and
artifacts.sh exits non-zero rather than guessing. One direction is silent
and one is loud; only the loud one was ever going to get noticed.

What to read from the shadow lines over the next few pushes, in order of
how badly each would bite:

  * a push touching the extension must move BOTH the extension and web,
    because build-web bakes the XPI in. If web does not move, its path set
    is too narrow and a pinned web image will serve an extension it does
    not name.
  * a push touching only agent/ must leave web and ml still. If they move,
    their sets are too wide and they will rebuild for changes they do not
    ship.
  * a docs-only push must move nothing.
  * dev and main must derive the same values for the same source.

Step 3 only lets these values name a tag once those hold.
2026-08-27 21:41:14 -04:00
bvandeusen cf06c81db9 build: one definition per artifact of what it ships (milestone 313 step 1)
Build images / sign-extension (push) Successful in 4s
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 5s
CI / frontend-build (push) Successful in 41s
CI / backend-lint-and-test (push) Successful in 2m4s
CI / integration (push) Successful in 4m15s
Build images / build-web (push) Successful in 4m50s
Build images / build-ml (push) Successful in 5m43s
Build images / build-agent (push) Successful in 10m45s
scripts/artifacts.sh generalises what packaging.sh established for the
extension: four published artifacts, four path sets, four independent
versions derived from the newest commit touching each set.

Measured on this commit, and this is the point of the whole thing:

  web        tag=2026.8.27  version=2026.8.27.1547  rev=a7e626a
  ml         tag=2026.8.27  version=2026.8.27.1547  rev=a7e626a
  agent      tag=2026.7.17  version=2026.7.17.1657  rev=57e5243
  extension  tag=2026.8.27  version=2026.8.27.1547  rev=a7e626a

The agent is six weeks behind because agent/fc_agent has not changed since
57e5243. Today it rebuilds and re-tags on every push regardless; from step
4 it will not.

Three outputs, because they answer different questions and conflating them
is how this goes wrong:

  tag       YYYY.M.D        the published image tag. Day precision, per the
                            operator: same-day work is not worth pinning, so
                            a second build that day replaces the first.
  version   YYYY.M.D.HHMM   the ordering key. The extension needs this and
                            cannot use `tag`: Firefox compares it to decide
                            whether an update exists, so two same-day builds
                            must be distinguishable or the second hits the
                            ext-<version> cache and ships stale bytes. That
                            is issue #2397's failure mode exactly.
  revision  <sha>           content identity. Because `tag` is only
                            day-precise, "does this tag already exist" cannot
                            decide whether a build can be skipped — two
                            different builds legitimately share a tag. Step 4
                            keys on this instead.

Path sets read from the Dockerfiles rather than guessed. Notable calls:

  - web includes the extension's packaged set, because build.yml bakes the
    signed XPI into frontend/public/extension/ before the docker build. Miss
    that and :latest serves a NEW extension under an unchanged web version.
  - web excludes frontend/test: vite builds from src/, index.html and
    public/, so a spec change lands in the builder layer but never in dist.
  - agent is agent/Dockerfile + agent/requirements.txt + agent/fc_agent,
    NOT agent/. README.md, ruff.toml and docker-compose.yml sit in that
    directory and never reach the image.
  - every set includes its own Dockerfile and requirements: a base-image
    bump changes the artifact as surely as a source edit does.
  - the extension's set is read from packaging.sh, not restated. One
    definition, per #2397.

tests/test_artifact_paths.py guards both directions of being wrong, since
both are silent. Too narrow — a COPY'd file missing from the set — means the
version does not move when the content does, and a pin serves stale bytes.
Too wide means re-versioning for a change the artifact does not ship. The
test parses each Dockerfile's COPY lines and compares them against the
declaration, so adding a COPY without updating the set fails the lane.

No workflow reads any of this yet. Step 2 shadows it.
2026-08-27 21:36:17 -04:00
bvandeusen 0db38cc111 ci: log in to the registry with the docker CLI, not docker/login-action
Build images / sign-extension (push) Successful in 4s
CI / frontend-build (push) Successful in 28s
CI / backend-lint-and-test (push) Successful in 32s
CI / lint (push) Successful in 6s
CI / extension-version (push) Successful in 6s
extension / lint (push) Successful in 45s
Build images / build-web (push) Successful in 2m21s
Build images / build-ml (push) Successful in 2m59s
CI / integration (push) Successful in 4m9s
Build images / build-agent (push) Failing after 10m58s
extension / lint (pull_request) Successful in 27s
build-ml failed at the login step twice on a7e626a, five seconds in, with
MODULE_NOT_FOUND on the action's own dist/index.js. Not the token — the
secret resolved to *** and the action never ran far enough to use it.

The cause is a race in act_runner's shared action cache, not corruption.
A remote action is cached at one /root/.cache/act/<hash> per runner, and
build-web, build-ml and build-agent all start in the same second and all
want docker/login-action@v3. One job re-clones that directory — emptying
and repopulating it — while another walks it to copy into its container,
and the walker lstat()s a file that just vanished. The two failures named
DIFFERENT missing files, eslint.config.mjs then jest.config.ts, which is
what rules out a bad cache entry and points at the race: a dangling entry
would name the same file every time.

Re-running does not help, because the re-run starts the three jobs
simultaneously again. It reproduced immediately.

Dropping the action removes FC from that race for this step. Logging in is
one command, the docker CLI is already in the CI image per
ci-requirements.md, and the same reasoning as family rule 5 applies: a
marketplace action buys nothing when the tool is baked into the image the
workflow already selected. Password on stdin, never as an argument — an
argument lands in the process table and draws docker's own deprecation
warning.

This narrows the exposure rather than closing it. All three jobs also
share docker/build-push-action@v5 and can race on it the same way; that
one has not lost yet, and replacing it means hand-rolling buildx
invocation including the build-args and provenance handling, which is a
bigger change than this failure justifies. Recorded on #3118.

Live consequence being cleared: fabledcurator-ml:dev was left a commit
behind fabledcurator:dev, which is the stale-pairing trap the trigger
comment on 239b1ed warns about.
2026-08-27 12:08:46 -04:00
bvandeusen a7e626a67a feat(extension): report the channel beside the version (step 7)
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 5s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 32s
extension / lint (push) Successful in 28s
CI / integration (push) Successful in 3m52s
Build images / sign-extension (push) Successful in 4s
Build images / build-ml (push) Failing after 5s
Build images / build-agent (push) Successful in 13s
Build images / build-web (push) Successful in 2m4s
Closes the half of the ask the signing work didn't: a way to tell a dev
build from a main one. FC_CHANNEL is baked into the web image at build
time and /api/extension/manifest reports it as its own key, next to
version — the popup banner, the toolbar tooltip and the Settings card all
name it.

Beside the version, never inside it. A `1.0.3499884-dev` suffix is the
obvious shortcut and it is the exact failure this design comes from:
versionIsNewer parses each dotted segment with parseInt, so a suffixed
segment reads as 0, every dev build compares equal to every other, and
"no update available" stops being distinguishable from "I cannot read this
version". The comparator already degrades rather than discarding (rule
150), which is a reason not to NEED the suffix, not a licence to add one.
Two tests hold the line — one backend, asserting version and channel are
separate keys; one frontend, asserting the rendered version text stays the
bare derived number.

Optional on the read side, and absent rather than defaulted. An image
built before this field says nothing by not having the key; an image built
without a channel now says nothing the same way, so there is one absence
to handle instead of a second spelling of "unknown". Every reader drops
the label entirely when it is missing and reads exactly as it did before.
Reported verbatim rather than validated against {dev, main}: if an image
declares something else, showing what it claims helps whoever is debugging
more than dropping it would.

FC_CHANNEL is declared LAST in the Dockerfile. An ARG invalidates every
layer below it, and this is the one value that differs between the dev and
main builds of identical source — earlier, and the two channels could
never share a cached pip install. A tag push counts as main: a vYY.MM.DD
tag is cut from main, so that image is a main-channel artifact wearing an
immutable name.

No channel switcher, deliberately. background.js:34 already records that
Firefox's static update_url cannot apply, because every FC instance is a
different host — so the extension asks its configured backend, and the
channel IS the instance it points at. Switching is repointing apiUrl and
reinstalling from that host. A separate setting would contradict each
server build shipping its own extension.

This commit touches packaged extension files, so it moves the derived
version and will sign a new one via AMO — the first push to exercise the
extension-changed path from dev end to end.
2026-08-27 11:47:30 -04:00
bvandeusen fe48e77821 ci(extension): retire the manual-bump guard, true up the docs (step 5)
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 3s
Build images / build-agent (push) Successful in 8s
CI / frontend-build (push) Successful in 22s
extension / lint (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 29s
Build images / build-web (push) Successful in 1m57s
Build images / build-ml (push) Successful in 2m38s
CI / integration (push) Successful in 3m50s
The guard asked whether a packaged extension file changed without the
version moving. Since step 4 nobody moves the version by hand, so it was
checking a fact that had stopped existing — and it was not merely dead
weight: it would have failed the lane on every real extension change,
demanding a bump that decides nothing. Removed rather than left running
beside the new mechanism (rule 22).

What replaces it is thinner and true. The extension-version lane now
asserts the derivation resolves on this commit, that the derived value is
the plain dotted-numeric shape AMO accepts, and that MAJOR.MINOR agrees
between manifest.json and package.json. MAJOR.MINOR is the one part still
hand-set, and packaging.sh reads it from manifest.json ALONE, so a
divergence ships a version package.json disagrees with. The lane keeps
fetch-depth: 0 — checking that the derivation survives a real checkout is
half its remaining value.

Deliberately not checked there: that the derived value beats what is
already signed. That guard belongs in build.yml, where it compares against
the real ext-* releases. Comparing against origin/main in a lane would be
wrong, because dev legitimately derives a LOWER value whenever main is
ahead on the extension, and a lane that fails for being behind is a lane
people learn to ignore.

packaging.sh is down to two consumers from three. version.spec.js's
"ci.yml derives its pathspec" test would have gone red on that, so it is
rewritten to assert the property rather than the consumer: no workflow
inlines an :(exclude)extension/ literal, across all three. That keeps the
#2397 anti-regression value while surviving consumers coming and going.
A second test pins build.yml to packaging.sh version and fails if it goes
back to grepping the committed value — which is not a style regression but
the #3092 bug itself. build.yml joins extension.yml's trigger paths, since
the suite now asserts against it.

The lockstep test narrows from the whole version string to MAJOR.MINOR.
The committed patch numbers are inert now; asserting on them would fail
for a difference that changes nothing.

Docs. extension/README.md's Release section described extension.yml
signing on main and committing the XPI into frontend/public/ — untrue
since 2026-05-25, and it told the reader to hand-bump both files, which is
now exactly the wrong instruction. Rewritten, with a Versioning section
that says plainly that editing the patch number does nothing and why the
key is commit time rather than a count. ci-requirements.md drops the third
packaging.sh consumer and names every job that needs full history. Root
README no longer claims the extension is signed on main only.
2026-08-27 11:29:44 -04:00
18 changed files with 1510 additions and 193 deletions
+540 -59
View File
@@ -103,6 +103,27 @@ jobs:
# cache hit and holds AMO to one call per extension CHANGE. Only moving
# backwards is a failure, so this runs on every path — cache hit
# included — rather than only before a sign.
# --- shadow mode (milestone 313, step 2) -----------------------------
# Informational ONLY. Nothing reads this and it must never fail the
# build — no `set -e`, and every derivation falls back to UNAVAILABLE.
#
# What to watch across pushes, because this is what step 3 will trust:
# * a push touching only agent/ moves the agent and leaves web and ml
# STILL. If web moves, its path set is too wide.
# * a push touching only docs moves nothing.
# * a push touching the extension moves the extension AND web, since
# web bakes in the XPI. If web does not move, its set is too narrow
# — the direction that serves stale bytes on a pin.
# * dev and main derive the same values for the same source.
- name: Shadow — derived artifact version (informational)
run: |
set -u
A=extension
T=$(sh scripts/artifacts.sh tag "$A" 2>&1 || echo UNAVAILABLE)
V=$(sh scripts/artifacts.sh version "$A" 2>&1 || echo UNAVAILABLE)
R=$(sh scripts/artifacts.sh revision "$A" 2>&1 || echo UNAVAILABLE)
echo "derived: artifact=$A tag=$T version=$V revision=$R sha=$GITHUB_SHA"
- name: Guard — the derived version must never go backwards
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
@@ -320,6 +341,183 @@ jobs:
# that exists perfectly well under its real name.
fetch-depth: 0
# --- derived values, one line (milestone 313) ------------------------
# These stopped being shadow output at step 3: `tag` is published on
# main and `revision` decides whether the build below runs at all. This
# step prints all three anyway, because the load-bearing steps each
# print only the one they use, and on dev the date tag is computed
# nowhere else. When a build is skipped or a pin looks wrong, this is
# the line that says what the commit derived.
#
# Still diagnostic, so it still must not fail the build — no `set -e`,
# and every derivation falls back to UNAVAILABLE. A broken echo must
# never be the reason an image does not ship.
#
# What it should say:
# * a push touching only agent/ moves the agent and leaves web and ml
# STILL. If web moves, its path set is too wide.
# * a push touching only docs moves nothing.
# * a push touching the extension moves the extension AND web, since
# web bakes in the XPI. If web does not move, its set is too narrow
# — the direction that serves stale bytes on a pin.
# * dev and main derive the same values for the same source.
- name: Report the derived artifact version
run: |
set -u
A=web
T=$(sh scripts/artifacts.sh tag "$A" 2>&1 || echo UNAVAILABLE)
V=$(sh scripts/artifacts.sh version "$A" 2>&1 || echo UNAVAILABLE)
R=$(sh scripts/artifacts.sh revision "$A" 2>&1 || echo UNAVAILABLE)
echo "derived: artifact=$A tag=$T version=$V revision=$R sha=$GITHUB_SHA"
- name: Determine tag
id: tag
run: |
# Three trigger shapes:
# refs/tags/v… → tag-push: opt-in milestone label (vYY.MM.DD,
# plus `.N` when the day already carries a tag —
# family rule 148, amended 2026-08-24 after a
# same-day tag was retargeted and a release
# deleted to make room, note 2813).
# Publish ONLY the immutable version tag;
# don't touch :latest (the main-push build
# for the merge commit already did that).
# refs/heads/main → push to main: publish :main + :latest
# (floating) AND :c-<short_sha> (immutable
# per-commit rollback substrate, per family
# release-posture rule "Tags are milestones,
# not gates — commit-SHA images are the
# rollback unit"). Rollback to any commit
# becomes `docker pull …:c-<sha>` without a
# release ceremony.
# refs/heads/dev → push to dev: publish :dev, the rolling test
# channel (family rule 146). Rolling means it may
# carry newer contents than the :c-<sha> of the
# same commit; it never writes :c-<sha> itself,
# because that is the rollback unit (rule 145).
# POSIX-safe substring (the runner shell is dash/BusyBox sh, not
# bash — `${var:0:7}` errors with "Bad substitution"; cut works
# everywhere). Operator-flagged 2026-06-01 after first :c-<sha>
# main-push build failed at this step.
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
# The pinnable tag (milestone 313 step 3): YYYY.M.D of the commit
# THIS artifact's shipped files last changed in. Day precision is
# deliberate — same-day work is not something worth pinning, so a
# second main build the same day replaces the first rather than
# accumulating a tag nobody would roll back to.
#
# Derived per artifact, so an image whose files did not change keeps
# the tag it already had: the agent reads 2026.7.17 today while web
# reads 2026.8.27 — and the reuse step below turns that into a
# skipped build rather than a rebuild of bytes that already exist.
# `channel` is baked into the image as FC_CHANNEL and reported by
# /api/extension/manifest (milestone 271 step 7). A tag-push counts as
# `main`: a vYY.MM.DD tag is cut from main, so that image is a
# main-channel artifact wearing an immutable name.
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:${TAG_NAME}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
CALVER=$(sh scripts/artifacts.sh tag web)
# Guarded, and computed only on this path. There is no `set -e` in
# this step, so a failed derivation would otherwise leave CALVER
# empty and publish the tag `fabledcurator:` — an invalid
# name, from a green step. An empty pin must never reach the
# registry.
if [ -z "$CALVER" ]; then
echo "ERROR: could not derive a web version tag" >&2
exit 1
fi
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:main,git.fabledsword.com/bvandeusen/fabledcurator:latest,git.fabledsword.com/bvandeusen/fabledcurator:c-${SHORT_SHA},git.fabledsword.com/bvandeusen/fabledcurator:${CALVER}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:dev" >> "$GITHUB_OUTPUT"
echo "channel=dev" >> "$GITHUB_OUTPUT"
fi
# A shell step, not docker/login-action@v3, because the action's shared
# cache races itself (#3118). act_runner caches a remote action under one
# /root/.cache/act/<hash> per runner, and build-web, build-ml and
# build-agent all start in the same second and all want this same action.
# One job re-clones the directory — which empties and repopulates it —
# while another is walking it to copy into its container, and the walker
# lstat()s a file that has just vanished. It failed twice on 2026-08-27,
# naming a DIFFERENT missing file each time (`eslint.config.mjs`, then
# `jest.config.ts`), which is what rules out a corrupt cache and points at
# a race. The loser dies with MODULE_NOT_FOUND on dist/index.js before the
# action runs at all, so the secret is never even reached.
#
# Nothing is lost by dropping it: logging in is one command, the docker
# CLI is already in the CI image (ci-requirements.md), and the same
# reasoning as family rule 5 applies — a marketplace action buys nothing
# when the tool is baked into the image the workflow already selected.
#
# Password on stdin, never as an argument: an argument lands in the
# process table and draws docker's own deprecation warning.
- name: Login to Forgejo registry
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
ACTOR: ${{ github.actor }}
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
# --- reuse-if-published (milestone 313, step 4) ----------------------
# The identity tag names this artifact's CONTENT — r-<revision>, the
# commit its shipped files last changed in, plus the channel for images
# that bake one in. If the registry already carries it, the bytes this
# job would produce are already published and the build is pure waste:
# the channel and date tags get repointed at the existing manifest
# instead, registry-side, in seconds.
#
# This is what stops a push that touched only `agent/` from rebuilding
# web and ml, and a merge to main from rebuilding what dev already built.
#
# The failure direction is deliberate. An inspect that errors for ANY
# reason — network, auth, a registry hiccup — reads as a miss and the
# build runs. Only a genuine 200 skips one, so there is no path here
# that skips a build that was actually needed; the worst case is paying
# for a build we could have avoided.
#
# BASE-IMAGE FRESHNESS, decided rather than left implicit: an artifact
# whose source stops moving stops picking up base-image updates under
# its pinned tag. That is what a pin MEANS — a date tag has to keep
# serving the bytes it served (fabledcurator:2026.7.17 still
# resolves to July's image), or it is not a pin — and family rule
# 145 already says where the refresh goes instead: a rebuild with
# different contents publishes only the MOVING tag, never the immutable
# one. A scheduled channel-only refresh is tracked separately (#3154);
# it does not belong in the push path.
- name: Is this content already published?
id: reuse
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
CHANNEL: ${{ steps.tag.outputs.channel }}
TAGS: ${{ steps.tag.outputs.tags }}
IS_TAG_PUSH: ${{ startsWith(github.ref, 'refs/tags/') }}
run: |
set -eu
ID=$(sh scripts/artifacts.sh identity web "$CHANNEL")
echo "identity=$ID" >> "$GITHUB_OUTPUT"
# A tag-push builds a revision that main already published, so it
# must NOT claim the identity: image configs are not bit-reproducible
# (embedded timestamps), so re-pushing r-<rev> would point an
# immutable tag at fresh bytes — rule 145's exact prohibition. It
# publishes only its own :v… label and otherwise reuses.
if [ "$IS_TAG_PUSH" = "true" ]; then
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
else
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
fi
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
echo "hit=true" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is already published — skipping the build"
else
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is not published — building"
fi
- name: Download signed XPI from Forgejo release asset
# Fires on every trigger shape. dev and main each bundle the XPI their
# own sign-extension just published — that is the whole point of the
@@ -339,7 +537,10 @@ jobs:
# for up to 10min total) before giving up. Main-push's signing
# eventually wins and tag-push picks the release up on a later
# iteration.
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/')
# Gated on the reuse miss as well: if the image is already published it
# already contains its XPI, so this would download (and on a tag-push,
# poll up to 10 minutes for) a file nothing then reads.
if: steps.reuse.outputs.hit != 'true' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || startsWith(github.ref, 'refs/tags/'))
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
run: |
@@ -399,56 +600,46 @@ jobs:
cp "$DEST" "frontend/public/extension/fabledcurator-latest.xpi"
ls -la frontend/public/extension/
- name: Determine tag
id: tag
run: |
# Three trigger shapes:
# refs/tags/v… → tag-push: opt-in milestone label (vYY.MM.DD,
# no `.N` per family release-posture rule).
# Publish ONLY the immutable version tag;
# don't touch :latest (the main-push build
# for the merge commit already did that).
# refs/heads/main → push to main: publish :main + :latest
# (floating) AND :c-<short_sha> (immutable
# per-commit rollback substrate, per family
# release-posture rule "Tags are milestones,
# not gates — commit-SHA images are the
# rollback unit"). Rollback to any commit
# becomes `docker pull …:c-<sha>` without a
# release ceremony.
# refs/heads/dev → push to dev: publish :dev, the rolling test
# channel (family rule 146). Rolling means it may
# carry newer contents than the :c-<sha> of the
# same commit; it never writes :c-<sha> itself,
# because that is the rollback unit (rule 145).
# POSIX-safe substring (the runner shell is dash/BusyBox sh, not
# bash — `${var:0:7}` errors with "Bad substitution"; cut works
# everywhere). Operator-flagged 2026-06-01 after first :c-<sha>
# main-push build failed at this step.
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:${TAG_NAME}" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:main,git.fabledsword.com/bvandeusen/fabledcurator:latest,git.fabledsword.com/bvandeusen/fabledcurator:c-${SHORT_SHA}" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator:dev" >> "$GITHUB_OUTPUT"
fi
- name: Login to Forgejo registry
uses: docker/login-action@v3
with:
registry: git.fabledsword.com
username: ${{ github.actor }}
password: ${{ secrets.RELEASE_TOKEN }}
- name: Build and push web image
if: steps.reuse.outputs.hit != 'true'
uses: docker/build-push-action@v5
with:
context: .
file: Dockerfile
push: true
tags: ${{ steps.tag.outputs.tags }}
tags: ${{ steps.reuse.outputs.build_tags }}
# Only the web image carries a channel: it is the one that serves
# /api/extension/manifest. The ml and agent images have nothing to
# report it to.
build-args: |
FC_CHANNEL=${{ steps.tag.outputs.channel }}
# Registry-side manifest copy: no layer transfer, no local daemon, no
# rebuild. Each -t becomes another reference to the SAME manifest the
# identity tag holds, so :latest and the date pin are byte-identical to
# what was published rather than a lookalike rebuild.
#
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
# rolling channel refreshes itself, so skipping a build must never mean
# leaving :dev or :latest pointing at something older than the commit
# that was just pushed.
- name: Repoint the tags at the published image (reuse)
if: steps.reuse.outputs.hit == 'true'
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator
IDENTITY: ${{ steps.reuse.outputs.identity }}
TAGS: ${{ steps.tag.outputs.tags }}
run: |
set -euf
# steps.tag emits ONE comma-separated list, because that is the shape
# docker/build-push-action takes; imagetools wants a -t per ref.
ARGS=""
IFS=,
for t in $TAGS; do ARGS="$ARGS -t $t"; done
unset IFS
# shellcheck disable=SC2086
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
build-ml:
runs-on: python-ci
@@ -456,6 +647,42 @@ jobs:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps:
- uses: actions/checkout@v4
with:
# Full history: this job derives its artifact's version from the
# commit its shipped files last changed in (milestone 313). A
# depth-1 clone cannot see that commit — it either derives a wrong,
# too-low value or finds nothing at all, and neither is a failure
# the build would otherwise notice.
fetch-depth: 0
# --- derived values, one line (milestone 313) ------------------------
# These stopped being shadow output at step 3: `tag` is published on
# main and `revision` decides whether the build below runs at all. This
# step prints all three anyway, because the load-bearing steps each
# print only the one they use, and on dev the date tag is computed
# nowhere else. When a build is skipped or a pin looks wrong, this is
# the line that says what the commit derived.
#
# Still diagnostic, so it still must not fail the build — no `set -e`,
# and every derivation falls back to UNAVAILABLE. A broken echo must
# never be the reason an image does not ship.
#
# What it should say:
# * a push touching only agent/ moves the agent and leaves web and ml
# STILL. If web moves, its path set is too wide.
# * a push touching only docs moves nothing.
# * a push touching the extension moves the extension AND web, since
# web bakes in the XPI. If web does not move, its set is too narrow
# — the direction that serves stale bytes on a pin.
# * dev and main derive the same values for the same source.
- name: Report the derived artifact version
run: |
set -u
A=ml
T=$(sh scripts/artifacts.sh tag "$A" 2>&1 || echo UNAVAILABLE)
V=$(sh scripts/artifacts.sh version "$A" 2>&1 || echo UNAVAILABLE)
R=$(sh scripts/artifacts.sh revision "$A" 2>&1 || echo UNAVAILABLE)
echo "derived: artifact=$A tag=$T version=$V revision=$R sha=$GITHUB_SHA"
- name: Determine tag
id: tag
@@ -469,29 +696,138 @@ jobs:
# everywhere). Operator-flagged 2026-06-01 after first :c-<sha>
# main-push build failed at this step.
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
# The pinnable tag (milestone 313 step 3): YYYY.M.D of the commit
# THIS artifact's shipped files last changed in. Day precision is
# deliberate — same-day work is not something worth pinning, so a
# second main build the same day replaces the first rather than
# accumulating a tag nobody would roll back to.
#
# Derived per artifact, so an image whose files did not change keeps
# the tag it already had: the agent reads 2026.7.17 today while web
# reads 2026.8.27 — and the reuse step below turns that into a
# skipped build rather than a rebuild of bytes that already exist.
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:${TAG_NAME}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:main,git.fabledsword.com/bvandeusen/fabledcurator-ml:latest,git.fabledsword.com/bvandeusen/fabledcurator-ml:c-${SHORT_SHA}" >> "$GITHUB_OUTPUT"
CALVER=$(sh scripts/artifacts.sh tag ml)
# Guarded, and computed only on this path. There is no `set -e` in
# this step, so a failed derivation would otherwise leave CALVER
# empty and publish the tag `fabledcurator-ml:` — an invalid
# name, from a green step. An empty pin must never reach the
# registry.
if [ -z "$CALVER" ]; then
echo "ERROR: could not derive a ml version tag" >&2
exit 1
fi
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:main,git.fabledsword.com/bvandeusen/fabledcurator-ml:latest,git.fabledsword.com/bvandeusen/fabledcurator-ml:c-${SHORT_SHA},git.fabledsword.com/bvandeusen/fabledcurator-ml:${CALVER}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-ml:dev" >> "$GITHUB_OUTPUT"
echo "channel=dev" >> "$GITHUB_OUTPUT"
fi
# Shell step rather than docker/login-action — see build-web's note on
# the shared action-cache race (#3118).
- name: Login to Forgejo registry
uses: docker/login-action@v3
with:
registry: git.fabledsword.com
username: ${{ github.actor }}
password: ${{ secrets.RELEASE_TOKEN }}
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
ACTOR: ${{ github.actor }}
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
# --- reuse-if-published (milestone 313, step 4) ----------------------
# The identity tag names this artifact's CONTENT — r-<revision>, the
# commit its shipped files last changed in, plus the channel for images
# that bake one in. If the registry already carries it, the bytes this
# job would produce are already published and the build is pure waste:
# the channel and date tags get repointed at the existing manifest
# instead, registry-side, in seconds.
#
# This is what stops a push that touched only `agent/` from rebuilding
# web and ml, and a merge to main from rebuilding what dev already built.
#
# The failure direction is deliberate. An inspect that errors for ANY
# reason — network, auth, a registry hiccup — reads as a miss and the
# build runs. Only a genuine 200 skips one, so there is no path here
# that skips a build that was actually needed; the worst case is paying
# for a build we could have avoided.
#
# BASE-IMAGE FRESHNESS, decided rather than left implicit: an artifact
# whose source stops moving stops picking up base-image updates under
# its pinned tag. That is what a pin MEANS — a date tag has to keep
# serving the bytes it served (fabledcurator-ml:2026.7.17 still
# resolves to July's image), or it is not a pin — and family rule
# 145 already says where the refresh goes instead: a rebuild with
# different contents publishes only the MOVING tag, never the immutable
# one. A scheduled channel-only refresh is tracked separately (#3154);
# it does not belong in the push path.
- name: Is this content already published?
id: reuse
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
CHANNEL: ${{ steps.tag.outputs.channel }}
TAGS: ${{ steps.tag.outputs.tags }}
IS_TAG_PUSH: ${{ startsWith(github.ref, 'refs/tags/') }}
run: |
set -eu
ID=$(sh scripts/artifacts.sh identity ml "$CHANNEL")
echo "identity=$ID" >> "$GITHUB_OUTPUT"
# A tag-push builds a revision that main already published, so it
# must NOT claim the identity: image configs are not bit-reproducible
# (embedded timestamps), so re-pushing r-<rev> would point an
# immutable tag at fresh bytes — rule 145's exact prohibition. It
# publishes only its own :v… label and otherwise reuses.
if [ "$IS_TAG_PUSH" = "true" ]; then
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
else
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
fi
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
echo "hit=true" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is already published — skipping the build"
else
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is not published — building"
fi
- name: Build and push ml image
if: steps.reuse.outputs.hit != 'true'
uses: docker/build-push-action@v5
with:
context: .
file: Dockerfile.ml
push: true
tags: ${{ steps.tag.outputs.tags }}
tags: ${{ steps.reuse.outputs.build_tags }}
# Registry-side manifest copy: no layer transfer, no local daemon, no
# rebuild. Each -t becomes another reference to the SAME manifest the
# identity tag holds, so :latest and the date pin are byte-identical to
# what was published rather than a lookalike rebuild.
#
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
# rolling channel refreshes itself, so skipping a build must never mean
# leaving :dev or :latest pointing at something older than the commit
# that was just pushed.
- name: Repoint the tags at the published image (reuse)
if: steps.reuse.outputs.hit == 'true'
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-ml
IDENTITY: ${{ steps.reuse.outputs.identity }}
TAGS: ${{ steps.tag.outputs.tags }}
run: |
set -euf
# steps.tag emits ONE comma-separated list, because that is the shape
# docker/build-push-action takes; imagetools wants a -t per ref.
ARGS=""
IFS=,
for t in $TAGS; do ARGS="$ARGS -t $t"; done
unset IFS
# shellcheck disable=SC2086
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
# The desktop GPU agent (#114) — published so the operator pulls + runs it on
# the GPU machine instead of building locally. Independent of web/ml (its own
@@ -502,31 +838,176 @@ jobs:
image: git.fabledsword.com/bvandeusen/ci-python:3.14
steps:
- uses: actions/checkout@v4
with:
# Full history: this job derives its artifact's version from the
# commit its shipped files last changed in (milestone 313). A
# depth-1 clone cannot see that commit — it either derives a wrong,
# too-low value or finds nothing at all, and neither is a failure
# the build would otherwise notice.
fetch-depth: 0
# --- derived values, one line (milestone 313) ------------------------
# These stopped being shadow output at step 3: `tag` is published on
# main and `revision` decides whether the build below runs at all. This
# step prints all three anyway, because the load-bearing steps each
# print only the one they use, and on dev the date tag is computed
# nowhere else. When a build is skipped or a pin looks wrong, this is
# the line that says what the commit derived.
#
# Still diagnostic, so it still must not fail the build — no `set -e`,
# and every derivation falls back to UNAVAILABLE. A broken echo must
# never be the reason an image does not ship.
#
# What it should say:
# * a push touching only agent/ moves the agent and leaves web and ml
# STILL. If web moves, its path set is too wide.
# * a push touching only docs moves nothing.
# * a push touching the extension moves the extension AND web, since
# web bakes in the XPI. If web does not move, its set is too narrow
# — the direction that serves stale bytes on a pin.
# * dev and main derive the same values for the same source.
- name: Report the derived artifact version
run: |
set -u
A=agent
T=$(sh scripts/artifacts.sh tag "$A" 2>&1 || echo UNAVAILABLE)
V=$(sh scripts/artifacts.sh version "$A" 2>&1 || echo UNAVAILABLE)
R=$(sh scripts/artifacts.sh revision "$A" 2>&1 || echo UNAVAILABLE)
echo "derived: artifact=$A tag=$T version=$V revision=$R sha=$GITHUB_SHA"
- name: Determine tag
id: tag
run: |
SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)
# The pinnable tag (milestone 313 step 3): YYYY.M.D of the commit
# THIS artifact's shipped files last changed in. Day precision is
# deliberate — same-day work is not something worth pinning, so a
# second main build the same day replaces the first rather than
# accumulating a tag nobody would roll back to.
#
# Derived per artifact, so an image whose files did not change keeps
# the tag it already had: the agent reads 2026.7.17 today while web
# reads 2026.8.27 — and the reuse step below turns that into a
# skipped build rather than a rebuild of bytes that already exist.
if [ "${GITHUB_REF#refs/tags/}" != "${GITHUB_REF}" ]; then
TAG_NAME="${GITHUB_REF#refs/tags/}"
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-agent:${TAG_NAME}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
elif [ "${GITHUB_REF##*/}" = "main" ]; then
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-agent:main,git.fabledsword.com/bvandeusen/fabledcurator-agent:latest,git.fabledsword.com/bvandeusen/fabledcurator-agent:c-${SHORT_SHA}" >> "$GITHUB_OUTPUT"
CALVER=$(sh scripts/artifacts.sh tag agent)
# Guarded, and computed only on this path. There is no `set -e` in
# this step, so a failed derivation would otherwise leave CALVER
# empty and publish the tag `fabledcurator-agent:` — an invalid
# name, from a green step. An empty pin must never reach the
# registry.
if [ -z "$CALVER" ]; then
echo "ERROR: could not derive a agent version tag" >&2
exit 1
fi
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-agent:main,git.fabledsword.com/bvandeusen/fabledcurator-agent:latest,git.fabledsword.com/bvandeusen/fabledcurator-agent:c-${SHORT_SHA},git.fabledsword.com/bvandeusen/fabledcurator-agent:${CALVER}" >> "$GITHUB_OUTPUT"
echo "channel=main" >> "$GITHUB_OUTPUT"
else
echo "tags=git.fabledsword.com/bvandeusen/fabledcurator-agent:dev" >> "$GITHUB_OUTPUT"
echo "channel=dev" >> "$GITHUB_OUTPUT"
fi
# Shell step rather than docker/login-action — see build-web's note on
# the shared action-cache race (#3118).
- name: Login to Forgejo registry
uses: docker/login-action@v3
with:
registry: git.fabledsword.com
username: ${{ github.actor }}
password: ${{ secrets.RELEASE_TOKEN }}
env:
TOKEN: ${{ secrets.RELEASE_TOKEN }}
ACTOR: ${{ github.actor }}
run: echo "$TOKEN" | docker login git.fabledsword.com -u "$ACTOR" --password-stdin
# --- reuse-if-published (milestone 313, step 4) ----------------------
# The identity tag names this artifact's CONTENT — r-<revision>, the
# commit its shipped files last changed in, plus the channel for images
# that bake one in. If the registry already carries it, the bytes this
# job would produce are already published and the build is pure waste:
# the channel and date tags get repointed at the existing manifest
# instead, registry-side, in seconds.
#
# This is what stops a push that touched only `agent/` from rebuilding
# web and ml, and a merge to main from rebuilding what dev already built.
#
# The failure direction is deliberate. An inspect that errors for ANY
# reason — network, auth, a registry hiccup — reads as a miss and the
# build runs. Only a genuine 200 skips one, so there is no path here
# that skips a build that was actually needed; the worst case is paying
# for a build we could have avoided.
#
# BASE-IMAGE FRESHNESS, decided rather than left implicit: an artifact
# whose source stops moving stops picking up base-image updates under
# its pinned tag. That is what a pin MEANS — a date tag has to keep
# serving the bytes it served (fabledcurator-agent:2026.7.17 still
# resolves to July's image), or it is not a pin — and family rule
# 145 already says where the refresh goes instead: a rebuild with
# different contents publishes only the MOVING tag, never the immutable
# one. A scheduled channel-only refresh is tracked separately (#3154);
# it does not belong in the push path.
- name: Is this content already published?
id: reuse
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
CHANNEL: ${{ steps.tag.outputs.channel }}
TAGS: ${{ steps.tag.outputs.tags }}
IS_TAG_PUSH: ${{ startsWith(github.ref, 'refs/tags/') }}
run: |
set -eu
ID=$(sh scripts/artifacts.sh identity agent "$CHANNEL")
echo "identity=$ID" >> "$GITHUB_OUTPUT"
# A tag-push builds a revision that main already published, so it
# must NOT claim the identity: image configs are not bit-reproducible
# (embedded timestamps), so re-pushing r-<rev> would point an
# immutable tag at fresh bytes — rule 145's exact prohibition. It
# publishes only its own :v… label and otherwise reuses.
if [ "$IS_TAG_PUSH" = "true" ]; then
echo "build_tags=$TAGS" >> "$GITHUB_OUTPUT"
else
echo "build_tags=$TAGS,$IMAGE:$ID" >> "$GITHUB_OUTPUT"
fi
if docker buildx imagetools inspect "$IMAGE:$ID" >/dev/null 2>&1; then
echo "hit=true" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is already published — skipping the build"
else
echo "hit=false" >> "$GITHUB_OUTPUT"
echo "reuse: $IMAGE:$ID is not published — building"
fi
- name: Build and push agent image
if: steps.reuse.outputs.hit != 'true'
uses: docker/build-push-action@v5
with:
context: agent
file: agent/Dockerfile
push: true
tags: ${{ steps.tag.outputs.tags }}
tags: ${{ steps.reuse.outputs.build_tags }}
# Registry-side manifest copy: no layer transfer, no local daemon, no
# rebuild. Each -t becomes another reference to the SAME manifest the
# identity tag holds, so :latest and the date pin are byte-identical to
# what was published rather than a lookalike rebuild.
#
# Runs on EVERY reuse, which is what keeps family rule 146 true: a
# rolling channel refreshes itself, so skipping a build must never mean
# leaving :dev or :latest pointing at something older than the commit
# that was just pushed.
- name: Repoint the tags at the published image (reuse)
if: steps.reuse.outputs.hit == 'true'
env:
IMAGE: git.fabledsword.com/bvandeusen/fabledcurator-agent
IDENTITY: ${{ steps.reuse.outputs.identity }}
TAGS: ${{ steps.tag.outputs.tags }}
run: |
set -euf
# steps.tag emits ONE comma-separated list, because that is the shape
# docker/build-push-action takes; imagetools wants a -t per ref.
ARGS=""
IFS=,
for t in $TAGS; do ARGS="$ARGS -t $t"; done
unset IFS
# shellcheck disable=SC2086
docker buildx imagetools create $ARGS "$IMAGE:$IDENTITY"
echo "repointed to $IMAGE:$IDENTITY: $TAGS"
+53 -95
View File
@@ -2,7 +2,7 @@ name: CI
# CI lanes per FabledRulebook/forgejo.md "CI philosophy":
# - lint: ruff only, no dep install — fast-fail for the common lint bounce.
# - extension-version: guards the extension publish path (see the job).
# - extension-version: the derived version resolves and MAJOR.MINOR agrees.
# - backend-lint-and-test: `pytest -m "not integration"`, no service containers.
# - frontend-build: vitest unit + vite build.
# - integration: pgvector + redis service containers; alembic + `pytest -m integration`.
@@ -42,18 +42,29 @@ jobs:
# catching syntax errors before the image build.
run: python -m compileall -q agent/fc_agent
# Guards the extension publish path, which has no self-correcting behavior.
# The extension version is DERIVED, not hand-maintained (milestone 271 step
# 4): build.yml computes it from the commit TIME of the newest packaged
# extension change and stamps it into manifest.json / package.json at build
# time. The guard that used to live here — "packaged files changed but nobody
# bumped the version" — was therefore checking a fact that had stopped
# existing. Worse than useless: it would have failed this lane on every real
# extension change, demanding a bump that decides nothing. Retired 2026-08-27
# rather than left running beside the new mechanism (rule 22).
#
# build.yml's sign-extension job keys its AMO-signing cache purely on the
# version string in extension/package.json: if an `ext-<version>` Forgejo
# release already carries an XPI, signing is SKIPPED and that old signed XPI
# is what build-web bakes into `:latest`. Nothing in that path inspects
# whether extension/ actually changed — so a forgotten version bump ships a
# stale extension on a fully green build, silently. (AMO can't help: it 409s
# on re-signing a version, which is exactly why the cache exists.)
# Two things are still worth asserting, and this is the only lane that can:
# the extension.yml suite runs on node:24-slim, which is exactly why
# version.spec.js sticks to packaging.sh's git-free subcommands.
# 1. the derivation actually resolves on this commit
# 2. MAJOR.MINOR agrees between the two files — the one part still hand-set,
# and packaging.sh reads it from manifest.json ALONE, so a divergence
# ships a version package.json disagrees with
#
# This job makes that case loud, on the dev push, instead of invisible at
# merge-to-main. It is pure git + text work — no deps, no services.
# Deliberately NOT checked here: that the derived value beats what has already
# been signed. That guard belongs in build.yml, where it compares against the
# real ext-* releases. Comparing against origin/main here would be wrong —
# dev legitimately derives a LOWER value whenever main is ahead on the
# extension, and a lane that fails for being behind is a lane people learn to
# ignore.
extension-version:
runs-on: python-ci
container:
@@ -61,97 +72,37 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
# Full history: the check diffs against the push's `before` SHA (or
# the PR base), which a depth-1 clone wouldn't contain.
# The derivation needs real history: a depth-1 clone sees one commit
# and produces a wrong, too-low value RATHER THAN FAILING. Checking
# that here is half the point of the lane.
fetch-depth: 0
- name: Extension version guard
env:
BEFORE: ${{ github.event.before }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
- name: Extension version derives cleanly
run: |
set -eu
# busybox sh on the act_runner — no bashisms (family rule).
ver() { grep -E '"version"' "$1" | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/'; }
PKG=$(ver extension/package.json)
MAN=$(ver extension/manifest.json)
test -n "$PKG" || { echo "ERROR: no version found in extension/package.json"; exit 1; }
test -n "$MAN" || { echo "ERROR: no version found in extension/manifest.json"; exit 1; }
# (1) Unconditional: the two version strings must agree. `web-ext sign`
# reads manifest.json (package.json sits in --ignore-files and isn't
# even inside the XPI), so AMO signs MAN and Firefox installs MAN.
# build.yml keys its cache, release tag, XPI filename — and therefore
# the version /api/extension/manifest reports to the update prompt —
# on PKG. Divergence either hard-fails at AMO or ships a mislabelled
# XPI whose update prompt lies about what's installed.
VERSION=$(sh extension/scripts/packaging.sh version)
echo "derived: $VERSION"
# The shape AMO accepts, and the shape build.yml will stamp.
if ! echo "$VERSION" | grep -qE '^[0-9]+(\.[0-9]+)*$'; then
echo "ERROR: derived version '$VERSION' is not plain dotted-numeric."
echo "AMO would reject it, and build.yml stamps it verbatim."
exit 1
fi
mm() { grep -E '"version"' "$1" | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([0-9]+\.[0-9]+).*/\1/'; }
MAN=$(mm extension/manifest.json)
PKG=$(mm extension/package.json)
test -n "$MAN" || { echo "ERROR: no parseable version in extension/manifest.json"; exit 1; }
test -n "$PKG" || { echo "ERROR: no parseable version in extension/package.json"; exit 1; }
if [ "$MAN" != "$PKG" ]; then
echo "ERROR: extension version mismatch."
echo " extension/manifest.json = $MAN <- what AMO signs / Firefox installs"
echo " extension/package.json = $PKG <- what CI caches, names, and reports"
echo "Set both to the same value."
echo "ERROR: MAJOR.MINOR disagrees between the two files."
echo " extension/manifest.json = $MAN <- packaging.sh reads MAJOR.MINOR from here"
echo " extension/package.json = $PKG"
echo "Only MAJOR.MINOR is hand-set. The patch component is derived from"
echo "commit time and overwritten at build time, so the committed patch"
echo "numbers are inert — but MAJOR.MINOR still ships. Set both the same."
exit 1
fi
# (2) If the SHIPPED extension changed, the version must have moved.
#
# Compare against MAIN, not against the previous push. The publish
# decision is made at merge-to-main against whatever ext-<version>
# already exists, so "differs from main" is the question that matters.
# Diffing against the previous dev push instead would demand a fresh
# bump on every iteration — push, tweak the extension again, and CI
# would insist on a second bump that buys nothing, inflating the
# version for no reason. On a main push there is no "main to compare
# to" yet, so fall back to that push's own before-SHA.
if [ "${GITHUB_REF##*/}" = "main" ]; then
BASE="${BEFORE:-}"
else
BASE=$(git rev-parse --verify -q origin/main 2>/dev/null || git rev-parse --verify -q main 2>/dev/null || echo "")
# PR base is the fallback when main isn't in the clone at all.
[ -n "$BASE" ] || BASE="${PR_BASE:-}"
fi
case "$BASE" in
''|0000000000000000000000000000000000000000)
echo "No usable base ref (no main in clone / first push) — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
;;
esac
if ! git cat-file -e "$BASE^{commit}" 2>/dev/null; then
echo "Base commit $BASE not in this clone — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
fi
# The exclusion list is NOT written out here — it comes from
# extension/scripts/packaging.sh, the one definition of what ships,
# shared with web-ext's --ignore-files and the derived-version patch
# count. Three hand-kept copies of that fact is how #2397 happened.
#
# `set -f` is required around the substitution: without it the shell
# globs `test/**` against the working tree and silently narrows it.
set -f
CHANGED=$(git diff --name-only "$BASE" HEAD -- extension/ $(sh extension/scripts/packaging.sh pathspec))
set +f
if [ -z "$CHANGED" ]; then
echo "No packaged extension files changed since $BASE — nothing to guard."
echo "OK: extension version $PKG"
exit 0
fi
echo "Packaged extension files changed since $BASE:"
echo "$CHANGED" | sed 's/^/ /'
PKG_OLD=$(git show "$BASE:extension/package.json" 2>/dev/null | grep -E '"version"' | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([^"]+)".*/\1/')
if [ -z "$PKG_OLD" ]; then
echo "Could not read the base version — skipping the bump check."
echo "OK: extension version $PKG"
exit 0
fi
if [ "$PKG_OLD" = "$PKG" ]; then
echo "ERROR: packaged extension files changed but the version is still $PKG."
echo "build.yml would find the existing ext-$PKG release, skip AMO signing,"
echo "and bake the OLD signed XPI into :latest — a green build shipping stale code."
echo "Bump the version in BOTH extension/package.json and extension/manifest.json."
exit 1
fi
echo "OK: extension version $PKG_OLD -> $PKG"
echo "OK: MAJOR.MINOR $MAN, derived version $VERSION"
backend-lint-and-test:
runs-on: python-ci
@@ -164,6 +115,13 @@ jobs:
SECRET_KEY: ci_unit_test_placeholder
steps:
- uses: actions/checkout@v4
with:
# Full history for tests/test_artifact_identity.py, which derives
# each artifact's revision to check the identity scheme. On a
# depth-1 clone that derivation either fails or returns the tip sha
# — so the lane would go green while asserting nothing, which is
# the one outcome worse than a red one.
fetch-depth: 0
# Cache step removed 2026-05-26: act_runner's cache backend has been
# broken on this homelab runner since 2026-05-15 (first as request-
+8 -3
View File
@@ -10,15 +10,20 @@ on:
paths:
- 'extension/**'
- '.forgejo/workflows/extension.yml'
# test/version.spec.js asserts ci.yml's extension-version guard never
# ignores a file web-ext actually packages, so a ci.yml-only edit can
# break this suite and must trigger it.
# test/version.spec.js asserts things ABOUT the other two workflows —
# that neither inlines the packaged-file set, and that build.yml derives
# the shipped version rather than reading it out of the repo. A
# workflow-only edit can therefore break this suite, so it has to trigger
# it. build.yml joined the list at milestone 271 step 5, when the spec
# started asserting against it.
- '.forgejo/workflows/ci.yml'
- '.forgejo/workflows/build.yml'
pull_request:
branches: [main]
paths:
- 'extension/**'
- '.forgejo/workflows/ci.yml'
- '.forgejo/workflows/build.yml'
workflow_dispatch:
jobs:
+17
View File
@@ -47,6 +47,23 @@ RUN chmod +x entrypoint.sh
COPY --from=frontend-builder /build/dist ./frontend/dist
# Which channel this image belongs to — `dev` or `main` (milestone 271 step 7).
# build.yml passes it; /api/extension/manifest reports it beside the version so
# an operator can tell which channel an install came from without the channel
# ever touching the version string.
#
# Empty by default, deliberately: a locally-built image then reports NO channel
# rather than claiming to be one, and the manifest omits the field entirely —
# indistinguishable from an image built before the field existed, which is
# exactly the shape every reader already has to handle.
#
# Declared LAST on purpose. An ARG/ENV invalidates every layer below it, and
# this is the one value that differs between the dev and main builds of
# identical source — put it any earlier and the two channels could never share
# a cached pip install.
ARG FC_CHANNEL=""
ENV FC_CHANNEL=${FC_CHANNEL}
EXPOSE 8080
ENTRYPOINT ["./entrypoint.sh"]
+2 -2
View File
@@ -19,7 +19,7 @@ Five deployable pieces, built by `.forgejo/workflows/build.yml`:
| **Web / workers** | `Dockerfile` | `fabledcurator` | Quart API + the built Vue SPA in one image. `entrypoint.sh` picks the role: `web`, `worker`, `scheduler`. The `maintenance-long` service is a second `worker` pinned to the long-running maintenance queue. |
| **ML worker** | `Dockerfile.ml` | `fabledcurator-ml` | Same app, plus `requirements-ml.txt` — tagging and embedding models that run in-container. |
| **GPU agent** | `agent/Dockerfile` | `fabledcurator-agent` | Optional desktop-GPU worker (`agent/`). Leases jobs over **HTTP only** — never touches the database or Redis. Run it for a burst, stop it to reclaim the card. See `agent/README.md`. |
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on `main` only, then bundled into the web image and served from Settings → Maintenance. See `extension/README.md`. |
| **Firefox extension** | `extension/` | signed XPI | MV3 extension: pushes platform session cookies into FC and adds a creator as a Source in one click. AMO-signed on both `dev` and `main` (one signature per extension change, shared by the two channels), bundled into that channel's web image and served from Settings → Maintenance. See `extension/README.md`. |
| **Data** | — | `pgvector/pgvector:pg16`, `redis:7-alpine` | Postgres with pgvector for embeddings; Redis as the Celery broker. |
## Quick start
@@ -52,7 +52,7 @@ FabledCurator is designed to run inside a self-hosted homelab environment over p
## CI / Forgejo setup
Three workflows: `ci.yml` (lint, extension-version guard, backend unit tests,
Three workflows: `ci.yml` (lint, extension-version check, backend unit tests,
frontend build, integration), `extension.yml` (extension lint, vitest, XPI
content verification), and `build.yml` (sign + publish).
+25 -1
View File
@@ -7,6 +7,7 @@ from __future__ import annotations
import asyncio
import hashlib
import hmac
import os
import re
from pathlib import Path
@@ -31,6 +32,12 @@ XPI_DIR = Path("/app/frontend/dist/extension")
_XPI_VERSION_RE = re.compile(r"fabledcurator-(?P<version>[\w.-]+)\.xpi$")
# Which channel this image belongs to — "dev" or "main" — baked in at build
# time from the FC_CHANNEL build arg (milestone 271 step 7). Empty for a local
# build, or for any image predating the field. Tests override by monkeypatching
# this constant, same as XPI_DIR above.
FC_CHANNEL = os.environ.get("FC_CHANNEL", "").strip()
async def _ext_key_required(session) -> bool:
"""Unlike /api/credentials (which accepts the browser path with no
@@ -133,13 +140,30 @@ def _read_manifest_sync() -> dict | None:
return None
versioned.sort(key=lambda p: p.stat().st_mtime)
latest = versioned[-1]
return {
info = {
"installed": True,
"version": _extract_version(latest.name),
"xpi_url": f"/extension/{latest.name}",
"latest_url": "/extension/fabledcurator-latest.xpi",
"sha256": _sha256(latest),
}
# The channel goes BESIDE the version, never inside it. A `-dev` suffix is
# what silently disabled the dev channel in the sibling project this design
# comes from: the comparator returned nothing for a non-integer segment, so
# every dev version compared equal and "no update available" became
# indistinguishable from "I cannot read this version".
#
# Omitted rather than defaulted when unset. Absence already has a meaning
# every reader must handle — an image built before this field existed says
# exactly the same thing by not having the key — so a blank channel reuses
# that path instead of inventing a second "unknown" spelling.
#
# Reported verbatim, not validated against {"dev", "main"}: if an image
# declares something else, showing what it actually claims is more useful
# to whoever is debugging it than dropping the value on the floor.
if FC_CHANNEL:
info["channel"] = FC_CHANNEL
return info
@extension_bp.route("/manifest", methods=["GET"])
+28 -11
View File
@@ -54,17 +54,34 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
shims to production code — the libs ship as `background.scripts`, not ES
modules, so the specs exercise exactly the bytes packaged into the XPI.
- **`extension/scripts/packaging.sh` is the single definition of what ships
inside the XPI.** Three consumers read from it rather than keeping their own
copy: web-ext's `--ignore-files` (`extension/package.json`), the `:(exclude)`
pathspec in `ci.yml`'s `extension-version` guard, and the `git log` pathspec
that derives the extension version. Three hand-kept copies of that one fact
is what allowed issue #2397.
- Jobs that derive the extension version check out with `fetch-depth: 0`. The
version is the commit TIME of the newest packaged-extension change (minutes
since 2020-01-01, per family rule 149 — never a commit count, which orders
by branch rather than by recency). A depth-1 clone sees one commit and
derives a wrong, too-low value rather than failing, so the full-history
checkout is load-bearing wherever `packaging.sh version` is called.
inside the XPI.** Two consumers read from it rather than keeping their own
copy: web-ext's `--ignore-files` (`extension/package.json`), and the `git log`
pathspec inside the script's own version derivation. It was three until
2026-08-27 — `ci.yml`'s `extension-version` guard held the third and went when
the manual bump it guarded did (milestone 271 step 5). Hand-kept copies of
that one fact is what allowed issue #2397, so `extension/test/version.spec.js`
asserts no workflow has reintroduced a literal `:(exclude)extension/…`.
- **The shipped extension version is derived, not committed.** It is the commit
TIME of the newest packaged-extension change (minutes since 2020-01-01, per
family rule 149 — never a commit count, which orders by branch rather than by
recency). `build.yml`'s `sign-extension` computes it and stamps it into
`extension/manifest.json` + `package.json` in the working tree before signing;
the stamp is never committed. Treat the version in the repo as a base: only
its MAJOR.MINOR is read, and its patch component is inert.
- Every job that calls `packaging.sh version` checks out with `fetch-depth: 0`
`build.yml`'s `sign-extension` and `build-web`, and `ci.yml`'s
`extension-version`. A depth-1 clone sees one commit and derives a wrong,
too-low value **rather than failing**, so the full-history checkout is
load-bearing rather than incidental.
- **`FC_CHANNEL` is a build arg, not a runtime setting.** `build.yml` passes
`dev` / `main` to the web image only (the ml and agent images have nothing to
report it to), and `/api/extension/manifest` reports it beside the version so
an install can be traced to a channel. It is declared LAST in the Dockerfile
on purpose: an ARG invalidates every layer below it, and this is the one value
that differs between the dev and main builds of identical source, so placing
it earlier would stop the two channels ever sharing a cached `pip install`.
Empty by default — a local build then reports no channel at all rather than
claiming one.
- Callers MUST `set -f` before substituting the script's output. Without it the
shell expands `test/**` against the working tree and silently narrows the
pattern to whatever files exist at that moment — a failure that looks like
+59 -6
View File
@@ -7,7 +7,8 @@ page in one click.
## Install (operator)
The signed XPI is bundled into the FC Docker image. Open FC →
The signed XPI is bundled into the FC Docker image`:dev` and
`:latest` each carry their own channel's build. Open FC →
Settings → Maintenance → Browser extension → click "Install Firefox
extension". Firefox shows its native install prompt. After installing,
open the extension's options page (about:addons → FabledCurator →
@@ -20,6 +21,7 @@ same card.
cd extension/
npm install --no-save # web-ext only
npm run lint # web-ext lint
npm run test:unit # vitest — lib/ logic + packaging/version checks
npm run start # launches Firefox with extension loaded
npm run build # unsigned XPI in web-ext-artifacts/
```
@@ -36,10 +38,61 @@ npm run build # unsigned XPI in web-ext-artifacts/
- [ ] Subscriptions list: popup → "Sources" tab → list renders
- [ ] Check now: click play icon on source row → no error toast
## Versioning — don't hand-edit the patch number
The shipped version is **derived**, not committed. `scripts/packaging.sh
version` returns `MAJOR.MINOR` from `manifest.json` plus a patch component
that is the commit *time* of the newest change to a packaged extension file,
in minutes since 2020-01-01. `build.yml` computes it and stamps it into both
`manifest.json` and `package.json` at build time. The stamp is never
committed — the commit carrying it would itself be a change to the extension,
which would move the version again.
So:
- **Editing the patch number does nothing.** It is overwritten before web-ext
ever reads it. There is no bump to make, and none to forget.
- **MAJOR.MINOR is still yours.** It carries the deliberate meaning, it is read
from `manifest.json` alone, and CI fails the `extension-version` lane if the
two files disagree on it.
- `npm run build` locally produces an XPI labelled with the *committed*
version, since nothing stamped it. Fine for loading into a test profile; not
what ships.
Why commit time and not a commit count: a count is per-branch, so `dev` and
`main` count different histories of the same code and their versions end up
ordered by which branch accumulated more commits rather than by which is newer.
Commit time gives both branches the same number for the same source — which is
exactly what lets one AMO signature serve both channels (family rule 149, FC
issue #3092).
## Channels
`dev` and `main` each build and sign their own extension, and an install is
tied to whichever FC instance it points at — Firefox's static `update_url`
cannot apply here, since every FC install is a different host, so the extension
asks its configured backend. **The channel therefore IS the instance.**
Switching channel means repointing the FC URL in options and reinstalling from
that host; there is no separate channel setting, and adding one would
contradict each server build shipping its own extension.
The channel is reported *beside* the version, never inside it:
`/api/extension/manifest` answers `{"version": "...", "channel": "dev"}`. It is
optional — an instance that declares none simply omits the key, and the popup,
the toolbar tooltip and the Settings card all read exactly as they did before
the field existed. Do not be tempted to make it a `-dev` version suffix: the
comparator parses each dotted segment with `parseInt`, so a suffixed segment
reads as 0 and every dev build compares equal to every other, collapsing "no
update available" and "I cannot read this version" into one answer.
## Release
Bump `manifest.json` + `package.json` SemVer (both files) and commit
under `extension/**`. The `.forgejo/workflows/extension.yml` workflow
runs `web-ext sign` on main, commits the signed XPI to
`frontend/public/extension/`, and the next FC server build bundles it
into the Docker image.
Nothing to do by hand. Push to `dev`: `build.yml` signs the extension if this
change moved the version, caches the signed XPI as a Forgejo `ext-<version>`
release, and bundles it into `fabledcurator:dev`. Merging to `main` derives the
same version, hits that cache, and bundles the byte-identical XPI into
`:latest` with no second AMO call.
AMO refuses to re-sign a version it has already issued, so signing is one-shot
per version — which is why the cache exists and why the version must never move
backwards.
+25 -2
View File
@@ -37,7 +37,16 @@ ensureInitialized().catch(e => console.error('init failed:', e));
// configured backend for the latest published version and nudge the operator to
// reinstall the freshly-signed XPI — surfaced as a popup banner (on demand) and
// a toolbar badge (daily). /api/extension/manifest is public and returns
// {version, latest_url, sha256}; the XPI is served from the web root (not /api).
// {version, latest_url, sha256} plus an OPTIONAL {channel} naming which channel
// that instance serves ("dev"/"main", #3113); the XPI is served from the web
// root (not /api).
//
// The channel IS the instance: Firefox's static update_url cannot apply here
// because every FC install is a different host, so the extension asks its
// configured backend — which means switching channel is repointing apiUrl in
// options and reinstalling from that host. There is no separate channel
// setting to build, and building one would contradict each server build
// shipping its own extension.
function versionIsNewer(candidate, current) {
// Dotted numeric compare so 1.0.10 > 1.0.9 (a plain string compare wouldn't).
@@ -60,12 +69,22 @@ async function checkForUpdateInfo() {
}
const currentVersion = browser.runtime.getManifest().version;
const latestVersion = info && info.version ? info.version : null;
// Which channel the configured instance serves — reported ALONGSIDE the
// version, never folded into it. A `-dev` suffix would have to survive
// versionIsNewer's parseInt above, and it wouldn't: the segment would read
// as 0 and every dev build would compare equal to every other.
//
// null is a normal answer, not a failure — an instance built before the
// field existed, or one built locally with no channel declared. Nothing
// below branches on it except the label.
const channel = info && info.channel ? info.channel : null;
// latest_url is served from the web root, not the JSON API.
const base = api.webRoot();
return {
updateAvailable: !!latestVersion && versionIsNewer(latestVersion, currentVersion),
currentVersion,
latestVersion,
channel,
xpiUrl: info && info.latest_url ? `${base}${info.latest_url}` : null,
};
}
@@ -77,7 +96,11 @@ async function refreshUpdateBadge() {
await browser.action.setBadgeText({ text: r.updateAvailable ? '↑' : '' });
if (r.updateAvailable) {
await browser.action.setBadgeBackgroundColor({ color: '#F4BA7A' });
await browser.action.setTitle({ title: `FabledCurator — update available (v${r.latestVersion})` });
// Channel first, version second, and the channel dropped entirely when
// the instance doesn't report one — so the tooltip reads exactly as it
// did before the field existed rather than saying "(unknown ...)".
const label = r.channel ? `${r.channel} v${r.latestVersion}` : `v${r.latestVersion}`;
await browser.action.setTitle({ title: `FabledCurator — update available (${label})` });
} else {
await browser.action.setTitle({ title: 'FabledCurator' });
}
+5 -1
View File
@@ -81,8 +81,12 @@ async function checkForUpdate() {
}
function showUpdateBanner(r) {
// The channel names itself beside the version, never inside it (#3113).
// Absent when the instance doesn't report one, and the banner then reads
// exactly as it did before the field existed.
const channel = r.channel ? ` (${r.channel})` : '';
document.getElementById('update-text').textContent =
`Update available — v${r.latestVersion} (installed v${r.currentVersion})`;
`Update available${channel} — v${r.latestVersion} (installed v${r.currentVersion})`;
// Opening the signed XPI triggers Firefox's native install prompt.
document.getElementById('update-btn').addEventListener('click', () => {
browser.tabs.create({ url: r.xpiUrl });
+31 -4
View File
@@ -6,7 +6,9 @@
# keeping three copies of one fact in sync by hand is how issue #2397 happened:
#
# 1. web-ext's --ignore-files (extension/package.json's four scripts)
# 2. the :(exclude) pathspec (ci.yml's extension-version guard)
# 2. the :(exclude) pathspec (what moves the version — a WIDER
# set than the ignore list; see
# NOT_VERSION_RELEVANT)
# 3. the git-log pathspec (the derived version, below)
#
# They now all read from here. POSIX sh only — CI's run shell is busybox.
@@ -35,6 +37,28 @@ set -euf
NOT_PACKAGED_TRACKED='package.json package-lock.json README.md .gitignore vitest.config.js scripts scripts/** test test/**'
NOT_PACKAGED_BUILD='web-ext-artifacts node_modules'
# Paths under extension/ that cannot change the SHIPPED BYTES, and so must not
# move the derived version.
#
# Deliberately NOT the same list as NOT_PACKAGED_TRACKED, and the whole
# difference is `scripts/`. packaging.sh is not packaged into the XPI — but it
# DECIDES the version string, and build.yml stamps that string into the
# manifest.json that is packaged. A change to how the version is computed is
# therefore a change to the shipped bytes.
#
# Excluding it was harmless only while every push rebuilt the web image.
# Milestone 313 step 4 made the rebuild conditional on the derived revision
# moving, which turned it into a silent failure: a packaging.sh change gives a
# NEW version, so sign-extension misses its ext-<version> cache and signs —
# while build-web sees an unmoved revision, reuses the published image, and
# ships the OLD XPI. An orphaned AMO signature, and an instance quietly serving
# code the registry says is current.
#
# The two directions are not symmetric, which is why this list is the narrower
# one. Too wide costs a re-sign and a rebuild for a change that ships nothing
# new. Too narrow serves stale bytes and says nothing.
NOT_VERSION_RELEVANT='package.json package-lock.json README.md .gitignore vitest.config.js test test/**'
usage() {
echo "usage: packaging.sh {ignore|pathspec|version|major-minor|patch}" >&2
exit 2
@@ -49,11 +73,14 @@ cmd_ignore() {
echo "$NOT_PACKAGED_TRACKED $NOT_PACKAGED_BUILD"
}
# git pathspec excluding the non-packaged tracked files, e.g.
# :(exclude)extension/package.json :(exclude)extension/test/**
# git pathspec excluding the tracked files that cannot change the shipped
# bytes, e.g. :(exclude)extension/package.json :(exclude)extension/test/**
#
# This answers "what moves the version?", NOT "what goes in the XPI?" — see
# NOT_VERSION_RELEVANT for why those differ. cmd_ignore answers the other one.
# Same `set -f` requirement as above.
cmd_pathspec() {
for entry in $NOT_PACKAGED_TRACKED; do
for entry in $NOT_VERSION_RELEVANT; do
printf ':(exclude)extension/%s ' "$entry"
done
echo
+74 -9
View File
@@ -44,9 +44,39 @@ describe('packaging.sh — the single definition of what ships', () => {
// covering anything added later.
const pathspec = packaging('pathspec')
expect(pathspec).toContain(':(exclude)extension/test/**')
expect(pathspec).toContain(':(exclude)extension/scripts/**')
expect(pathspec.some((e) => e.includes('.spec.js'))).toBe(false)
expect(pathspec.some((e) => e.includes('helpers'))).toBe(false)
const ignore = packaging('ignore')
expect(ignore).toContain('test/**')
expect(ignore).toContain('scripts/**')
expect(ignore.some((e) => e.includes('.spec.js'))).toBe(false)
})
it('lets packaging.sh move the version, though it never ships in the XPI', () => {
// The two lists answer different questions and this is the one place they
// disagree. scripts/ is ignored by web-ext — it is repo tooling, not addon
// code — but packaging.sh DECIDES the version string, and build.yml stamps
// that string into the manifest.json that does ship. So changing how the
// version is computed changes the shipped bytes.
//
// Excluding it from the pathspec was invisible while every push rebuilt the
// web image. Milestone 313 step 4 made that rebuild conditional on the
// derived revision moving, and the omission turned into a silent failure:
// a new version means sign-extension misses its ext-<version> cache and
// signs, while build-web sees an unmoved revision, reuses the published
// image and ships the OLD XPI. An orphaned signature, and an instance
// serving code the registry calls current.
const pathspec = packaging('pathspec')
expect(
pathspec.some((e) => e.startsWith(':(exclude)extension/scripts')),
'the pathspec excludes scripts/, so a change to how the version is '
+ 'derived would not move the version it derives',
).toBe(false)
// ...and it is still kept out of the package itself. Both must hold: the
// tempting "fix" for either half is to make the two lists one again.
expect(packaging('ignore')).toContain('scripts')
})
it('keeps its own scripts and specs out of the XPI', () => {
@@ -77,21 +107,56 @@ describe('consumers delegate rather than keeping their own copy', () => {
}
})
it('ci.yml derives its pathspec from the script and hardcodes none', () => {
const ci = readText('..', '.forgejo', 'workflows', 'ci.yml')
expect(ci).toContain('extension/scripts/packaging.sh pathspec')
// A literal :(exclude)extension/... in the workflow means someone bypassed
// the shared definition.
expect(ci).not.toMatch(/:\(exclude\)extension\//)
const WORKFLOWS = ['ci.yml', 'build.yml', 'extension.yml']
it('no workflow hardcodes the packaged-file set', () => {
// ci.yml used to substitute `packaging.sh pathspec` directly, for the
// manual-bump guard that milestone 271 step 5 retired. Nothing inlines the
// set today, and nothing should start to: a literal :(exclude)extension/...
// in a workflow means someone bypassed the shared definition, which is
// exactly the drift #2397 was about. Asserted across all three rather than
// against one named consumer, so it keeps holding as consumers come and go.
for (const wf of WORKFLOWS) {
const text = readText('..', '.forgejo', 'workflows', wf)
expect(text, `${wf} inlines an :(exclude) literal`).not.toMatch(/:\(exclude\)extension\//)
}
})
it('build.yml takes the shipped version from the script, not from the repo', () => {
// The version is DERIVED from commit time (#3092, milestone 271 step 4).
// Going back to reading the committed value is not a style regression, it
// is the bug: a hand-set version makes dev and main sign the same number
// for different code, and the ext-<version> cache then serves one channel
// the other's XPI.
const build = readText('..', '.forgejo', 'workflows', 'build.yml')
expect(build).toContain('packaging.sh version')
expect(build, 'build.yml re-reads the committed version instead of deriving it')
.not.toMatch(/grep[^\n]*'"version"'[^\n]*package\.json/)
})
})
describe('extension version', () => {
it('keeps manifest.json and package.json in lockstep', () => {
expect(read('manifest.json').version).toBe(read('package.json').version)
const majorMinor = (v) => v.split('.').slice(0, 2).join('.')
it('keeps the hand-set MAJOR.MINOR in lockstep across both files', () => {
// Narrowed from full-string equality at milestone 271 step 5. Since step 4
// the patch component is derived from commit time and stamped into both
// files at build time, so the committed patch numbers are inert — nothing
// reads them and they are not what ships. Asserting on them would fail for
// a difference that changes nothing.
//
// MAJOR.MINOR is the opposite: still hand-set, still shipped, and
// packaging.sh reads it from manifest.json ALONE. Let the two diverge and
// the extension ships a version package.json disagrees with, with no other
// signal.
expect(majorMinor(read('manifest.json').version))
.toBe(majorMinor(read('package.json').version))
})
it('uses a plain dotted numeric version AMO will accept', () => {
// The committed value seeds MAJOR.MINOR, so it still has to parse even
// though its patch component never ships. ci.yml asserts the same shape on
// the DERIVED value, which is the one AMO actually sees.
expect(read('package.json').version).toMatch(/^\d+(\.\d+)*$/)
})
@@ -4,6 +4,16 @@
<span v-if="manifest?.installed" class="text-caption fc-muted">
· Firefox · v{{ manifest.version }}
</span>
<!-- Which channel this instance serves, so it is visible without
installing anything. Rendered only when the image declares one: a
locally-built image, or one predating the field, says nothing rather
than guessing. Never merged into the version string beside it see
the endpoint's note on why a `-dev` suffix breaks the comparator. -->
<v-chip
v-if="manifest?.channel"
size="x-small" variant="tonal" class="ml-2"
:color="manifest.channel === 'dev' ? 'warning' : 'info'"
>{{ manifest.channel }}</v-chip>
</CardHeading>
<v-card-text>
@@ -0,0 +1,72 @@
// @vitest-environment happy-dom
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
import { nextTick } from 'vue'
import BrowserExtensionCard from '../../src/components/settings/BrowserExtensionCard.vue'
import { freshPinia, mountComponent } from '../support/mountComponent.js'
// useApi is a thin fetch wrapper, so the seam is fetch itself (same shape as
// showcase.spec.js) rather than a module mock.
function stubApi(manifest) {
globalThis.fetch = vi.fn(async (url) => {
const payload = String(url).includes('/api/extension/manifest')
? manifest
: { key: 'test-key' }
return {
ok: true, status: 200, statusText: '200',
text: async () => JSON.stringify(payload),
}
})
}
async function mountCard(manifest) {
stubApi(manifest)
const w = mountComponent(BrowserExtensionCard, { pinia: freshPinia() })
// onMounted fires two fetches (manifest + key) and each resolves through a
// chain of microtasks. Yielding to a macrotask drains the whole queue, which
// a fixed number of nextTicks would only do by luck.
await new Promise((resolve) => setTimeout(resolve, 0))
await nextTick()
return w
}
const INSTALLED = {
installed: true,
version: '1.0.3499884',
xpi_url: '/extension/fabledcurator-1.0.3499884.xpi',
latest_url: '/extension/fabledcurator-latest.xpi',
sha256: 'abc',
}
describe('BrowserExtensionCard — channel', () => {
beforeEach(() => { vi.restoreAllMocks() })
afterEach(() => { delete globalThis.fetch })
it('names the channel the instance reports', async () => {
// The point of the whole channel scheme: an operator can tell a dev
// instance from a main one without installing anything.
const w = await mountCard({ ...INSTALLED, channel: 'dev' })
expect(w.text()).toContain('dev')
})
it('shows the version and the channel as SEPARATE text, never merged', async () => {
// Regression guard with teeth: the tempting shortcut is a `-dev` version
// suffix, and that is precisely what breaks the extension's comparator —
// it parses each dotted segment with parseInt, so a suffixed segment reads
// as 0 and every dev build compares equal to every other. If someone ever
// "simplifies" by folding the channel into the version, the version text
// stops being the bare derived number and this fails.
const w = await mountCard({ ...INSTALLED, channel: 'dev' })
expect(w.text()).toContain('v1.0.3499884')
expect(w.text()).not.toContain('1.0.3499884-dev')
})
it('renders no channel when the instance declares none', async () => {
// A locally-built image, or one predating the field. The card must read
// exactly as it did before the channel existed rather than inventing an
// "unknown" badge — absence is a normal answer here, not a fault.
const w = await mountCard(INSTALLED)
expect(w.text()).toContain('v1.0.3499884')
expect(w.findAll('v-chip')).toHaveLength(0)
})
})
+206
View File
@@ -0,0 +1,206 @@
#!/bin/sh
# Single definition of WHAT EACH PUBLISHED ARTIFACT IS BUILT FROM, and the
# version derived from it. Milestone 313; generalises the shape
# extension/scripts/packaging.sh established for the extension alone.
#
# Four artifacts, four independent versions. An artifact whose shipped files
# did not change keeps its version and does not rebuild — that is the whole
# point, and it is why each path set must match its Dockerfile rather than
# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
#
# too narrow -> a pin serves stale bytes, because the version did not move
# when the content did. This is the dangerous one.
# too wide -> the artifact re-versions and rebuilds for a change it does
# not ship. Merely wasteful.
#
# tests/test_artifact_paths.py asserts every COPY source in each Dockerfile is
# covered here, so adding a COPY without updating this file fails CI.
#
# POSIX sh only — CI's run shell is busybox on some paths.
#
# -f (no pathname expansion) is load-bearing for the whole script: the lists
# below are iterated with deliberate word-splitting, and without it the shell
# would glob `frontend/test/**` against the working tree and silently narrow
# the pattern. Callers substituting the output need their own `set -f` too;
# the two guards protect different expansions.
set -euf
ROOT=$(git rev-parse --show-toplevel)
# --- what each artifact ships ------------------------------------------------
#
# Each set includes its own Dockerfile and requirements: changing a base image
# or a pin changes the artifact just as surely as changing a source file.
#
# web (Dockerfile, context `.`) — the runtime stage copies backend/, alembic/,
# alembic.ini, entrypoint.sh and requirements.txt; the frontend-builder stage
# copies frontend/ and the runtime takes its `dist` output.
#
# frontend/test is excluded: `npm run build` is vite, which builds from src/,
# index.html and public/ and never reads test/. It lands in the builder layer
# but not in `dist`, so it cannot reach the shipped image.
#
# The web image ALSO bundles the signed XPI (build.yml downloads it into
# frontend/public/extension/ before the docker build), so an extension change
# changes the web image. The extension's packaged set is appended in cmd_paths
# rather than restated — one definition, per #2397.
WEB_PATHS='Dockerfile requirements.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
# ml (Dockerfile.ml, context `.`) — no frontend, no extension. Note it copies
# BOTH requirements-ml.txt and requirements.txt.
ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic alembic.ini entrypoint.sh'
# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
# live in the directory but never reach the image, so they must not re-version
# it: this is deliberately NOT `agent/`.
AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
# Which artifacts bake the BUILD CHANNEL into the image, and therefore cannot
# share a content identity across channels. The web image takes FC_CHANNEL as
# a build-arg and reports it from /api/extension/manifest (milestone 271 step
# 7), so `main` and `dev` builds of one revision are genuinely different
# images — reusing the dev one on main would ship an instance that names
# itself `dev` forever.
#
# ml and agent take no build-args at all: one revision, one image, and a merge
# to main can reuse exactly what dev already built. That is not a detail, it is
# most of what step 4 saves — merges would otherwise rebuild the agent's CUDA
# image to produce bytes that already exist.
#
# Extend this list if a second artifact ever gains a build-arg;
# tests/test_artifact_identity.py reads the Dockerfiles and fails if it drifts.
CHANNELLED='web'
usage() {
echo "usage: artifacts.sh {paths|revision|version|tag} {web|ml|agent|extension}" >&2
echo " artifacts.sh identity {web|ml|agent} [channel]" >&2
exit 2
}
# The extension's packaged set, read from its own definition rather than
# copied. packaging.sh emits `:(exclude)extension/...` entries, so the bare
# `extension` include has to come with them.
ext_paths() {
echo "extension $(sh "$ROOT/extension/scripts/packaging.sh" pathspec)"
}
cmd_paths() {
case "$1" in
web) echo "$WEB_PATHS $(ext_paths)" ;;
ml) echo "$ML_PATHS" ;;
agent) echo "$AGENT_PATHS" ;;
extension) ext_paths ;;
*) usage ;;
esac
}
# "<unix ts> <sha>" of the newest commit touching this artifact's shipped set.
# Unquoted on purpose: the pathspec must word-split into separate args.
# Globbing is already off script-wide.
newest() {
# shellcheck disable=SC2046
set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
| sort -n | tail -1)"
if [ -z "$1" ]; then
echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
exit 1
fi
echo "$1"
}
# Formatted through git rather than date(1): busybox date does not reliably
# accept `-d @<epoch>`, and git's own --date=format-local is available wherever
# git is. TZ=UTC so the value does not depend on the runner's timezone.
fmt() {
(cd "$ROOT" && TZ=UTC git show -s --format=%cd --date="format-local:$2" "$1")
}
# Leading zeros stripped so every segment is a plain integer — some version
# validators reject `08`, and a leading zero buys nothing. `0000` (midnight)
# must survive as `0`, not as the empty string.
strip0() {
printf '%s' "$1" | sed -e 's/^0*//' -e 's/^$/0/'
}
# The IDENTITY of an artifact's content: the commit its shipped files last
# changed in. This — not the tag — is what decides whether a build can be
# skipped, because the published tag is only day-precise and two different
# builds can share it.
cmd_revision() {
echo "$(newest "$1")" | cut -d' ' -f2 | cut -c1-12
}
# The ORDERING KEY: full precision, YYYY.M.D.HHMM. Used by the extension,
# where the value is what Firefox compares to decide whether an update exists
# — two same-day builds MUST be distinguishable or the second never reaches
# anyone.
cmd_version() {
sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
printf '%s.%s.%s.%s\n' \
"$(fmt "$sha" %Y)" \
"$(strip0 "$(fmt "$sha" %m)")" \
"$(strip0 "$(fmt "$sha" %d)")" \
"$(strip0 "$(fmt "$sha" %H%M)")"
}
# The PUBLISHED IMAGE TAG: day precision, YYYY.M.D. Deliberately coarser than
# the ordering key, per the operator 2026-08-28 — same-day work is not
# something worth pinning, so a second build the same day replaces the first
# rather than accumulating a tag nobody would roll back to. Safe only because
# skip decisions key on cmd_revision, never on this.
cmd_tag() {
sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
printf '%s.%s.%s\n' \
"$(fmt "$sha" %Y)" \
"$(strip0 "$(fmt "$sha" %m)")" \
"$(strip0 "$(fmt "$sha" %d)")"
}
# The CONTENT IDENTITY of a published image: an immutable tag naming exactly
# what a build of this commit would produce. build.yml asks the registry for it
# and, on a hit, skips the build entirely and repoints the channel and date
# tags at the manifest that is already there (milestone 313 step 4).
#
# It is deliberately NOT either of the other two values:
# * the date tag is day-precise and last-one-wins, so two different builds
# share it — it cannot answer "is this content published?".
# * the commit sha moves on every push, so it would never hit, which is the
# redundant rebuild this exists to remove.
#
# The revision does both jobs: it is content-unique AND stable across pushes
# that did not touch the artifact.
cmd_identity() {
_art=$1
_chan=${2:-}
case "$_art" in
web|ml|agent) ;;
extension)
echo "artifacts.sh: the extension is cached as an ext-<version> Forgejo release, not an image tag — use \`version\`" >&2
exit 2 ;;
*) usage ;;
esac
for _c in $CHANNELLED; do
if [ "$_art" = "$_c" ]; then
# Refused rather than defaulted: an unqualified identity for a
# channelled artifact would let a dev image be reused as the main one.
if [ -z "$_chan" ]; then
echo "artifacts.sh: $_art bakes the channel into the image — identity needs one" >&2
exit 2
fi
printf 'r-%s-%s\n' "$(cmd_revision "$_art")" "$_chan"
return
fi
done
printf 'r-%s\n' "$(cmd_revision "$_art")"
}
[ $# -ge 2 ] || usage
case "$1" in
paths) cmd_paths "$2" ;;
revision) cmd_revision "$2" ;;
version) cmd_version "$2" ;;
tag) cmd_tag "$2" ;;
identity) cmd_identity "$2" "${3:-}" ;;
*) usage ;;
esac
+47
View File
@@ -383,6 +383,53 @@ async def test_extension_manifest_returns_metadata_when_xpi_present(client, monk
assert body["sha256"] == hashlib.sha256(b"fake-xpi-content").hexdigest()
@pytest.mark.asyncio
async def test_extension_manifest_reports_the_channel_the_image_declares(
client, monkeypatch, tmp_path
):
"""The channel travels BESIDE the version, never inside it.
Folding it in as a `1.0.3499884-dev` suffix is the failure this design
exists to avoid: the extension's comparator parses each dotted segment as
an integer, so a suffixed segment collapses to 0 and every dev build
compares equal to every other — "no update available" and "I cannot read
this version" stop being distinguishable. Asserting the two are separate
keys is what keeps a future edit from merging them.
"""
(tmp_path / "fabledcurator-1.2.3.xpi").write_bytes(b"x")
monkeypatch.setattr(extension_module, "XPI_DIR", tmp_path)
monkeypatch.setattr(extension_module, "FC_CHANNEL", "dev")
resp = await client.get("/api/extension/manifest")
assert resp.status_code == 200
body = await resp.get_json()
assert body["channel"] == "dev"
assert body["version"] == "1.2.3"
@pytest.mark.asyncio
async def test_extension_manifest_omits_the_channel_when_the_image_declares_none(
client, monkeypatch, tmp_path
):
"""A local build, or any image from before the field existed.
The key must be ABSENT rather than present-and-empty: absence is the state
every consumer already handles (an older image conveys it by not having the
key at all), so a blank channel reuses that path instead of introducing a
second spelling of "unknown" for each reader to special-case.
"""
(tmp_path / "fabledcurator-1.2.3.xpi").write_bytes(b"x")
monkeypatch.setattr(extension_module, "XPI_DIR", tmp_path)
monkeypatch.setattr(extension_module, "FC_CHANNEL", "")
resp = await client.get("/api/extension/manifest")
assert resp.status_code == 200
body = await resp.get_json()
assert "channel" not in body
# Everything else still answers — an image with no channel is not a
# degraded one, it just cannot say which channel it came from.
assert body["installed"] is True
assert body["latest_url"] == "/extension/fabledcurator-latest.xpi"
# --- /extension/<filename> -----------------------------------------
+140
View File
@@ -0,0 +1,140 @@
"""`artifacts.sh identity` is what decides whether a build gets skipped.
Milestone 313 step 4: build.yml asks the registry for `<image>:<identity>` and,
on a hit, publishes NO new bytes — it repoints the channel and date tags at the
manifest already there. So the identity has to be a true name for the content.
Both ways of getting it wrong are silent at build time and only surface in
production:
* **too coarse** — two genuinely different images share an identity, so the
second one never gets built and its tags point at the first one's bytes. The
live case is FC_CHANNEL: a `dev` and a `main` build of one revision differ,
and collapsing them ships an instance that reports the wrong channel forever.
* **too fine** — the identity moves when the content did not, nothing ever
hits, and step 4 buys nothing. A commit sha would do exactly this.
The Dockerfiles are read here rather than trusted, because the coarse direction
appears the moment someone adds a build-arg without touching `CHANNELLED`.
"""
from __future__ import annotations
import re
import subprocess
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
# Only image artifacts have an identity — the extension is cached as an
# ext-<version> Forgejo release, not a registry tag.
IMAGE_ARTIFACTS = {
"web": "Dockerfile",
"ml": "Dockerfile.ml",
"agent": "agent/Dockerfile",
}
CHANNELS = ("main", "dev")
# docker's own tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}
_TAG = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9._-]{0,127}$")
# `ARG FC_CHANNEL` in a Dockerfile means build.yml passes a per-channel value
# in, so the channel is part of what the image IS.
_ARG_CHANNEL = re.compile(r"^\s*ARG\s+FC_CHANNEL\b", re.MULTILINE)
def identity(artifact: str, channel: str | None = None) -> subprocess.CompletedProcess:
cmd = ["sh", str(ROOT / "scripts" / "artifacts.sh"), "identity", artifact]
if channel is not None:
cmd.append(channel)
return subprocess.run(cmd, capture_output=True, text=True, cwd=ROOT)
def ok(artifact: str, channel: str | None = None) -> str:
proc = identity(artifact, channel)
assert proc.returncode == 0, f"identity {artifact} {channel}: {proc.stderr}"
return proc.stdout.strip()
def bakes_the_channel(artifact: str) -> bool:
return bool(_ARG_CHANNEL.search((ROOT / IMAGE_ARTIFACTS[artifact]).read_text()))
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
def test_channel_dependence_matches_the_dockerfile(artifact):
"""The coarse direction, caught at its source.
Whether the channel belongs in the identity is not a preference — it is
dictated by whether the Dockerfile takes it as a build-arg. Adding an
`ARG FC_CHANNEL` to another image without adding it to `CHANNELLED` would
make its dev and main builds collide, and nothing else would notice.
"""
per_channel = {c: ok(artifact, c) for c in CHANNELS}
differs = len(set(per_channel.values())) > 1
if bakes_the_channel(artifact):
assert differs, (
f"{IMAGE_ARTIFACTS[artifact]} declares ARG FC_CHANNEL, so a dev "
f"build and a main build of one revision are different images — "
f"but both derive the identity {per_channel['main']!r}. The main "
f"build would reuse the dev image and report the wrong channel. "
f"Add {artifact!r} to CHANNELLED in scripts/artifacts.sh."
)
else:
assert not differs, (
f"{IMAGE_ARTIFACTS[artifact]} takes no channel build-arg, so one "
f"revision is one image and a merge to main should reuse what dev "
f"already built — but the identity differs per channel "
f"({per_channel}), so every merge rebuilds it for nothing. Remove "
f"{artifact!r} from CHANNELLED in scripts/artifacts.sh."
)
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
def test_identity_tracks_the_artifacts_own_revision(artifact):
"""The fine direction: the identity must be the revision, not the push.
`revision` is the commit this artifact's shipped files last changed in, so
it holds still across pushes that did not touch it. Anything derived from
HEAD instead would move every push and never hit the registry.
"""
rev = subprocess.run(
["sh", str(ROOT / "scripts" / "artifacts.sh"), "revision", artifact],
capture_output=True, text=True, check=True, cwd=ROOT,
).stdout.strip()
value = ok(artifact, "main")
assert rev and rev in value, (
f"identity {value!r} does not contain the {artifact} revision {rev!r}"
)
@pytest.mark.parametrize("artifact", sorted(IMAGE_ARTIFACTS))
def test_identity_is_a_legal_docker_tag(artifact):
"""It is pushed as a tag, so an illegal one fails at the registry — after
the build has already run."""
for channel in CHANNELS:
value = ok(artifact, channel)
assert _TAG.match(value), f"{value!r} is not a valid docker tag"
def test_a_channelled_artifact_refuses_an_unqualified_identity():
"""Refusing beats defaulting. If `identity web` quietly returned the
unqualified `r-<rev>`, a workflow that forgot to pass the channel would
publish one image under a name both channels then reuse — the exact
collision the CHANNELLED list exists to prevent, reintroduced by an
omission rather than by an edit."""
proc = identity("web")
assert proc.returncode != 0, (
"identity web returned a value with no channel: "
f"{proc.stdout.strip()!r}"
)
def test_the_extension_has_no_image_identity():
"""It is cached as an ext-<version> release asset, and its cache key is the
version. Answering with a plausible image tag would invite a second,
divergent cache."""
proc = identity("extension", "main")
assert proc.returncode != 0
assert "ext-" in proc.stderr
+168
View File
@@ -0,0 +1,168 @@
"""`scripts/artifacts.sh` path sets must match what the Dockerfiles copy.
Each published artifact's version derives from the newest commit touching its
own shipped file set (milestone 313). The whole scheme rests on those sets
being right, and both ways of being wrong are silent:
* **too narrow** — a file ships but is not in the set, so the version does not
move when the content does, and a pin serves stale bytes. This is the
dangerous direction and the one this module exists for.
* **too wide** — a file is in the set but never reaches the image, so the
artifact re-versions and rebuilds for a change it does not ship.
Nothing else notices either. The version still derives, CI still goes green,
and the mismatch only surfaces as "I pinned that build and got the wrong
bytes". So the Dockerfiles are read here and compared against the declaration.
"""
from __future__ import annotations
import re
import subprocess
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
# artifact -> (dockerfile, build context relative to the repo root)
ARTIFACTS = {
"web": ("Dockerfile", ""),
"ml": ("Dockerfile.ml", ""),
"agent": ("agent/Dockerfile", "agent"),
}
# COPY --from=<stage> copies from an earlier build stage, not from the build
# context, so its source is not a repo path and cannot be in a path set.
_COPY = re.compile(r"^\s*COPY\s+(?!--from=)(?P<args>.+)$", re.MULTILINE)
def declared_paths(artifact: str) -> list[str]:
out = subprocess.run(
["sh", str(ROOT / "scripts" / "artifacts.sh"), "paths", artifact],
capture_output=True, text=True, check=True, cwd=ROOT,
).stdout
return out.split()
def includes(artifact: str) -> list[str]:
"""The set minus its `:(exclude)…` entries."""
return [p for p in declared_paths(artifact) if not p.startswith(":(exclude)")]
def copy_sources(dockerfile: str, context: str) -> list[str]:
"""Repo-relative sources of every context COPY in a Dockerfile."""
text = (ROOT / dockerfile).read_text()
sources: list[str] = []
for m in _COPY.finditer(text):
args = m.group("args").split()
# Last arg is the destination; everything before it is a source.
for src in args[:-1]:
# `frontend/package-lock.json*` — the glob is an optional-file
# idiom; the directory it sits in is what matters for coverage.
src = src.rstrip("*")
sources.append(f"{context}/{src}" if context else src)
return sources
def covered_by(path: str, include: str) -> bool:
"""`path` ships if an include names it or one of its ancestors."""
path = path.rstrip("/").lstrip("./")
include = include.rstrip("/")
return path == include or path.startswith(include + "/")
@pytest.mark.parametrize("artifact", sorted(ARTIFACTS))
def test_every_copied_path_is_in_the_artifacts_path_set(artifact):
"""The too-narrow direction — the one that serves stale bytes on a pin."""
dockerfile, context = ARTIFACTS[artifact]
inc = includes(artifact)
for src in copy_sources(dockerfile, context):
assert any(covered_by(src, i) for i in inc), (
f"{dockerfile} copies {src!r} into the {artifact} image, but no "
f"include in scripts/artifacts.sh covers it. The {artifact} "
f"version will not move when that file changes, so a pinned build "
f"will serve stale bytes. Add it to the path set.\n"
f" declared includes: {inc}"
)
@pytest.mark.parametrize("artifact", sorted(ARTIFACTS))
def test_the_dockerfile_itself_is_in_the_path_set(artifact):
"""Changing a base image or a RUN changes the artifact as surely as
changing a source file, so each set must include its own Dockerfile."""
dockerfile, _ = ARTIFACTS[artifact]
assert any(covered_by(dockerfile, i) for i in includes(artifact)), (
f"{dockerfile} is not in the {artifact} path set — a base-image bump "
f"would not move the version."
)
def test_the_web_image_versions_on_an_extension_change():
"""The web image bundles the signed XPI, so the extension's packaged files
are part of what it ships. Miss this and `:latest` serves a NEW extension
under an unchanged web version — a pin that quietly disagrees with itself.
"""
inc = includes("web")
assert any(covered_by("extension/background/background.js", i) for i in inc), (
"the web path set does not cover the extension's packaged files, but "
"build.yml downloads the signed XPI into frontend/public/extension/ "
"before the docker build"
)
def test_the_web_image_versions_on_a_version_derivation_change():
"""packaging.sh ships in no image, yet it belongs in the sets that bundle
the XPI — because it decides the version string build.yml stamps into the
packaged manifest.json. Changing the derivation changes the shipped bytes.
Left out, milestone 313 step 4 turns it silent: the new version misses the
ext-<version> cache and gets signed, while web's revision has not moved, so
the reuse path republishes the old image and the fresh signature is
orphaned. Guarded for web and the extension both, since web bundles what
the extension produces.
"""
for artifact in ("extension", "web"):
inc = includes(artifact)
excluded = [
p[len(":(exclude)"):] for p in declared_paths(artifact)
if p.startswith(":(exclude)")
]
path = "extension/scripts/packaging.sh"
assert any(covered_by(path, i) for i in inc), (
f"{path} is not in the {artifact} path set"
)
assert not any(
covered_by(path, e.rstrip("*").rstrip("/")) for e in excluded
), (
f"{path} is excluded from the {artifact} path set, so a change to "
f"how the version is derived would not move the version — and "
f"step 4 would reuse the image that carries the old one"
)
@pytest.mark.parametrize(
"artifact, path",
[
# Deliberate exclusions — the too-wide direction. Each of these lives
# beside shipped code but never reaches an image, and including it
# would re-version the artifact for a change it does not carry.
("agent", "agent/README.md"),
("agent", "agent/ruff.toml"),
("agent", "agent/docker-compose.yml"),
# vite builds from src/, index.html and public/; it never reads test/,
# so a frontend test change cannot reach `dist`.
("web", "frontend/test/gallery.spec.js"),
],
)
def test_files_that_never_reach_an_image_do_not_version_it(artifact, path):
paths = declared_paths(artifact)
excluded = [p[len(":(exclude)"):] for p in paths if p.startswith(":(exclude)")]
inc = [p for p in paths if not p.startswith(":(exclude)")]
included = any(covered_by(path, i) for i in inc)
exempted = any(covered_by(path, e.rstrip("*").rstrip("/")) for e in excluded)
assert not included or exempted, (
f"{path} is in the {artifact} path set but is not copied into the "
f"image — it would re-version and rebuild {artifact} for a change it "
f"does not ship."
)