Extension packaging: one source of truth, shadow-mode versioning, real XPI verification #235

Merged

bvandeusen merged 3 commits from dev into main 2026-08-03 20:23:27 -04:00

Conversation 0 Commits 3 Files Changed 7

+291 -63

bvandeusen commented 2026-08-03 20:23:21 -04:00

Owner

Copy Link

Copy Source

Milestone #271 steps 1–3. No change to what gets published — the shipped extension is byte-identical to what's on main today (verified: zero packaged files differ). This is tooling and CI only.

Step 1 — one definition of what ships (597b91d)

"Which files end up in the XPI" was stated in two places and about to become three. Three hand-kept copies of one fact is what allowed #2397, where the publish path could republish a stale XPI because its cache key had no link to the content it stood for.

extension/scripts/packaging.sh now holds the single declaration. package.json's web-ext scripts and ci.yml's guard both delegate to it; the version derivation will be the third consumer.

Two set -f guards are load-bearing and protect different expansions: one inside the script (its lists are iterated with deliberate word-splitting, and without -f the shell also globs them — calling pathspec from a directory containing test/ would expand it into the individual spec files), and one in each caller for the substituted result. A caller's set -f cannot cover the first; the script is a separate sh process.

Step 2 — shadow-mode versioning (1c6452e)

sign-extension and the CI guard now log the version that would be derived from git history alongside the hand-maintained one. Nothing reads it; neither site can fail because of it.

This exists because web-ext sign is one-shot per version — AMO 409s on a repeat, so a wrong formula burns a version number that can't be reclaimed. Comparing the two across real builds validates it at zero cost. Current reading: manual=1.0.10 derived=1.0.19.

sign-extension now uses fetch-depth: 0; a commit count needs real history.

Step 3 — verify the actual XPI (1c6452e, fixed in 11dd324)

Every other packaging assertion checks our declaration against itself. This is the first that asks web-ext what it actually wrote into the archive — and the two disagreed on the first run.

The files were correctly excluded, but web-ext writes a separate entry for each directory, and test/** doesn't match test/. The archive carried empty test/ and scripts/ entries. Fixed by listing both forms per directory (the bare name alone wouldn't match the contents). The archive is now exactly the 19 shipping entries.

---

Expected on merge: ext-1.0.10 already exists and no packaged file changed, so sign-extension will cache-hit and skip AMO — a fast build, and the correct outcome since the shipped content is unchanged.

One consequence to be aware of: the published 1.0.10 XPI was signed before 11dd324, so it still carries the empty test/ directory entry. That's cosmetic (an empty entry, no code) and it won't clear until the next version bump, because the cache correctly refuses to re-sign an unchanged version. It's also a live demonstration of exactly the cache behaviour milestone #271 exists to make content-aware.

Not included: Step 4 (cutover) stays gated until shadow data accumulates across several main builds — sign-extension only runs on main, so those are the only builds that can evidence the formula.

Milestone #271 steps 1–3. **No change to what gets published** — the shipped extension is byte-identical to what's on `main` today (verified: zero packaged files differ). This is tooling and CI only. ## Step 1 — one definition of what ships (`597b91d`) "Which files end up in the XPI" was stated in two places and about to become three. Three hand-kept copies of one fact is what allowed #2397, where the publish path could republish a stale XPI because its cache key had no link to the content it stood for. `extension/scripts/packaging.sh` now holds the single declaration. `package.json`'s web-ext scripts and `ci.yml`'s guard both delegate to it; the version derivation will be the third consumer. Two `set -f` guards are load-bearing and protect *different* expansions: one inside the script (its lists are iterated with deliberate word-splitting, and without `-f` the shell also globs them — calling `pathspec` from a directory containing `test/` would expand it into the individual spec files), and one in each caller for the substituted result. A caller's `set -f` cannot cover the first; the script is a separate `sh` process. ## Step 2 — shadow-mode versioning (`1c6452e`) `sign-extension` and the CI guard now log the version that *would* be derived from git history alongside the hand-maintained one. Nothing reads it; neither site can fail because of it. This exists because `web-ext sign` is one-shot per version — AMO 409s on a repeat, so a wrong formula burns a version number that can't be reclaimed. Comparing the two across real builds validates it at zero cost. Current reading: `manual=1.0.10 derived=1.0.19`. `sign-extension` now uses `fetch-depth: 0`; a commit count needs real history. ## Step 3 — verify the actual XPI (`1c6452e`, fixed in `11dd324`) Every other packaging assertion checks our declaration against itself. This is the first that asks web-ext what it actually wrote into the archive — **and the two disagreed on the first run.** The files were correctly excluded, but web-ext writes a separate entry for each *directory*, and `test/**` doesn't match `test/`. The archive carried empty `test/` and `scripts/` entries. Fixed by listing both forms per directory (the bare name alone wouldn't match the contents). The archive is now exactly the 19 shipping entries. --- **Expected on merge:** `ext-1.0.10` already exists and no packaged file changed, so `sign-extension` will **cache-hit and skip AMO** — a fast build, and the correct outcome since the shipped content is unchanged. One consequence to be aware of: the published 1.0.10 XPI was signed before `11dd324`, so it still carries the empty `test/` directory entry. That's cosmetic (an empty entry, no code) and it won't clear until the next version bump, because the cache correctly refuses to re-sign an unchanged version. It's also a live demonstration of exactly the cache behaviour milestone #271 exists to make content-aware. **Not included:** Step 4 (cutover) stays gated until shadow data accumulates across several `main` builds — `sign-extension` only runs on `main`, so those are the only builds that can evidence the formula.

bvandeusen added 3 commits 2026-08-03 20:23:21 -04:00

refactor(extension): one definition of what ships in the XPI ... 597b91d29b

Milestone #271 step 1. Groundwork for deriving the extension version from git;
no behavior change yet -- nothing consumes `version` so far.

"Which files end up in the XPI" was stated in two places and about to become
three. Three hand-kept copies of one fact is what allowed #2397, where the
publish path could republish a stale XPI because its cache key had no link to
the content it stood for.

New extension/scripts/packaging.sh holds the single declaration and exposes:
  ignore       web-ext --ignore-files values
  pathspec     :(exclude)extension/... for git
  version      <MAJOR.MINOR from manifest>.<commit count over packaged files>
  major-minor  /  patch

Consumers now delegate instead of restating it:
- extension/package.json -- all four web-ext scripts
- .forgejo/workflows/ci.yml -- the extension-version guard's exclusions
- (step 4) the rev-list that derives the version

scripts/** joins the non-packaged set; the script must not ship to users.

Two shell hazards, both load-bearing:

The script runs `set -euf`. Its lists are iterated with deliberate word
splitting, and without -f the shell ALSO globs them -- invoking `pathspec`
from a directory where test/ exists (exactly how ci.yml calls it) would expand
`test/**` into the individual spec files and silently stop covering anything
added later. A caller's own `set -f` cannot prevent this: the script is a
separate sh process and does not inherit it.

Callers additionally need their own `set -f` for the substituted RESULT, which
is a different expansion. version.spec.js asserts every --ignore-files caller
sets it, that the pathspec comes through with `test/**` literal and no
.spec.js paths, and that neither consumer has reinstated a hardcoded list --
the easy future regression is "simplifying" by inlining one again.

Verified: all five subcommands plus the usage/exit-2 path. Derived version on
main (8300029) is 1.0.19, matching dev. Last published is 1.0.10, so the
eventual cutover moves strictly upward and needs no offset -- Firefox refuses
downgrades. (An earlier note recorded 18; that was measured against a stale
origin/main from before the PR #234 merge.)

Refs #2398

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

ci(extension): shadow the derived version + verify real XPI contents ... 1c6452e10e

Milestone #271 steps 2 and 3. Neither changes what gets published.

STEP 2 -- shadow mode.

build.yml's sign-extension and ci.yml's extension-version guard now log the
version that WOULD be derived from git history alongside the hand-maintained
one. Nothing reads the derived value, and neither site can fail because of it.

This exists because `web-ext sign` is one-shot per version: AMO 409s on a
repeat, so a wrong formula burns a real version number that cannot be
reclaimed. Comparing the two across real builds is the only way to validate it
at zero cost. sign-extension runs on main only, so main pushes are the sole
source of truth for whether the derived number moves exactly when the shipped
extension changes -- the dev-side log is a convenience, not the evidence.

sign-extension now checks out with fetch-depth: 0. The derived version is a
commit count and a depth-1 clone cannot produce one.

STEP 3 -- XPI content verification.

Every other packaging assertion checks our declaration against itself. This is
the first that asks web-ext what it ACTUALLY wrote into the archive.

That assumption was both unverified and fragile: `test/**` only survives to
web-ext because callers `set -f` before substituting it, so losing that
quoting would silently start shipping dev files with no other signal. The step
builds the XPI and asserts test/, scripts/, vitest.config.js, package.json,
package-lock.json, README.md and node_modules are absent -- and, because an
over-matching exclusion would break the extension at runtime rather than at
build time, that manifest.json, all four lib/*.js and every UI directory are
present.

unzip is installed only when missing; node:24-bookworm-slim may not carry it.

Refs #2399, #2400

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(extension): exclude the test/ and scripts/ directory entries from the XPI ... 11dd324f89

The XPI-content check added in 1c6452e did its job on its first run: the
archive carried empty `test/` and `scripts/` entries.

`test/**` matches the files inside a directory but not the directory entry
itself, and web-ext writes an entry for the directory separately -- so the
contents were correctly excluded while the empty directories shipped anyway.
Nothing harmful reached users (no dev code, just two empty entries), but our
single declaration claimed these do not ship and something was shipping.

Both forms are now listed per directory. The bare name alone would not do:
minimatch's `test` does not match `test/url.spec.js`, so dropping the glob
would ship the contents instead.

Verified locally: the derived version is unchanged at 1.0.19, confirming the
added entries are redundant for the git pathspec and only affect web-ext.

Refs #2400

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

bvandeusen merged commit 0d204e6837 into main 2026-08-03 20:23:27 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#235