Compare commits
19
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2e01242381 | ||
|
|
41f2bec3af | ||
|
|
d38585ed94 | ||
|
|
a3071a7549 | ||
|
|
b6b9fd8287 | ||
|
|
bce894ba24 | ||
|
|
5771fd5770 | ||
|
|
b3989d0224 | ||
|
|
cd0b0ff04a | ||
|
|
454eb3f973 | ||
|
|
7e065fed70 | ||
|
|
dee93faa37 | ||
|
|
d9aa5aa832 | ||
|
|
fb2c4d5b80 | ||
|
|
609bc82acc | ||
|
|
7a20c55441 | ||
|
|
0c43fa3eb2 | ||
|
|
cf06c81db9 | ||
|
|
0db38cc111 |
+799
-142
File diff suppressed because it is too large
Load Diff
+46
-22
@@ -2,7 +2,7 @@ name: CI
|
||||
|
||||
# CI lanes per FabledRulebook/forgejo.md "CI philosophy":
|
||||
# - lint: ruff only, no dep install — fast-fail for the common lint bounce.
|
||||
# - extension-version: the derived version resolves and MAJOR.MINOR agrees.
|
||||
# - extension-version: the derived version resolves and is a shape AMO takes.
|
||||
# - backend-lint-and-test: `pytest -m "not integration"`, no service containers.
|
||||
# - frontend-build: vitest unit + vite build.
|
||||
# - integration: pgvector + redis service containers; alembic + `pytest -m integration`.
|
||||
@@ -35,7 +35,10 @@ jobs:
|
||||
- name: Ruff lint
|
||||
# agent/ included so the GPU-agent is linted before its image is built
|
||||
# (build.yml only `docker build`s it — this is where it gets checked).
|
||||
run: ruff check backend/ tests/ alembic/ agent/
|
||||
# scripts/ likewise: release_notes.py runs only on a tag push, so a
|
||||
# syntax or import error there would otherwise surface at the one
|
||||
# moment nobody wants to debug a workflow.
|
||||
run: ruff check backend/ tests/ alembic/ agent/ scripts/
|
||||
- name: Agent syntax check
|
||||
# The agent's runtime deps (torch/transformers/ultralytics) aren't in the
|
||||
# CI image, so we can't import it — but compileall parses every module,
|
||||
@@ -55,9 +58,12 @@ jobs:
|
||||
# the extension.yml suite runs on node:24-slim, which is exactly why
|
||||
# version.spec.js sticks to packaging.sh's git-free subcommands.
|
||||
# 1. the derivation actually resolves on this commit
|
||||
# 2. MAJOR.MINOR agrees between the two files — the one part still hand-set,
|
||||
# and packaging.sh reads it from manifest.json ALONE, so a divergence
|
||||
# ships a version package.json disagrees with
|
||||
# 2. the derived string is one AMO will accept, checked against Mozilla's
|
||||
# own published grammar rather than a loose "digits and dots"
|
||||
#
|
||||
# The MAJOR.MINOR-agreement check that used to be (2) is gone with milestone
|
||||
# 318 step 8: the committed version no longer seeds anything, so there is no
|
||||
# hand-set part left for the two files to disagree about.
|
||||
#
|
||||
# Deliberately NOT checked here: that the derived value beats what has already
|
||||
# been signed. That guard belongs in build.yml, where it compares against the
|
||||
@@ -82,27 +88,38 @@ jobs:
|
||||
# busybox sh on the act_runner — no bashisms (family rule).
|
||||
VERSION=$(sh extension/scripts/packaging.sh version)
|
||||
echo "derived: $VERSION"
|
||||
# The shape AMO accepts, and the shape build.yml will stamp.
|
||||
if ! echo "$VERSION" | grep -qE '^[0-9]+(\.[0-9]+)*$'; then
|
||||
echo "ERROR: derived version '$VERSION' is not plain dotted-numeric."
|
||||
echo "AMO would reject it, and build.yml stamps it verbatim."
|
||||
|
||||
# Mozilla's published grammar for AMO, transcribed verbatim from
|
||||
# MDN's manifest.json/version page:
|
||||
#
|
||||
# ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$
|
||||
#
|
||||
# Not the looser `^[0-9]+(\.[0-9]+)*$` this lane used to carry. That
|
||||
# one passes `2026.08.29.0201`, which AMO REJECTS — a segment must be
|
||||
# the single digit 0 or start 1-9 — and it also passes five segments,
|
||||
# where AMO allows four. Both would surface as a failed sign with the
|
||||
# version already burned: AMO 409s on re-signing, so a rejected value
|
||||
# cannot be reclaimed and cannot be reused. This lane is the cheap
|
||||
# place to find out. (#3138, milestone 318 step 8.)
|
||||
if ! echo "$VERSION" | grep -qE '^(0|[1-9][0-9]{0,8})(\.(0|[1-9][0-9]{0,8})){0,3}$'; then
|
||||
echo "ERROR: derived version '$VERSION' is not a version AMO accepts."
|
||||
echo "AMO's grammar: ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$"
|
||||
echo "Most likely cause: a zero-padded segment (08, 0201). The rest"
|
||||
echo "of the family pads; the extension must not — see packaging.sh."
|
||||
exit 1
|
||||
fi
|
||||
mm() { grep -E '"version"' "$1" | head -1 | sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([0-9]+\.[0-9]+).*/\1/'; }
|
||||
MAN=$(mm extension/manifest.json)
|
||||
PKG=$(mm extension/package.json)
|
||||
test -n "$MAN" || { echo "ERROR: no parseable version in extension/manifest.json"; exit 1; }
|
||||
test -n "$PKG" || { echo "ERROR: no parseable version in extension/package.json"; exit 1; }
|
||||
if [ "$MAN" != "$PKG" ]; then
|
||||
echo "ERROR: MAJOR.MINOR disagrees between the two files."
|
||||
echo " extension/manifest.json = $MAN <- packaging.sh reads MAJOR.MINOR from here"
|
||||
echo " extension/package.json = $PKG"
|
||||
echo "Only MAJOR.MINOR is hand-set. The patch component is derived from"
|
||||
echo "commit time and overwritten at build time, so the committed patch"
|
||||
echo "numbers are inert — but MAJOR.MINOR still ships. Set both the same."
|
||||
|
||||
# ...and the shape this project actually derives. AMO would happily
|
||||
# take `1.0.3500147` too, so the grammar check alone would not notice
|
||||
# a regression to the pre-318 shape — which orders BELOW everything
|
||||
# signed since, and is unrecoverable once Firefox has the higher one.
|
||||
if ! echo "$VERSION" | grep -qE '^20[0-9][0-9]\.[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,4}$'; then
|
||||
echo "ERROR: derived version '$VERSION' is not YYYY.M.D.HHMM."
|
||||
echo "Rule 148's CalVer is what build.yml signs; the old"
|
||||
echo "1.0.<minutes> shape would order below every ext-2026.* release."
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: MAJOR.MINOR $MAN, derived version $VERSION"
|
||||
echo "OK: derived version $VERSION"
|
||||
|
||||
backend-lint-and-test:
|
||||
runs-on: python-ci
|
||||
@@ -115,6 +132,13 @@ jobs:
|
||||
SECRET_KEY: ci_unit_test_placeholder
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Full history for tests/test_artifact_identity.py, which derives
|
||||
# each artifact's revision to check the identity scheme. On a
|
||||
# depth-1 clone that derivation either fails or returns the tip sha
|
||||
# — so the lane would go green while asserting nothing, which is
|
||||
# the one outcome worse than a red one.
|
||||
fetch-depth: 0
|
||||
|
||||
# Cache step removed 2026-05-26: act_runner's cache backend has been
|
||||
# broken on this homelab runner since 2026-05-15 (first as request-
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
name: Release
|
||||
|
||||
# A `v*` tag publishes a changelog. It does NOT build anything.
|
||||
#
|
||||
# Milestone 318 step 2 removed the tag trigger from build.yml: by the time
|
||||
# anyone tags a commit, `main` has already built and published it, and a
|
||||
# rebuild would re-push `:c-<sha>` — which rule 145 forbids even when the
|
||||
# source matches, since image configs carry timestamps and "same source" does
|
||||
# not mean "same manifest". That left the tag with no consequence at all.
|
||||
#
|
||||
# This is the consequence it has instead. Step 6 put the derived version in the
|
||||
# Settings footer, so an operator can say WHICH build they are running; this
|
||||
# says what is IN it that was not in the one they ran last month. Both halves
|
||||
# of one question (note #3127 §5).
|
||||
#
|
||||
# Nothing here runs on a schedule and nothing auto-tags on merge. Release tags
|
||||
# are bookmarks — cut one when you will want to point at that day by name,
|
||||
# otherwise don't (note #3127 §0). FC went twelve weeks between v26.06.04.0 and
|
||||
# the next one and nothing was wrong. A schedule would turn an optional
|
||||
# bookmark back into ceremony, which is the thing this milestone is removing.
|
||||
#
|
||||
# Cutting the tag is an explicit operator action under rule 2 ("`main` — never
|
||||
# without explicit request", which since 2026-08-28 covers PR, merge and tag
|
||||
# alike). This lane only decides what happens once they do.
|
||||
#
|
||||
# Requires repo secret RELEASE_TOKEN with the `write:release` scope — the same
|
||||
# PAT build.yml uses for the ext-<version> XPI asset cache.
|
||||
|
||||
on:
|
||||
push:
|
||||
tags: ['v*']
|
||||
# So a release body can be regenerated after the fact — the publisher PATCHes
|
||||
# an existing release rather than falling through on a conflict, so re-running
|
||||
# this on a tag rewrites the body instead of silently keeping the first one
|
||||
# (note #3127 §6.7).
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Tag to (re)publish notes for'
|
||||
required: true
|
||||
|
||||
jobs:
|
||||
changelog:
|
||||
runs-on: python-ci
|
||||
container:
|
||||
image: git.fabledsword.com/bvandeusen/ci-python:3.14
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Load-bearing twice over: the previous release is found by walking
|
||||
# ancestry back through the tag graph, and the cross-check against
|
||||
# the derived web version calls artifacts.sh, which reads commit
|
||||
# times. A shallow clone would find no previous tag and emit the
|
||||
# entire history as the changelog — plausible-looking and wrong.
|
||||
fetch-depth: 0
|
||||
ref: ${{ github.event.inputs.tag || github.ref }}
|
||||
|
||||
# The `:c-<sha>` rollback refs are only real if `main` built this commit.
|
||||
# The script checks that against origin/main and downgrades the claim to
|
||||
# "unverified" when it cannot resolve one; fetching it here means that
|
||||
# downgrade stays an actual signal instead of firing on every release.
|
||||
- name: Make main's history resolvable
|
||||
run: git fetch --no-tags --quiet origin +main:refs/remotes/origin/main || true
|
||||
|
||||
# TAG goes through the environment, not through `${{ }}` inside the
|
||||
# run block. The value is operator-supplied, and an expression expanded
|
||||
# into a shell line is expanded BEFORE the shell sees it — there is no
|
||||
# quoting that makes that safe. On a tag push it is empty and the script
|
||||
# falls back to GITHUB_REF.
|
||||
- name: Publish the derived changelog
|
||||
env:
|
||||
RELEASE_TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
||||
TAG: ${{ github.event.inputs.tag }}
|
||||
run: |
|
||||
set -eu
|
||||
if [ -n "${TAG:-}" ]; then
|
||||
python3 scripts/release_notes.py "$TAG"
|
||||
else
|
||||
python3 scripts/release_notes.py
|
||||
fi
|
||||
+9
-3
@@ -58,11 +58,17 @@ COPY --from=frontend-builder /build/dist ./frontend/dist
|
||||
# exactly the shape every reader already has to handle.
|
||||
#
|
||||
# Declared LAST on purpose. An ARG/ENV invalidates every layer below it, and
|
||||
# this is the one value that differs between the dev and main builds of
|
||||
# identical source — put it any earlier and the two channels could never share
|
||||
# a cached pip install.
|
||||
# these are the values that differ between builds of otherwise identical
|
||||
# source — put them any earlier and the two channels could never share a
|
||||
# cached pip install.
|
||||
#
|
||||
# FC_VERSION is what the instance reports about itself in the UI. Since
|
||||
# milestone 318 stopped publishing version image tags, that self-report is
|
||||
# the only answer to "which build is this?" — nothing else names it.
|
||||
ARG FC_CHANNEL=""
|
||||
ENV FC_CHANNEL=${FC_CHANNEL}
|
||||
ARG FC_VERSION=""
|
||||
ENV FC_VERSION=${FC_VERSION}
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
|
||||
@@ -10,6 +10,35 @@ In production. `main` is continuously deployed — every merge to `main` builds
|
||||
and publishes `:latest` images, so whatever is on `main` is what is running.
|
||||
Day-to-day work happens on `dev`, which publishes `:dev` images.
|
||||
|
||||
## Versions and tags
|
||||
|
||||
Three image tags exist, and no others:
|
||||
|
||||
| Tag | Branch | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `:latest` | `main` | Production. Moves on every merge. |
|
||||
| `:c-<sha>` | `main` | Immutable — the rollback unit, all three images together. |
|
||||
| `:dev` | `dev` | The rolling test channel. Moves on every push. |
|
||||
|
||||
There are deliberately **no version tags**. Nothing pins one, and a per-build
|
||||
name nobody reads is upkeep for a model FC does not run (family rule 145; the
|
||||
reasoning is note #3127 §5). Rolling back is `docker pull …:c-<sha>`.
|
||||
|
||||
Each artifact still has a version, derived rather than chosen: the commit time
|
||||
of the newest change to that artifact's *own* shipped files, as
|
||||
`YYYY.MM.DD.HHMM` UTC (rule 148). Four artifacts, four independent versions —
|
||||
a push touching only `agent/` re-versions the agent and leaves web and ml
|
||||
alone, and CI skips the builds whose content did not move.
|
||||
|
||||
Because no registry name carries it, the running instance's own report is the
|
||||
only answer to "which build is this?". The foot of Settings shows
|
||||
`FabledCurator 2026.08.29.0201 · dev`, and `/api/health` returns the same two
|
||||
fields.
|
||||
|
||||
Release tags are optional bookmarks — FC went twelve weeks without one and
|
||||
nothing was wrong. Pushing `v<version>` publishes a Forgejo release listing the
|
||||
commits since the previous tag; it builds no image.
|
||||
|
||||
## What's in here
|
||||
|
||||
Five deployable pieces, built by `.forgejo/workflows/build.yml`:
|
||||
@@ -52,9 +81,10 @@ FabledCurator is designed to run inside a self-hosted homelab environment over p
|
||||
|
||||
## CI / Forgejo setup
|
||||
|
||||
Three workflows: `ci.yml` (lint, extension-version check, backend unit tests,
|
||||
Four workflows: `ci.yml` (lint, extension-version check, backend unit tests,
|
||||
frontend build, integration), `extension.yml` (extension lint, vitest, XPI
|
||||
content verification), and `build.yml` (sign + publish).
|
||||
content verification), `build.yml` (sign + publish), and `release.yml`, which
|
||||
runs only on a `v*` tag and publishes a changelog without building anything.
|
||||
|
||||
**The toolchain each job runs in is its `container.image`, not its `runs-on`
|
||||
label.** `runs-on: python-ci` only schedules the job onto a runner; every job
|
||||
@@ -71,8 +101,12 @@ The repo expects one secret:
|
||||
|
||||
Generate at https://git.fabledsword.com/user/settings/applications. The injected `GITHUB_TOKEN` cannot be used because it lacks `write:package`.
|
||||
|
||||
AMO signing additionally needs `MOZILLA_AMO_JWT_KEY` / `MOZILLA_AMO_JWT_SECRET`; it runs on
|
||||
`main` only and is cached per version, since AMO rejects a re-signed version.
|
||||
AMO signing additionally needs `MOZILLA_AMO_JWT_KEY` / `MOZILLA_AMO_JWT_SECRET`.
|
||||
It runs on **both** channels and is cached per version: because the version is
|
||||
derived from commit time, `dev` and `main` derive the same number for the same
|
||||
source, so `main` finds `dev`'s signature already cached and makes no second AMO
|
||||
call. That cache is why signing must be one-shot — AMO rejects a re-signed
|
||||
version.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
@@ -7,13 +7,13 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from quart import Blueprint, jsonify, request
|
||||
from sqlalchemy import select
|
||||
|
||||
from ..build_info import FC_CHANNEL as _FC_CHANNEL
|
||||
from ..extensions import get_session
|
||||
from ..models import AppSetting
|
||||
from ..services.extension_service import (
|
||||
@@ -33,10 +33,13 @@ XPI_DIR = Path("/app/frontend/dist/extension")
|
||||
_XPI_VERSION_RE = re.compile(r"fabledcurator-(?P<version>[\w.-]+)\.xpi$")
|
||||
|
||||
# Which channel this image belongs to — "dev" or "main" — baked in at build
|
||||
# time from the FC_CHANNEL build arg (milestone 271 step 7). Empty for a local
|
||||
# build, or for any image predating the field. Tests override by monkeypatching
|
||||
# this constant, same as XPI_DIR above.
|
||||
FC_CHANNEL = os.environ.get("FC_CHANNEL", "").strip()
|
||||
# time (milestone 271 step 7). Read from build_info rather than the environment
|
||||
# a second time: /api/health reports the same value, and two independent
|
||||
# `os.environ.get` calls are two things that can drift.
|
||||
#
|
||||
# Still bound as a module-level name here, so tests monkeypatch
|
||||
# `extension.FC_CHANNEL` exactly as they did before, same as XPI_DIR above.
|
||||
FC_CHANNEL = _FC_CHANNEL
|
||||
|
||||
|
||||
async def _ext_key_required(session) -> bool:
|
||||
|
||||
@@ -1,5 +1,20 @@
|
||||
"""Health endpoint — no DB or Redis touch; just liveness."""
|
||||
"""Health endpoint — no DB or Redis touch; liveness, plus the build's identity.
|
||||
|
||||
The identity rides here rather than on a route of its own because it answers
|
||||
at the same cost: two module constants, no I/O, nothing that can be slow or
|
||||
fail. It is also already fetched app-wide — TopNav calls `refreshHealth` on
|
||||
mount — so a separate endpoint would mean a second request for two strings.
|
||||
|
||||
Both fields are OMITTED when unset rather than sent empty. See build_info.
|
||||
"""
|
||||
|
||||
from ..build_info import FC_CHANNEL, FC_VERSION
|
||||
|
||||
|
||||
async def get_health():
|
||||
return {"status": "ok"}, 200
|
||||
body = {"status": "ok"}
|
||||
if FC_VERSION:
|
||||
body["version"] = FC_VERSION
|
||||
if FC_CHANNEL:
|
||||
body["channel"] = FC_CHANNEL
|
||||
return body, 200
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""What this build IS — stamped at image build time, not configurable.
|
||||
|
||||
Deliberately separate from `config.py`. Those are operator settings, read from
|
||||
the environment and meant to be changed. These describe the artifact itself and
|
||||
are baked in by CI (the `FC_VERSION` / `FC_CHANNEL` build args); an operator
|
||||
setting them by hand is not a supported thing to do, it is just how a value
|
||||
gets from the build into the running process.
|
||||
|
||||
**Absent rather than empty when unknown.** A locally-built image has no version,
|
||||
and neither did any image predating the field — one spelling of "cannot say",
|
||||
which every reader already has to handle, instead of a second one to
|
||||
special-case (note #3127 §7).
|
||||
|
||||
**Why this matters more than it used to.** Milestone 318 stopped publishing
|
||||
version image tags, so a running instance's self-report is now the *only*
|
||||
answer to "which build is this?" — there is no registry name left to check it
|
||||
against. A wrong value here has nothing to contradict it. That is why the UI
|
||||
renders `unknown` rather than a blank or a plausible default: an empty footer
|
||||
reads as "no version", which is a different and false claim.
|
||||
|
||||
The channel lives BESIDE the version and is never folded into it (rule 149).
|
||||
A `-dev` suffix would be parsed by the extension's comparator as a segment
|
||||
worth 0, making every dev build compare equal to every other — issue #2993's
|
||||
exact failure.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
FC_VERSION = os.environ.get("FC_VERSION", "").strip()
|
||||
FC_CHANNEL = os.environ.get("FC_CHANNEL", "").strip()
|
||||
+98
-29
@@ -9,14 +9,17 @@ git.fabledsword.com/bvandeusen/ci-python:3.14
|
||||
## Image deps used
|
||||
|
||||
- python 3.14
|
||||
- ruff (analyzer for `backend/`, `tests/`, `alembic/`)
|
||||
- ruff (analyzer for `backend/`, `tests/`, `alembic/`, `agent/`, `scripts/`)
|
||||
- node (frontend job: `npm install` + vitest + vite build)
|
||||
- docker CLI + buildx (`.forgejo/workflows/build.yml`: build-web, build-ml — Fabled-Git registry push)
|
||||
- docker CLI + buildx (`.forgejo/workflows/build.yml`: build-web, build-ml, build-agent — Fabled-Git registry push, and `imagetools inspect`/`create` for the reuse path)
|
||||
|
||||
## Secondary runtime image
|
||||
|
||||
node:24-bookworm-slim — `.forgejo/workflows/extension.yml` only.
|
||||
|
||||
`.forgejo/workflows/release.yml` runs on `ci-python:3.14` like everything else
|
||||
and installs nothing: it needs git and stdlib python, and builds no image.
|
||||
|
||||
The extension lane is the one job that does NOT run on `ci-python:3.14`: it
|
||||
needs a current Node for `web-ext` and vitest and nothing Python at all. Kept
|
||||
on the upstream slim image rather than adding a Node toolchain to `ci-python`,
|
||||
@@ -54,34 +57,100 @@ per `docs/process.md`'s "add deps to the image when used by >1 project".
|
||||
shims to production code — the libs ship as `background.scripts`, not ES
|
||||
modules, so the specs exercise exactly the bytes packaged into the XPI.
|
||||
- **`extension/scripts/packaging.sh` is the single definition of what ships
|
||||
inside the XPI.** Two consumers read from it rather than keeping their own
|
||||
copy: web-ext's `--ignore-files` (`extension/package.json`), and the `git log`
|
||||
pathspec inside the script's own version derivation. It was three until
|
||||
2026-08-27 — `ci.yml`'s `extension-version` guard held the third and went when
|
||||
the manual bump it guarded did (milestone 271 step 5). Hand-kept copies of
|
||||
that one fact is what allowed issue #2397, so `extension/test/version.spec.js`
|
||||
asserts no workflow has reintroduced a literal `:(exclude)extension/…`.
|
||||
inside the XPI.** Three consumers read from it rather than keeping their own
|
||||
copy: web-ext's `--ignore-files` (`extension/package.json`), the `git log`
|
||||
pathspec inside the script's own version derivation, and `scripts/artifacts.sh`,
|
||||
which appends the extension's set to web's because the web image bundles the
|
||||
signed XPI. Hand-kept copies of that one fact is what allowed issue #2397, so
|
||||
`extension/test/version.spec.js` asserts no workflow has reintroduced a
|
||||
literal `:(exclude)extension/…`.
|
||||
- **Packaged and version-relevant are two different sets** (#3156). `scripts/`
|
||||
is excluded from the XPI and is NOT excluded from the version derivation,
|
||||
because `packaging.sh` decides the version string stamped into the packaged
|
||||
`manifest.json`. The membership test is *"can changing this file change the
|
||||
published bytes?"*, not *"is this file copied in?"* — which is why the script
|
||||
keeps two lists rather than one.
|
||||
- **The shipped extension version is derived, not committed.** It is the commit
|
||||
TIME of the newest packaged-extension change (minutes since 2020-01-01, per
|
||||
family rule 149 — never a commit count, which orders by branch rather than by
|
||||
recency). `build.yml`'s `sign-extension` computes it and stamps it into
|
||||
`extension/manifest.json` + `package.json` in the working tree before signing;
|
||||
the stamp is never committed. Treat the version in the repo as a base: only
|
||||
its MAJOR.MINOR is read, and its patch component is inert.
|
||||
- Every job that calls `packaging.sh version` checks out with `fetch-depth: 0` —
|
||||
`build.yml`'s `sign-extension` and `build-web`, and `ci.yml`'s
|
||||
`extension-version`. A depth-1 clone sees one commit and derives a wrong,
|
||||
too-low value **rather than failing**, so the full-history checkout is
|
||||
load-bearing rather than incidental.
|
||||
- **`FC_CHANNEL` is a build arg, not a runtime setting.** `build.yml` passes
|
||||
`dev` / `main` to the web image only (the ml and agent images have nothing to
|
||||
report it to), and `/api/extension/manifest` reports it beside the version so
|
||||
an install can be traced to a channel. It is declared LAST in the Dockerfile
|
||||
on purpose: an ARG invalidates every layer below it, and this is the one value
|
||||
that differs between the dev and main builds of identical source, so placing
|
||||
it earlier would stop the two channels ever sharing a cached `pip install`.
|
||||
Empty by default — a local build then reports no channel at all rather than
|
||||
claiming one.
|
||||
TIME of the newest packaged-extension change, rendered `YYYY.M.D.HHMM` UTC
|
||||
(family rules 148/149 — never a commit count, which orders by branch rather
|
||||
than by recency). `build.yml`'s `sign-extension` computes it and stamps it
|
||||
into `extension/manifest.json` + `package.json` in the working tree before
|
||||
signing; the stamp is never committed. The version in the repo is **wholly
|
||||
inert** — since milestone 318 step 8 there is no hand-set MAJOR.MINOR either.
|
||||
- **The extension is the one artifact that does not zero-pad, and that is not a
|
||||
drift** (#3138). Mozilla's grammar for AMO is
|
||||
`^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$` — a segment is the single
|
||||
digit `0` or starts 1-9, and there are at most four. `2026.08.29.0201` is
|
||||
rejected; `2026.8.29.201` is the same value one character narrower per
|
||||
segment, and rule 148 defines comparison as numeric per segment, so nothing is
|
||||
reordered. `ci.yml`'s `extension-version` lane asserts the derived string
|
||||
against that exact regex, plus a `YYYY.M.D.HHMM` shape check that would catch
|
||||
a regression to the pre-318 `1.0.<minutes>` — which AMO would accept and which
|
||||
orders below everything already signed. Checking here is the whole point: AMO
|
||||
409s on re-signing, so a version it rejects is burned and cannot be reused.
|
||||
`scripts/artifacts.sh version extension` **delegates** to `packaging.sh` so
|
||||
the two cannot answer differently.
|
||||
- Every job that derives anything checks out with `fetch-depth: 0` — all four
|
||||
`build.yml` jobs, `ci.yml`'s `extension-version` and `backend-lint-and-test`
|
||||
(for `tests/test_artifact_paths.py` and `test_artifact_identity.py`), and
|
||||
`release.yml`, which additionally walks the tag graph. A depth-1 clone sees
|
||||
one commit and derives a wrong, too-low value **rather than failing**, so the
|
||||
full-history checkout is load-bearing rather than incidental.
|
||||
- **`scripts/artifacts.sh` is the same shape one level up: one definition per
|
||||
artifact of what it is built from, and the two values derived from it.**
|
||||
`revision` (12 hex of the newest commit touching that set) and `version`
|
||||
(`YYYY.MM.DD.HHMM` UTC, rule 148). Four artifacts, four independent answers,
|
||||
so a push touching only `agent/` leaves web and ml alone.
|
||||
`tests/test_artifact_paths.py` reads each Dockerfile and asserts every COPY
|
||||
source is covered, so adding a COPY without updating the script fails CI.
|
||||
- **A file that DECIDES an artifact's identity belongs in its set even though it
|
||||
is copied into nothing** — `packaging.sh` for the extension and web (#3156),
|
||||
and `artifacts.sh` itself for web (#3202), which decides the `FC_VERSION`
|
||||
baked into that image. Only web needs the second entry: every artifact stamps
|
||||
a revision, but a revision has a backstop (a changed derivation stops matching
|
||||
the published label and forces a rebuild) and a version has none, because
|
||||
nothing compares it to anything. `tests/test_artifact_paths.py`'s `DERIVERS`
|
||||
table is the guard.
|
||||
- **Builds are skipped when the content is already published.** Each image
|
||||
carries its revision as an `fc.revision` LABEL, and `build.yml` reads that
|
||||
label back off the moving channel tag (`imagetools inspect --format`). Equal
|
||||
to the derived revision means the bytes are already published, so the job
|
||||
repoints the remaining tags at the existing manifest instead of rebuilding.
|
||||
Two things this depends on: an inspect that errors for ANY reason reads as a
|
||||
MISS so no needed build is ever skipped, and the repoint must EXCLUDE the
|
||||
source tag — `imagetools create` wraps its source in a manifest index, and
|
||||
config labels do not resolve through an index, so writing the channel tag
|
||||
from itself destroys the label the next run reads (#3183).
|
||||
- **The build pushes exactly ONE tag — the channel's — and every other tag is
|
||||
written registry-side afterwards** (#3190). buildx on this runner pushes the
|
||||
first tag to the registry and then re-pushes the rest through the docker
|
||||
driver, out of a local image store that a registry-direct build never fills;
|
||||
it fails intermittently with `tag does not exist`. On `dev` that only reddens
|
||||
a job, but on `main` it silently skips `:c-<sha>` while `:latest` publishes
|
||||
fine — a missing rollback tag has no consumer that fails, so nothing but the
|
||||
red job would notice until somebody needs to roll back. `imagetools create`
|
||||
has no local store to be absent from, and it is the code the reuse path
|
||||
already ran, so both paths now share one proven route. The cost: `:c-<sha>`
|
||||
is an index rather than a plain image, so `fc.revision` does not resolve
|
||||
through it — nothing reads it there, and the index names the same manifest.
|
||||
- **`FC_CHANNEL` and `FC_VERSION` are build args, not runtime settings.**
|
||||
`build.yml` passes them to the web image only — the ml and agent images have
|
||||
nothing to report them to. `/api/health` returns both, the foot of Settings
|
||||
renders them, and `/api/extension/manifest` reports the channel beside the
|
||||
extension version so an install can be traced to a channel. With no version image tags, that
|
||||
self-report is the ONLY answer to "which build is this?" — which is why a
|
||||
missing version renders `unknown` rather than a blank: an empty footer reads
|
||||
as "no version", a different and false claim.
|
||||
Both are declared LAST in the Dockerfile on purpose: an ARG invalidates every
|
||||
layer below it, and these are the values that differ between the dev and main
|
||||
builds of identical source, so placing them earlier would stop the two
|
||||
channels ever sharing a cached `pip install`. Empty by default — a local build
|
||||
then reports nothing rather than claiming a channel it is not on.
|
||||
- **The channel is never folded into the version.** A `-dev` suffix makes the
|
||||
extension's per-segment `parseInt` comparator read that segment as 0, so every
|
||||
dev build compares equal to every other — issue #2993 exactly (rule 149).
|
||||
`frontend/test/systemBuild.spec.js` pins the rendered version to the bare
|
||||
number.
|
||||
- Callers MUST `set -f` before substituting the script's output. Without it the
|
||||
shell expands `test/**` against the working tree and silently narrows the
|
||||
pattern to whatever files exist at that moment — a failure that looks like
|
||||
|
||||
+24
-9
@@ -38,27 +38,42 @@ npm run build # unsigned XPI in web-ext-artifacts/
|
||||
- [ ] Subscriptions list: popup → "Sources" tab → list renders
|
||||
- [ ] Check now: click play icon on source row → no error toast
|
||||
|
||||
## Versioning — don't hand-edit the patch number
|
||||
## Versioning — the committed number decides nothing
|
||||
|
||||
The shipped version is **derived**, not committed. `scripts/packaging.sh
|
||||
version` returns `MAJOR.MINOR` from `manifest.json` plus a patch component
|
||||
that is the commit *time* of the newest change to a packaged extension file,
|
||||
in minutes since 2020-01-01. `build.yml` computes it and stamps it into both
|
||||
version` returns `YYYY.M.D.HHMM` in UTC: the commit *time* of the newest change
|
||||
to a packaged extension file. `build.yml` computes it and stamps it into both
|
||||
`manifest.json` and `package.json` at build time. The stamp is never
|
||||
committed — the commit carrying it would itself be a change to the extension,
|
||||
which would move the version again.
|
||||
|
||||
So:
|
||||
|
||||
- **Editing the patch number does nothing.** It is overwritten before web-ext
|
||||
ever reads it. There is no bump to make, and none to forget.
|
||||
- **MAJOR.MINOR is still yours.** It carries the deliberate meaning, it is read
|
||||
from `manifest.json` alone, and CI fails the `extension-version` lane if the
|
||||
two files disagree on it.
|
||||
- **Editing the version does nothing.** All of it is overwritten before web-ext
|
||||
ever reads it. There is no bump to make, and none to forget. There is no
|
||||
hand-set part left either: MAJOR.MINOR went away with milestone 318 step 8.
|
||||
- `npm run build` locally produces an XPI labelled with the *committed*
|
||||
version, since nothing stamped it. Fine for loading into a test profile; not
|
||||
what ships.
|
||||
|
||||
**Why the extension is the one artifact that does not zero-pad.** Every other
|
||||
FC artifact emits rule 148's `YYYY.MM.DD.HHMM`. AMO will not take it: Mozilla's
|
||||
grammar for addons.mozilla.org is
|
||||
|
||||
```
|
||||
^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$
|
||||
```
|
||||
|
||||
— each segment is the single digit `0` or starts 1-9, so `08` and `0201` are
|
||||
rejected, and at most four segments are allowed. The extension therefore emits
|
||||
**the same numbers unpadded**: `2026.8.29.201` where the rest of the family
|
||||
says `2026.08.29.0201`. Rule 148 already defines comparison as numeric per
|
||||
segment, under which the two are equal, so nothing is reordered by the choice
|
||||
and left-padding each segment recovers the family string exactly. `ci.yml`'s
|
||||
`extension-version` lane checks the derived string against that regex on every
|
||||
push — the cheap place to find out, because AMO 409s on re-signing and a
|
||||
rejected version is burned for good.
|
||||
|
||||
Why commit time and not a commit count: a count is per-branch, so `dev` and
|
||||
`main` count different histories of the same code and their versions end up
|
||||
ordered by which branch accumulated more commits rather than by which is newer.
|
||||
|
||||
@@ -6,7 +6,9 @@
|
||||
# keeping three copies of one fact in sync by hand is how issue #2397 happened:
|
||||
#
|
||||
# 1. web-ext's --ignore-files (extension/package.json's four scripts)
|
||||
# 2. the :(exclude) pathspec (ci.yml's extension-version guard)
|
||||
# 2. the :(exclude) pathspec (what moves the version — a WIDER
|
||||
# set than the ignore list; see
|
||||
# NOT_VERSION_RELEVANT)
|
||||
# 3. the git-log pathspec (the derived version, below)
|
||||
#
|
||||
# They now all read from here. POSIX sh only — CI's run shell is busybox.
|
||||
@@ -35,8 +37,30 @@ set -euf
|
||||
NOT_PACKAGED_TRACKED='package.json package-lock.json README.md .gitignore vitest.config.js scripts scripts/** test test/**'
|
||||
NOT_PACKAGED_BUILD='web-ext-artifacts node_modules'
|
||||
|
||||
# Paths under extension/ that cannot change the SHIPPED BYTES, and so must not
|
||||
# move the derived version.
|
||||
#
|
||||
# Deliberately NOT the same list as NOT_PACKAGED_TRACKED, and the whole
|
||||
# difference is `scripts/`. packaging.sh is not packaged into the XPI — but it
|
||||
# DECIDES the version string, and build.yml stamps that string into the
|
||||
# manifest.json that is packaged. A change to how the version is computed is
|
||||
# therefore a change to the shipped bytes.
|
||||
#
|
||||
# Excluding it was harmless only while every push rebuilt the web image.
|
||||
# Milestone 313 step 4 made the rebuild conditional on the derived revision
|
||||
# moving, which turned it into a silent failure: a packaging.sh change gives a
|
||||
# NEW version, so sign-extension misses its ext-<version> cache and signs —
|
||||
# while build-web sees an unmoved revision, reuses the published image, and
|
||||
# ships the OLD XPI. An orphaned AMO signature, and an instance quietly serving
|
||||
# code the registry says is current.
|
||||
#
|
||||
# The two directions are not symmetric, which is why this list is the narrower
|
||||
# one. Too wide costs a re-sign and a rebuild for a change that ships nothing
|
||||
# new. Too narrow serves stale bytes and says nothing.
|
||||
NOT_VERSION_RELEVANT='package.json package-lock.json README.md .gitignore vitest.config.js test test/**'
|
||||
|
||||
usage() {
|
||||
echo "usage: packaging.sh {ignore|pathspec|version|major-minor|patch}" >&2
|
||||
echo "usage: packaging.sh {ignore|pathspec|version}" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
@@ -49,46 +73,61 @@ cmd_ignore() {
|
||||
echo "$NOT_PACKAGED_TRACKED $NOT_PACKAGED_BUILD"
|
||||
}
|
||||
|
||||
# git pathspec excluding the non-packaged tracked files, e.g.
|
||||
# :(exclude)extension/package.json :(exclude)extension/test/**
|
||||
# git pathspec excluding the tracked files that cannot change the shipped
|
||||
# bytes, e.g. :(exclude)extension/package.json :(exclude)extension/test/**
|
||||
#
|
||||
# This answers "what moves the version?", NOT "what goes in the XPI?" — see
|
||||
# NOT_VERSION_RELEVANT for why those differ. cmd_ignore answers the other one.
|
||||
# Same `set -f` requirement as above.
|
||||
cmd_pathspec() {
|
||||
for entry in $NOT_PACKAGED_TRACKED; do
|
||||
for entry in $NOT_VERSION_RELEVANT; do
|
||||
printf ':(exclude)extension/%s ' "$entry"
|
||||
done
|
||||
echo
|
||||
}
|
||||
|
||||
# MAJOR.MINOR stays hand-set in manifest.json — it's the part that carries
|
||||
# deliberate meaning. Only the patch component is derived.
|
||||
cmd_major_minor() {
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
grep -E '"version"' "$root/extension/manifest.json" \
|
||||
| head -1 \
|
||||
| sed -E 's/.*"version"[[:space:]]*:[[:space:]]*"([0-9]+)\.([0-9]+).*/\1.\2/'
|
||||
# Strip leading zeros from one segment, leaving at least one digit.
|
||||
#
|
||||
# This exists for AMO and nothing else. Mozilla's version grammar for
|
||||
# addons.mozilla.org is documented as
|
||||
#
|
||||
# ^(0|[1-9][0-9]{0,8})([.](0|[1-9][0-9]{0,8})){0,3}$
|
||||
#
|
||||
# — each segment is either the single digit `0` or starts 1-9, so `08` and
|
||||
# `0201` are rejected outright, while `0` itself is fine. MDN states it in
|
||||
# prose too: "Non-zero numbers must not include a leading zero."
|
||||
#
|
||||
# POSIX sh has no trim-loop, hence the while.
|
||||
unpad() {
|
||||
s=$1
|
||||
while [ "${#s}" -gt 1 ]; do
|
||||
case "$s" in
|
||||
0*) s=${s#0} ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
printf '%s' "$s"
|
||||
}
|
||||
|
||||
# 2020-01-01T00:00:00Z — the anchor for the derived patch component. Fixed
|
||||
# forever; moving it would renumber every version downwards.
|
||||
VERSION_EPOCH=1577836800
|
||||
|
||||
# Minutes since VERSION_EPOCH of the LATEST commit that touched a PACKAGED
|
||||
# extension file.
|
||||
# The extension's version: `YYYY.M.D.HHMM`, UTC, derived from the commit TIME
|
||||
# of the newest change to a PACKAGED extension file.
|
||||
#
|
||||
# Time-derived, per family rule 149: an artifact's ordering key must never be a
|
||||
# commit count. A count is per-branch — `dev` and `main` count different
|
||||
# histories of the same code — so the moment BOTH channels publish, their
|
||||
# versions order by which branch accumulated more commits rather than by which
|
||||
# is newer. A squash-merge makes that permanent: main gains one commit where dev
|
||||
# gained five, so dev climbs away from main and a dev install can never cross
|
||||
# back. That is Roundtable's 2026-08-24 incident (`versionCode` was the branch's
|
||||
# commit count) in a different repo. Measured here on 2026-08-27: main=23,
|
||||
# dev=24 under the old formula — one apart, which is exactly how the inversion
|
||||
# stays invisible until it strands somebody.
|
||||
# THE ONE DELIBERATE DEPARTURE FROM THE FAMILY SHAPE, and it is a rendering
|
||||
# difference only. Rule 148 says `YYYY.MM.DD.HHMM` zero-padded, and every other
|
||||
# FC artifact emits exactly that. AMO's grammar (see unpad) forbids the padding,
|
||||
# and AMO is not negotiable: a rejected version is burned, since AMO 409s on
|
||||
# re-signing a version it has already seen. So the extension emits THE SAME
|
||||
# NUMBERS unpadded — 2026.08.29.0201 and 2026.8.29.201 are one value in two
|
||||
# renderings, and rule 148 already specifies comparison as numeric per segment,
|
||||
# under which they are equal. Nothing published is reordered by the choice, and
|
||||
# left-padding each segment recovers the family string exactly.
|
||||
#
|
||||
# HHMM is one segment, not two, because AMO allows at most FOUR. Unpadded that
|
||||
# reads oddly (00:14 -> `14`, midnight -> `0`) but stays strictly increasing
|
||||
# within a day, which is all the ordering needs.
|
||||
#
|
||||
# Why the commit's time and not the build's:
|
||||
# * MONOTONIC — max() over a set that only ever gains members. Verified
|
||||
# across all 24 extension-touching commits: zero non-monotonic steps.
|
||||
# * MONOTONIC — max() over a set that only ever gains members.
|
||||
# * STABLE while the extension is unchanged, so an unchanged extension keeps
|
||||
# its version, the ext-<version> signature cache still hits, and AMO is
|
||||
# called once per extension CHANGE rather than once per push. Build-time
|
||||
@@ -99,32 +138,40 @@ VERSION_EPOCH=1577836800
|
||||
# produced for byte-identical code. Same code, same version, one signing.
|
||||
# * REPRODUCIBLE — any checkout of a commit yields that commit's version.
|
||||
#
|
||||
# Never a commit count (family rule 149): a count is per-branch, so `dev` and
|
||||
# `main` count different histories of the same code and order by which branch
|
||||
# accumulated more commits rather than by which is newer. A squash-merge makes
|
||||
# that permanent. Roundtable's 2026-08-24 incident, in a different repo.
|
||||
#
|
||||
# Requires real history: a depth-1 clone sees one commit and will derive a wrong
|
||||
# (too low) value. Every consumer must check out with fetch-depth: 0.
|
||||
cmd_patch() {
|
||||
#
|
||||
# Formatted through git rather than date(1): busybox date does not reliably
|
||||
# accept `-d @<epoch>`, and git's --date=format-local is available wherever git
|
||||
# is. TZ=UTC so the value does not depend on the runner's timezone.
|
||||
cmd_version() {
|
||||
root=$(git rev-parse --show-toplevel)
|
||||
# Unquoted on purpose: the pathspec must word-split into separate args.
|
||||
# Globbing is already off script-wide (set -euf above).
|
||||
# shellcheck disable=SC2046
|
||||
ts=$(cd "$root" && git log --format=%ct HEAD -- extension/ $(cmd_pathspec) \
|
||||
| sort -n | tail -1)
|
||||
if [ -z "$ts" ]; then
|
||||
sha=$(cd "$root" && git log --format='%ct %H' HEAD -- extension/ $(cmd_pathspec) \
|
||||
| sort -n | tail -1 | cut -d' ' -f2)
|
||||
if [ -z "$sha" ]; then
|
||||
echo "packaging.sh: no commit touches a packaged extension file" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo $(( (ts - VERSION_EPOCH) / 60 ))
|
||||
}
|
||||
|
||||
cmd_version() {
|
||||
echo "$(cmd_major_minor).$(cmd_patch)"
|
||||
padded=$(cd "$root" && TZ=UTC git show -s --format=%cd \
|
||||
--date='format-local:%Y.%m.%d.%H%M' "$sha")
|
||||
# Rebinding the function's own positional params, which are unused here.
|
||||
# shellcheck disable=SC2046
|
||||
set -- $(echo "$padded" | tr '.' ' ')
|
||||
echo "$(unpad "$1").$(unpad "$2").$(unpad "$3").$(unpad "$4")"
|
||||
}
|
||||
|
||||
[ $# -ge 1 ] || usage
|
||||
case "$1" in
|
||||
ignore) cmd_ignore ;;
|
||||
pathspec) cmd_pathspec ;;
|
||||
version) cmd_version ;;
|
||||
major-minor) cmd_major_minor ;;
|
||||
patch) cmd_patch ;;
|
||||
*) usage ;;
|
||||
ignore) cmd_ignore ;;
|
||||
pathspec) cmd_pathspec ;;
|
||||
version) cmd_version ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { readdirSync, readFileSync } from 'node:fs'
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import path from 'node:path'
|
||||
@@ -8,10 +8,10 @@ const EXT_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..')
|
||||
const read = (name) => JSON.parse(readFileSync(path.join(EXT_DIR, name), 'utf8'))
|
||||
const readText = (...seg) => readFileSync(path.join(EXT_DIR, ...seg), 'utf8')
|
||||
|
||||
// Only the git-free subcommands are exercised here: `version`/`patch` shell out
|
||||
// to git, and the extension lane runs on node:24-bookworm-slim which may not
|
||||
// ship it. Those two are covered where git is guaranteed — ci.yml and build.yml
|
||||
// run on ci-python.
|
||||
// Only the git-free subcommands are exercised here: `version` shells out to
|
||||
// git, and the extension lane runs on node:24-bookworm-slim which may not ship
|
||||
// it. That one is covered where git is guaranteed — ci.yml's extension-version
|
||||
// lane and build.yml both run on ci-python.
|
||||
const packaging = (cmd) =>
|
||||
execFileSync('sh', [path.join(EXT_DIR, 'scripts', 'packaging.sh'), cmd], {
|
||||
cwd: EXT_DIR,
|
||||
@@ -44,9 +44,39 @@ describe('packaging.sh — the single definition of what ships', () => {
|
||||
// covering anything added later.
|
||||
const pathspec = packaging('pathspec')
|
||||
expect(pathspec).toContain(':(exclude)extension/test/**')
|
||||
expect(pathspec).toContain(':(exclude)extension/scripts/**')
|
||||
expect(pathspec.some((e) => e.includes('.spec.js'))).toBe(false)
|
||||
expect(pathspec.some((e) => e.includes('helpers'))).toBe(false)
|
||||
|
||||
const ignore = packaging('ignore')
|
||||
expect(ignore).toContain('test/**')
|
||||
expect(ignore).toContain('scripts/**')
|
||||
expect(ignore.some((e) => e.includes('.spec.js'))).toBe(false)
|
||||
})
|
||||
|
||||
it('lets packaging.sh move the version, though it never ships in the XPI', () => {
|
||||
// The two lists answer different questions and this is the one place they
|
||||
// disagree. scripts/ is ignored by web-ext — it is repo tooling, not addon
|
||||
// code — but packaging.sh DECIDES the version string, and build.yml stamps
|
||||
// that string into the manifest.json that does ship. So changing how the
|
||||
// version is computed changes the shipped bytes.
|
||||
//
|
||||
// Excluding it from the pathspec was invisible while every push rebuilt the
|
||||
// web image. Milestone 313 step 4 made that rebuild conditional on the
|
||||
// derived revision moving, and the omission turned into a silent failure:
|
||||
// a new version means sign-extension misses its ext-<version> cache and
|
||||
// signs, while build-web sees an unmoved revision, reuses the published
|
||||
// image and ships the OLD XPI. An orphaned signature, and an instance
|
||||
// serving code the registry calls current.
|
||||
const pathspec = packaging('pathspec')
|
||||
expect(
|
||||
pathspec.some((e) => e.startsWith(':(exclude)extension/scripts')),
|
||||
'the pathspec excludes scripts/, so a change to how the version is '
|
||||
+ 'derived would not move the version it derives',
|
||||
).toBe(false)
|
||||
|
||||
// ...and it is still kept out of the package itself. Both must hold: the
|
||||
// tempting "fix" for either half is to make the two lists one again.
|
||||
expect(packaging('ignore')).toContain('scripts')
|
||||
})
|
||||
|
||||
it('keeps its own scripts and specs out of the XPI', () => {
|
||||
@@ -77,15 +107,24 @@ describe('consumers delegate rather than keeping their own copy', () => {
|
||||
}
|
||||
})
|
||||
|
||||
const WORKFLOWS = ['ci.yml', 'build.yml', 'extension.yml']
|
||||
// Read from disk rather than listed by hand. The point of this assertion is
|
||||
// that it survives consumers coming and going, and a hardcoded list is the
|
||||
// one part of it that cannot — release.yml (milestone 318 step 7) would have
|
||||
// joined the directory without joining the check.
|
||||
const WORKFLOWS = readdirSync(path.join(EXT_DIR, '..', '.forgejo', 'workflows')).filter((f) =>
|
||||
f.endsWith('.yml')
|
||||
)
|
||||
|
||||
it('no workflow hardcodes the packaged-file set', () => {
|
||||
// ci.yml used to substitute `packaging.sh pathspec` directly, for the
|
||||
// manual-bump guard that milestone 271 step 5 retired. Nothing inlines the
|
||||
// set today, and nothing should start to: a literal :(exclude)extension/...
|
||||
// in a workflow means someone bypassed the shared definition, which is
|
||||
// exactly the drift #2397 was about. Asserted across all three rather than
|
||||
// against one named consumer, so it keeps holding as consumers come and go.
|
||||
// exactly the drift #2397 was about.
|
||||
expect(
|
||||
WORKFLOWS.length,
|
||||
'no workflows found — the glob is not looking where it thinks'
|
||||
).toBeGreaterThan(2)
|
||||
for (const wf of WORKFLOWS) {
|
||||
const text = readText('..', '.forgejo', 'workflows', wf)
|
||||
expect(text, `${wf} inlines an :(exclude) literal`).not.toMatch(/:\(exclude\)extension\//)
|
||||
@@ -106,28 +145,33 @@ describe('consumers delegate rather than keeping their own copy', () => {
|
||||
})
|
||||
|
||||
describe('extension version', () => {
|
||||
const majorMinor = (v) => v.split('.').slice(0, 2).join('.')
|
||||
// Mozilla's published grammar for addons.mozilla.org, transcribed from MDN's
|
||||
// manifest.json/version page. Each segment is the single digit 0 or starts
|
||||
// 1-9 — so no leading zeros — and there are at most four of them.
|
||||
const AMO = /^(0|[1-9][0-9]{0,8})(\.(0|[1-9][0-9]{0,8})){0,3}$/
|
||||
|
||||
it('keeps the hand-set MAJOR.MINOR in lockstep across both files', () => {
|
||||
// Narrowed from full-string equality at milestone 271 step 5. Since step 4
|
||||
// the patch component is derived from commit time and stamped into both
|
||||
// files at build time, so the committed patch numbers are inert — nothing
|
||||
// reads them and they are not what ships. Asserting on them would fail for
|
||||
// a difference that changes nothing.
|
||||
it('keeps a committed version AMO would accept, though it ships nothing', () => {
|
||||
// The committed value is wholly inert since milestone 318 step 8: there is
|
||||
// no hand-set MAJOR.MINOR left for packaging.sh to read, and build.yml
|
||||
// stamps the derived string over both files before web-ext sees them.
|
||||
//
|
||||
// MAJOR.MINOR is the opposite: still hand-set, still shipped, and
|
||||
// packaging.sh reads it from manifest.json ALONE. Let the two diverge and
|
||||
// the extension ships a version package.json disagrees with, with no other
|
||||
// signal.
|
||||
expect(majorMinor(read('manifest.json').version))
|
||||
.toBe(majorMinor(read('package.json').version))
|
||||
// It is still asserted, for one reason: `npm run build` locally packages
|
||||
// whatever is committed, so a value AMO would reject turns a local build
|
||||
// into a confusing failure with no CI signal ahead of it. ci.yml checks
|
||||
// the same grammar against the DERIVED value, which is the one AMO sees.
|
||||
for (const file of ['manifest.json', 'package.json']) {
|
||||
expect(read(file).version, `${file} version is not AMO-shaped`).toMatch(AMO)
|
||||
}
|
||||
})
|
||||
|
||||
it('uses a plain dotted numeric version AMO will accept', () => {
|
||||
// The committed value seeds MAJOR.MINOR, so it still has to parse even
|
||||
// though its patch component never ships. ci.yml asserts the same shape on
|
||||
// the DERIVED value, which is the one AMO actually sees.
|
||||
expect(read('package.json').version).toMatch(/^\d+(\.\d+)*$/)
|
||||
it('rejects the zero-padded family shape, which is why the extension unpads', () => {
|
||||
// Guards the reason for the exception, not just its result. If this ever
|
||||
// starts passing, someone has loosened the pattern and the next sign burns
|
||||
// an AMO version to find out. (#3138.)
|
||||
expect('2026.08.29.0201').not.toMatch(AMO)
|
||||
expect('2026.8.29.201').toMatch(AMO)
|
||||
// Five segments: AMO allows four.
|
||||
expect('2026.8.29.2.1').not.toMatch(AMO)
|
||||
})
|
||||
|
||||
it('declares manifest v3', () => {
|
||||
|
||||
@@ -5,6 +5,18 @@ import { useApi } from '../composables/useApi.js'
|
||||
export const useSystemStore = defineStore('system', () => {
|
||||
const api = useApi()
|
||||
const healthy = ref(null) // null=unknown, true=ok, false=down
|
||||
// What the instance says it is. Since milestone 318 stopped publishing
|
||||
// version image tags, this is the only answer to "which build is this?" —
|
||||
// there is no registry name left to check it against.
|
||||
//
|
||||
// Three states, and collapsing any two of them would lie:
|
||||
// buildLoaded=false we have not asked yet -> render nothing
|
||||
// buildLoaded=true, version='' the build cannot say -> render "unknown"
|
||||
// buildLoaded=true, version=x this build is x
|
||||
// A blank footer would read as "no version", which is a different claim.
|
||||
const buildVersion = ref('')
|
||||
const buildChannel = ref('')
|
||||
const buildLoaded = ref(false)
|
||||
const stats = ref(null)
|
||||
const statsLoading = ref(false)
|
||||
|
||||
@@ -12,8 +24,17 @@ export const useSystemStore = defineStore('system', () => {
|
||||
try {
|
||||
const body = await api.get('/api/health')
|
||||
healthy.value = body.status === 'ok'
|
||||
// Absent means "cannot say" — the server omits these rather than
|
||||
// sending empty strings, so `?? ''` preserves that rather than
|
||||
// inventing a value for it.
|
||||
buildVersion.value = body.version ?? ''
|
||||
buildChannel.value = body.channel ?? ''
|
||||
buildLoaded.value = true
|
||||
} catch {
|
||||
healthy.value = false
|
||||
// Deliberately NOT setting buildLoaded: a failed health call tells us
|
||||
// nothing about the build, and claiming "unknown" would present a
|
||||
// network blip as a defective image.
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,5 +47,8 @@ export const useSystemStore = defineStore('system', () => {
|
||||
}
|
||||
}
|
||||
|
||||
return { healthy, stats, statsLoading, refreshHealth, refreshStats }
|
||||
return {
|
||||
healthy, stats, statsLoading, refreshHealth, refreshStats,
|
||||
buildVersion, buildChannel, buildLoaded,
|
||||
}
|
||||
})
|
||||
|
||||
@@ -54,6 +54,21 @@
|
||||
<MaintenancePanel />
|
||||
</v-window-item>
|
||||
</v-window>
|
||||
|
||||
<!-- Which build is this? With no version image tags (milestone 318) the
|
||||
instance's own report is the only answer, so it is shown rather than
|
||||
hidden. The instinct to treat it as information disclosure does not
|
||||
survive contact: the JS bundle and asset hashes fingerprint the build
|
||||
anyway, and "I'm on 2026.08.28.1249" is the single most useful line in
|
||||
a bug report.
|
||||
|
||||
Channel sits BESIDE the version, never inside it (rule 149) — a
|
||||
`-dev` suffix would read as a 0 segment to the extension's comparator
|
||||
and make every dev build compare equal (#2993). -->
|
||||
<div v-if="system.buildLoaded" class="text-caption text-medium-emphasis text-center mt-8">
|
||||
FabledCurator {{ system.buildVersion || 'unknown' }}
|
||||
<span v-if="system.buildChannel"> · {{ system.buildChannel }}</span>
|
||||
</div>
|
||||
</v-container>
|
||||
</template>
|
||||
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
|
||||
import { setActivePinia, createPinia } from 'pinia'
|
||||
import { useSystemStore } from '../src/stores/system.js'
|
||||
|
||||
// Which build am I running? Milestone 318 stopped publishing version image
|
||||
// tags, so the instance's own report is the ONLY answer — there is no registry
|
||||
// name left to check it against. That promotes this from a convenience to the
|
||||
// mechanism, and it means the three states below have to stay distinct: a
|
||||
// wrong answer here has nothing to contradict it.
|
||||
//
|
||||
// not asked yet -> render nothing
|
||||
// asked, no version -> render "unknown"
|
||||
// asked, has a version -> render it
|
||||
//
|
||||
// Collapsing the first two would show "unknown" during every page load, and
|
||||
// collapsing either into a blank would read as "no version", which is a
|
||||
// different and false claim.
|
||||
|
||||
function stubHealth(body, { fail = false } = {}) {
|
||||
globalThis.fetch = vi.fn(async () => {
|
||||
if (fail) throw new Error('network down')
|
||||
return {
|
||||
ok: true, status: 200, statusText: '200',
|
||||
text: async () => JSON.stringify(body),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
describe('system store — build identity', () => {
|
||||
beforeEach(() => setActivePinia(createPinia()))
|
||||
afterEach(() => { vi.restoreAllMocks(); delete globalThis.fetch })
|
||||
|
||||
it('starts having asked nothing, so the footer renders nothing', () => {
|
||||
const s = useSystemStore()
|
||||
expect(s.buildLoaded).toBe(false)
|
||||
})
|
||||
|
||||
it('reports the version and channel the instance claims', async () => {
|
||||
stubHealth({ status: 'ok', version: '2026.08.28.1249', channel: 'dev' })
|
||||
const s = useSystemStore()
|
||||
await s.refreshHealth()
|
||||
|
||||
expect(s.buildLoaded).toBe(true)
|
||||
expect(s.buildVersion).toBe('2026.08.28.1249')
|
||||
expect(s.buildChannel).toBe('dev')
|
||||
})
|
||||
|
||||
it('keeps the channel OUT of the version string', async () => {
|
||||
// The tempting shortcut is a `-dev` suffix. The extension's comparator
|
||||
// parses each dotted segment with parseInt, so a suffixed segment reads as
|
||||
// 0 and every dev build compares equal to every other — #2993 exactly
|
||||
// (rule 149). If anyone ever "simplifies" by folding them together, the
|
||||
// version stops being the bare derived number and this fails.
|
||||
stubHealth({ status: 'ok', version: '2026.08.28.1249', channel: 'dev' })
|
||||
const s = useSystemStore()
|
||||
await s.refreshHealth()
|
||||
|
||||
expect(s.buildVersion).toBe('2026.08.28.1249')
|
||||
expect(s.buildVersion).not.toContain('dev')
|
||||
})
|
||||
|
||||
it('treats an absent version as "cannot say", not as a value', async () => {
|
||||
// A locally-built image, or one predating the field. The server omits the
|
||||
// key rather than sending an empty string; `?? ''` must preserve that
|
||||
// rather than inventing something. The view renders "unknown" from it.
|
||||
stubHealth({ status: 'ok' })
|
||||
const s = useSystemStore()
|
||||
await s.refreshHealth()
|
||||
|
||||
expect(s.buildLoaded).toBe(true)
|
||||
expect(s.buildVersion).toBe('')
|
||||
expect(s.buildChannel).toBe('')
|
||||
})
|
||||
|
||||
it('reports a version with no channel without inventing one', async () => {
|
||||
stubHealth({ status: 'ok', version: '2026.08.28.1249' })
|
||||
const s = useSystemStore()
|
||||
await s.refreshHealth()
|
||||
|
||||
expect(s.buildVersion).toBe('2026.08.28.1249')
|
||||
expect(s.buildChannel).toBe('')
|
||||
})
|
||||
|
||||
it('does not claim "unknown" when the health call itself failed', async () => {
|
||||
// A network blip says nothing about the build. Marking it loaded here
|
||||
// would present a transient failure as a defective image — and since
|
||||
// nothing else names the build, there would be no second source to
|
||||
// correct the impression.
|
||||
stubHealth(null, { fail: true })
|
||||
const s = useSystemStore()
|
||||
await s.refreshHealth()
|
||||
|
||||
expect(s.healthy).toBe(false)
|
||||
expect(s.buildLoaded).toBe(false)
|
||||
})
|
||||
})
|
||||
Executable
+201
@@ -0,0 +1,201 @@
|
||||
#!/bin/sh
|
||||
# Single definition of WHAT EACH PUBLISHED ARTIFACT IS BUILT FROM, and the
|
||||
# version derived from it. Milestone 313; generalises the shape
|
||||
# extension/scripts/packaging.sh established for the extension alone.
|
||||
#
|
||||
# "Built from" is deliberately wider than "copied into". A file that DECIDES an
|
||||
# artifact's identity is part of what that artifact is built from even though it
|
||||
# never reaches the image — see DERIVER below, and #3156 for the same finding
|
||||
# about packaging.sh.
|
||||
#
|
||||
# Four artifacts, four independent versions. An artifact whose shipped files
|
||||
# did not change keeps its version and does not rebuild — that is the whole
|
||||
# point, and it is why each path set must match its Dockerfile rather than
|
||||
# being a plausible guess. Getting a set wrong is quiet in BOTH directions:
|
||||
#
|
||||
# too narrow -> a pin serves stale bytes, because the version did not move
|
||||
# when the content did. This is the dangerous one.
|
||||
# too wide -> the artifact re-versions and rebuilds for a change it does
|
||||
# not ship. Merely wasteful.
|
||||
#
|
||||
# tests/test_artifact_paths.py asserts every COPY source in each Dockerfile is
|
||||
# covered here, so adding a COPY without updating this file fails CI.
|
||||
#
|
||||
# POSIX sh only — CI's run shell is busybox on some paths.
|
||||
#
|
||||
# -f (no pathname expansion) is load-bearing for the whole script: the lists
|
||||
# below are iterated with deliberate word-splitting, and without it the shell
|
||||
# would glob `frontend/test/**` against the working tree and silently narrow
|
||||
# the pattern. Callers substituting the output need their own `set -f` too;
|
||||
# the two guards protect different expansions.
|
||||
set -euf
|
||||
|
||||
ROOT=$(git rev-parse --show-toplevel)
|
||||
|
||||
# --- what each artifact ships ------------------------------------------------
|
||||
#
|
||||
# Each set includes its own Dockerfile and requirements: changing a base image
|
||||
# or a pin changes the artifact just as surely as changing a source file.
|
||||
#
|
||||
# web (Dockerfile, context `.`) — the runtime stage copies backend/, alembic/,
|
||||
# alembic.ini, entrypoint.sh and requirements.txt; the frontend-builder stage
|
||||
# copies frontend/ and the runtime takes its `dist` output.
|
||||
#
|
||||
# frontend/test is excluded: `npm run build` is vite, which builds from src/,
|
||||
# index.html and public/ and never reads test/. It lands in the builder layer
|
||||
# but not in `dist`, so it cannot reach the shipped image.
|
||||
#
|
||||
# The web image ALSO bundles the signed XPI (build.yml downloads it into
|
||||
# frontend/public/extension/ before the docker build), so an extension change
|
||||
# changes the web image. The extension's packaged set is appended in cmd_paths
|
||||
# rather than restated — one definition, per #2397.
|
||||
WEB_PATHS='Dockerfile requirements.txt backend alembic alembic.ini entrypoint.sh frontend :(exclude)frontend/test :(exclude)frontend/test/**'
|
||||
|
||||
# ml (Dockerfile.ml, context `.`) — no frontend, no extension. Note it copies
|
||||
# BOTH requirements-ml.txt and requirements.txt.
|
||||
ML_PATHS='Dockerfile.ml requirements-ml.txt requirements.txt backend alembic alembic.ini entrypoint.sh'
|
||||
|
||||
# agent (agent/Dockerfile, context `agent`) — copies requirements.txt and
|
||||
# fc_agent only. agent/README.md, agent/docker-compose.yml and agent/ruff.toml
|
||||
# live in the directory but never reach the image, so they must not re-version
|
||||
# it: this is deliberately NOT `agent/`.
|
||||
AGENT_PATHS='agent/Dockerfile agent/requirements.txt agent/fc_agent'
|
||||
|
||||
# This file. It is copied into no image and it is still part of what the web
|
||||
# image is built from, because it DECIDES the FC_VERSION baked into that image
|
||||
# (#3202). Same finding as #3156 about packaging.sh, one level up.
|
||||
#
|
||||
# Why web and nothing else. Every artifact stamps `fc.revision`, but only web
|
||||
# also stamps a version (build.yml line ~488 feeds `version web` to the
|
||||
# FC_VERSION build arg; ml and agent ask for `revision` alone, and the
|
||||
# extension takes its version from packaging.sh). For a revision-only artifact
|
||||
# this file needs no entry: any change to how the revision is COMPUTED changes
|
||||
# the derived value, which then disagrees with the label on the published image
|
||||
# and forces a rebuild. That mechanism is self-correcting because it compares
|
||||
# against a string stamped into a real artifact.
|
||||
#
|
||||
# The version is compared against nothing, so it has no such backstop. Before
|
||||
# this entry, a change to cmd_version alone left every artifact's revision
|
||||
# untouched, the reuse check hit, the build was skipped, and the published
|
||||
# image went on reporting the OLD version format — silently, until some
|
||||
# unrelated commit happened to force a rebuild. Milestone 318 step 5 is the
|
||||
# worked instance: b3989d0 and 5771fd5 share revision fb2c4d5b80be while the
|
||||
# version moved 2026.8.28.1249 -> 2026.08.28.1249. It cost nothing only because
|
||||
# FC_VERSION did not exist until one commit later.
|
||||
#
|
||||
# Named as a file, not as `scripts`: release_notes.py lives beside it and only
|
||||
# READS derived values, so it decides nothing and must not re-version anything.
|
||||
# A future script that derives an identity belongs here explicitly.
|
||||
DERIVER='scripts/artifacts.sh'
|
||||
|
||||
|
||||
usage() {
|
||||
echo "usage: artifacts.sh {paths|revision|version} {web|ml|agent|extension}" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
# The extension's packaged set, read from its own definition rather than
|
||||
# copied. packaging.sh emits `:(exclude)extension/...` entries, so the bare
|
||||
# `extension` include has to come with them.
|
||||
ext_paths() {
|
||||
echo "extension $(sh "$ROOT/extension/scripts/packaging.sh" pathspec)"
|
||||
}
|
||||
|
||||
cmd_paths() {
|
||||
case "$1" in
|
||||
web) echo "$WEB_PATHS $DERIVER $(ext_paths)" ;;
|
||||
ml) echo "$ML_PATHS" ;;
|
||||
agent) echo "$AGENT_PATHS" ;;
|
||||
extension) ext_paths ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# "<unix ts> <sha>" of the newest commit touching this artifact's shipped set.
|
||||
# Unquoted on purpose: the pathspec must word-split into separate args.
|
||||
# Globbing is already off script-wide.
|
||||
newest() {
|
||||
# shellcheck disable=SC2046
|
||||
set -- "$(cd "$ROOT" && git log --format='%ct %H' HEAD -- $(cmd_paths "$1") \
|
||||
| sort -n | tail -1)"
|
||||
if [ -z "$1" ]; then
|
||||
echo "artifacts.sh: no commit touches this artifact's shipped files" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "$1"
|
||||
}
|
||||
|
||||
# Formatted through git rather than date(1): busybox date does not reliably
|
||||
# accept `-d @<epoch>`, and git's own --date=format-local is available wherever
|
||||
# git is. TZ=UTC so the value does not depend on the runner's timezone.
|
||||
fmt() {
|
||||
(cd "$ROOT" && TZ=UTC git show -s --format=%cd --date="format-local:$2" "$1")
|
||||
}
|
||||
|
||||
# The IDENTITY of an artifact's content: the commit its shipped files last
|
||||
# changed in. This is what decides whether a build can be skipped.
|
||||
#
|
||||
# It is published as the `fc.revision` LABEL on the image itself, and read
|
||||
# back off the moving channel tag — not as a tag of its own (milestone 318
|
||||
# step 3). A tag would be a name minted per build that only one thing reads,
|
||||
# which is what rule 145 narrowed against; it would also be prunable under the
|
||||
# registry's keep_pattern (#3157), so the cache would silently expire.
|
||||
#
|
||||
# A published image with no such label reads as a MISS and rebuilds. That is
|
||||
# the migration path, not a fault: `imagetools create` copies a manifest and
|
||||
# config labels are not manifest annotations, so the reuse path cannot stamp
|
||||
# one and there is nothing to backfill. Each artifact pays one rebuild, once.
|
||||
cmd_revision() {
|
||||
echo "$(newest "$1")" | cut -d' ' -f2 | cut -c1-12
|
||||
}
|
||||
|
||||
# The VERSION: `YYYY.MM.DD.HHMM`, zero-padded, UTC. One shape across the whole
|
||||
# family (note #3127 §1, rule 148) — the number an instance reports about
|
||||
# itself, and, with a `v` in front, the release tag naming the same build.
|
||||
#
|
||||
# Zero-padded since 2026-08-28. This stripped leading zeros until then, on the
|
||||
# reasoning that every segment should read as a plain integer — which never
|
||||
# held, since comparison strips them on parse anyway. Padding costs nothing,
|
||||
# sorts lexically as well as numerically, and keeps this project emitting the
|
||||
# same string as its siblings: unpadded, a `2026.8.28.1432` here sits beside a
|
||||
# `2026.08.28.1432` there, two shapes one character apart. Two obviously
|
||||
# different formats are safer than two nearly identical ones.
|
||||
#
|
||||
# Comparison is numeric per dot-segment, so `08` and `8` are equal and nothing
|
||||
# already published is reordered by the change.
|
||||
#
|
||||
# HHMM is not decoration: it is what makes the value unique per build with no
|
||||
# lookup. A date alone collides on the second build of a day, and resolving
|
||||
# that needs a `.N` suffix, which needs asking the registry what already
|
||||
# exists — at which point two lanes derive different answers for one source
|
||||
# and the shared-signature property is lost.
|
||||
cmd_version() {
|
||||
# The extension is the one artifact this script does not FORMAT, only route.
|
||||
# AMO's version grammar forbids leading zeros, so the extension emits the
|
||||
# same numbers unpadded (#3138) — a rendering exception, documented in
|
||||
# packaging.sh beside the signing step that has to obey it. Delegating keeps
|
||||
# one answer per artifact: `artifacts.sh version extension` and
|
||||
# `packaging.sh version` cannot drift into two.
|
||||
#
|
||||
# The direction is deliberate. artifacts.sh already asks packaging.sh for the
|
||||
# extension's PATH SET (ext_paths above), so the version has to flow the same
|
||||
# way; reversing it would have packaging.sh call back into this script, which
|
||||
# would call packaging.sh for the paths again.
|
||||
if [ "$1" = extension ]; then
|
||||
sh "$ROOT/extension/scripts/packaging.sh" version
|
||||
return
|
||||
fi
|
||||
sha=$(echo "$(newest "$1")" | cut -d' ' -f2)
|
||||
# One git call for the whole string rather than four and a sed. git's
|
||||
# format-local takes the complete format, and doing it in pieces was only
|
||||
# ever there to strip the padding between them.
|
||||
fmt "$sha" '%Y.%m.%d.%H%M'
|
||||
}
|
||||
|
||||
[ $# -ge 2 ] || usage
|
||||
case "$1" in
|
||||
paths) cmd_paths "$2" ;;
|
||||
revision) cmd_revision "$2" ;;
|
||||
version) cmd_version "$2" ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
@@ -0,0 +1,312 @@
|
||||
"""Publish a Forgejo release whose body is derived from git, not written by hand.
|
||||
|
||||
Milestone 318 step 2 took the build consequence away from a `v*` tag: `main`
|
||||
has already built and published the commit by the time anyone tags it, and
|
||||
rebuilding would re-push `:c-<sha>`, which rule 145 forbids even when the bytes
|
||||
match. That left the tag with nothing to do. This gives it the job it has left.
|
||||
|
||||
**The half of the question a version string cannot answer.** Step 6 puts
|
||||
`2026.08.28.2208` in the Settings footer, so an operator can say which build
|
||||
they are running. They still cannot say what is in it that was not in the one
|
||||
they ran last month. A dated release carrying the commits since the previous
|
||||
one is the object that interprets the identifier (note #3127 §5).
|
||||
|
||||
**Derived, so it cannot drift.** The alternative is a hand-maintained
|
||||
`CHANGELOG.md`, which goes aspirational the first time someone forgets — and
|
||||
nothing ever catches it, because there is no second source to disagree with.
|
||||
Every line below comes out of `git log` at publish time.
|
||||
|
||||
**Optional by construction.** Release tags are bookmarks: cut one when you will
|
||||
want to point at that day by name, otherwise don't. FC went twelve weeks
|
||||
without one and nothing was wrong (note #3127 §0). This runs on a tag push and
|
||||
on nothing else — deliberately no schedule and no auto-tag on merge, either of
|
||||
which would turn an optional bookmark back into ceremony.
|
||||
|
||||
## Finding the previous release
|
||||
|
||||
`git describe --exclude <this tag>`, which walks ANCESTRY, not a sorted list.
|
||||
That is not fussiness: this repo's existing tags are the old `v26.05.22.0`
|
||||
shape and the next one will be rule 148's `v2026.08.28.2208`. Lexicographically
|
||||
`v2026...` sorts BEFORE `v26...` — every release from here on would report its
|
||||
predecessor as itself-or-nothing and emit a changelog covering the entire
|
||||
history. Ancestry is immune to the shape change, and it is also the more honest
|
||||
question: "what is in this that was not in the last one" IS a reachability
|
||||
question.
|
||||
|
||||
## Re-runs update, they do not fall through
|
||||
|
||||
Note #3127 §6.7: a publisher that POSTs and recovers the id from a `409` never
|
||||
rewrites the body, so a re-run silently keeps the first version. Harmless for a
|
||||
`v*` tag created once — and wrong the moment anything re-points. This one GETs
|
||||
first and PATCHes when the release exists, so it is correct either way rather
|
||||
than correct by luck (ThoughtSync #2182 is the same bug).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
API = "https://git.fabledsword.com/api/v1/repos/bvandeusen/FabledCurator"
|
||||
|
||||
IMAGES = (
|
||||
"git.fabledsword.com/bvandeusen/fabledcurator",
|
||||
"git.fabledsword.com/bvandeusen/fabledcurator-ml",
|
||||
"git.fabledsword.com/bvandeusen/fabledcurator-agent",
|
||||
)
|
||||
|
||||
# Rule 148: `v` + the artifact's own version, zero-padded, no `.N`, no lookup.
|
||||
RULE_148 = re.compile(r"^v\d{4}\.\d{2}\.\d{2}\.\d{4}$")
|
||||
|
||||
# Past this, the list has stopped being something anyone reads. It is reached
|
||||
# in exactly one situation — no previous tag is reachable, so the span is the
|
||||
# whole history — which happens on a genuine first release and on a tag cut
|
||||
# somewhere `main`'s tags cannot be seen from. Truncating says so; emitting
|
||||
# 1100 lines would bury the note explaining why there are 1100 of them.
|
||||
MAX_COMMITS = 200
|
||||
|
||||
|
||||
def git(*args: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", *args], capture_output=True, text=True, check=True
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def git_ok(*args: str) -> str | None:
|
||||
"""Run git, returning None instead of raising when it fails.
|
||||
|
||||
Used for the questions that legitimately have no answer — no previous tag,
|
||||
no local `main` — where the absence is information rather than a fault.
|
||||
"""
|
||||
try:
|
||||
return git(*args)
|
||||
except subprocess.CalledProcessError:
|
||||
return None
|
||||
|
||||
|
||||
def previous_tag(ref: str, tag: str | None) -> str | None:
|
||||
"""The most recent `v*` tag reachable from `ref`, excluding `tag` itself.
|
||||
|
||||
`--exclude` rather than `<ref>^` so this is the same call whether or not
|
||||
`ref` is the tag being released — and so it does not blow up on a root
|
||||
commit that has no parent to walk to.
|
||||
"""
|
||||
args = ["describe", "--tags", "--abbrev=0", "--match", "v*"]
|
||||
if tag:
|
||||
args += ["--exclude", tag]
|
||||
return git_ok(*args, ref)
|
||||
|
||||
|
||||
def commits(previous: str | None, ref: str) -> list[str]:
|
||||
"""The subjects between the previous release and this one.
|
||||
|
||||
`--no-merges` because rule 153 merges `dev` into `main` with a plain merge
|
||||
commit, so `main`'s first-parent view is a list of "Merge pull request #N"
|
||||
and nothing else. The work is in the commits under those merges.
|
||||
"""
|
||||
span = f"{previous}..{ref}" if previous else ref
|
||||
out = git("log", "--no-merges", "--format=%s (%h)", span)
|
||||
return [line for line in out.split("\n") if line.strip()]
|
||||
|
||||
|
||||
def truncate(log: list[str]) -> tuple[list[str], str | None]:
|
||||
if len(log) <= MAX_COMMITS:
|
||||
return log, None
|
||||
return log[:MAX_COMMITS], (
|
||||
f"{len(log)} commits in this span — more than a changelog is for. "
|
||||
f"Listing the newest {MAX_COMMITS}. This usually means no previous "
|
||||
f"`v*` tag was reachable from here."
|
||||
)
|
||||
|
||||
|
||||
def render(tag: str, sha: str, previous: str | None, log: list[str], notes: list[str]) -> str:
|
||||
short = sha[:7]
|
||||
parts = []
|
||||
|
||||
if notes:
|
||||
# Anything the derivation could not stand behind goes at the TOP, not
|
||||
# in a footnote. A release that quietly names a build nobody can find
|
||||
# is the failure this whole milestone is about.
|
||||
parts.append("\n".join(f"> **Note:** {n}" for n in notes))
|
||||
|
||||
parts.append(
|
||||
f"Built from `{short}`. The rollback unit is the immutable `:c-` tag "
|
||||
f"(rule 145) — these three move together:\n\n```\n"
|
||||
+ "\n".join(f"{image}:c-{short}" for image in IMAGES)
|
||||
+ "\n```"
|
||||
)
|
||||
|
||||
heading = f"## Changes since {previous}" if previous else "## Changes"
|
||||
if log:
|
||||
parts.append(heading + "\n\n" + "\n".join(f"- {line}" for line in log))
|
||||
else:
|
||||
parts.append(
|
||||
heading
|
||||
+ "\n\n_No non-merge commits since the previous release. This tag "
|
||||
"names the same source under a new name._"
|
||||
)
|
||||
|
||||
span = f"{previous}..{tag}" if previous else tag
|
||||
parts.append(
|
||||
f"---\n\n_Derived at publish time from `git log --no-merges {span}`. "
|
||||
f"Nothing here is hand-maintained._"
|
||||
)
|
||||
return "\n\n".join(parts)
|
||||
|
||||
|
||||
def cross_checks(tag: str, sha: str) -> list[str]:
|
||||
"""Everything the derivation knows that would make the release a lie.
|
||||
|
||||
Reported rather than enforced. The tag is already pushed by the time this
|
||||
runs, so failing here would leave the operator with a tag and no release
|
||||
and nothing but a red lane to explain it — while the release itself is
|
||||
still the useful object. Say what is wrong, on the release, and publish.
|
||||
"""
|
||||
notes = []
|
||||
|
||||
if not RULE_148.match(tag):
|
||||
notes.append(
|
||||
f"`{tag}` is not rule 148's `vYYYY.MM.DD.HHMM` shape. Published "
|
||||
f"anyway — the old `v26.*` tags predate the rule."
|
||||
)
|
||||
else:
|
||||
derived = artifact_version("web")
|
||||
if derived and derived != tag[1:]:
|
||||
notes.append(
|
||||
f"This tag names `{tag[1:]}`, but the web image built from "
|
||||
f"`{sha[:7]}` reports `{derived}`. The Settings footer will not "
|
||||
f"match this release's name."
|
||||
)
|
||||
|
||||
# `:c-<sha>` only exists if `main` built this commit. Checking costs one
|
||||
# git call; claiming it without checking costs a rollback that 404s at the
|
||||
# moment someone needs it.
|
||||
main = git_ok("rev-parse", "--verify", "-q", "refs/remotes/origin/main")
|
||||
if main is None:
|
||||
notes.append(
|
||||
"Could not resolve `origin/main` here, so the `:c-` tags above are "
|
||||
"unverified — they exist only if `main` built this commit."
|
||||
)
|
||||
elif subprocess.run(
|
||||
["git", "merge-base", "--is-ancestor", sha, main], capture_output=True
|
||||
).returncode != 0:
|
||||
notes.append(
|
||||
f"`{sha[:7]}` is not on `main`, so no `:c-{sha[:7]}` images were "
|
||||
f"ever published. The refs above will not pull."
|
||||
)
|
||||
|
||||
return notes
|
||||
|
||||
|
||||
def artifact_version(artifact: str) -> str | None:
|
||||
"""What `artifacts.sh` derives for one artifact in the CURRENT checkout.
|
||||
|
||||
It takes no ref because `artifacts.sh` takes none — it walks history from
|
||||
HEAD. That is right here only because a tag push checks out the tagged
|
||||
commit; calling this after `--dry-run some-other-ref` would compare the
|
||||
tag against the working tree, which is why the mismatch note below is
|
||||
reported and not enforced.
|
||||
|
||||
Returns None rather than raising if the script is missing or unhappy: a
|
||||
cross-check that cannot run should not take the release down with it.
|
||||
"""
|
||||
root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
try:
|
||||
return subprocess.run(
|
||||
["sh", os.path.join(root, "scripts", "artifacts.sh"), "version", artifact],
|
||||
capture_output=True, text=True, check=True, cwd=root,
|
||||
).stdout.strip()
|
||||
except (subprocess.CalledProcessError, OSError):
|
||||
return None
|
||||
|
||||
|
||||
def api(method: str, path: str, token: str, payload: dict | None = None) -> dict | None:
|
||||
body = json.dumps(payload).encode() if payload is not None else None
|
||||
req = urllib.request.Request(
|
||||
API + path, data=body, method=method,
|
||||
headers={
|
||||
"Authorization": "token " + token,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 404:
|
||||
return None
|
||||
sys.exit(f"release: {method} {path} failed with HTTP {exc.code}: {exc.read()!r}")
|
||||
|
||||
|
||||
def publish(tag: str, name: str, body: str, token: str) -> None:
|
||||
existing = api("GET", f"/releases/tags/{tag}", token)
|
||||
if existing:
|
||||
api("PATCH", f"/releases/{existing['id']}", token, {"name": name, "body": body})
|
||||
print(f"release: updated existing release {existing['id']} for {tag}")
|
||||
else:
|
||||
api("POST", "/releases", token, {"tag_name": tag, "name": name, "body": body})
|
||||
print(f"release: created release for {tag}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument(
|
||||
"ref", nargs="?", default=None,
|
||||
help="tag or commit to release. Defaults to GITHUB_REF's tag, else HEAD.",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--dry-run", action="store_true",
|
||||
help="render the body to stdout and publish nothing. Needs no token, "
|
||||
"so it also works as a preview before you decide to cut the tag.",
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
github_ref = os.environ.get("GITHUB_REF", "")
|
||||
if args.ref:
|
||||
ref = args.ref
|
||||
elif github_ref.startswith("refs/tags/"):
|
||||
ref = github_ref[len("refs/tags/"):]
|
||||
else:
|
||||
ref = "HEAD"
|
||||
|
||||
# A tag only if git knows it as one — `HEAD` and a raw sha are refs to
|
||||
# release FROM, never the name to exclude or to publish under.
|
||||
tag = ref if git_ok("rev-parse", "--verify", "-q", f"refs/tags/{ref}") else None
|
||||
sha = git("rev-parse", ref)
|
||||
previous = previous_tag(ref, tag)
|
||||
|
||||
print(f"release: ref={ref} sha={sha[:12]} previous={previous or '<none>'}")
|
||||
|
||||
notes = cross_checks(tag, sha) if tag else [
|
||||
f"Rendered for `{ref}`, which is not a tag. Nothing was published."
|
||||
]
|
||||
for note in notes:
|
||||
print(f"release: NOTE {note}")
|
||||
|
||||
log = commits(previous, ref)
|
||||
print(f"release: {len(log)} non-merge commits in the span")
|
||||
log, overflow = truncate(log)
|
||||
if overflow:
|
||||
print(f"release: NOTE {overflow}")
|
||||
notes.append(overflow)
|
||||
body = render(tag or ref, sha, previous, log, notes)
|
||||
|
||||
if args.dry_run or not tag:
|
||||
print("--- body ---")
|
||||
print(body)
|
||||
return
|
||||
|
||||
token = os.environ.get("RELEASE_TOKEN") or os.environ.get("TOKEN")
|
||||
if not token:
|
||||
sys.exit("release: no RELEASE_TOKEN in the environment")
|
||||
publish(tag, f"FabledCurator {tag[1:]}", body, token)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,166 @@
|
||||
"""The two values `artifacts.sh` derives, and what each of them promises.
|
||||
|
||||
`revision` decides whether a build gets skipped; `version` is what an instance
|
||||
reports about itself and what a release tag is named after. Neither has a
|
||||
consumer that would notice it going subtly wrong.
|
||||
|
||||
## revision
|
||||
|
||||
Milestone 318 step 3: each image carries its revision as an `fc.revision`
|
||||
label, and build.yml reads that label back off the moving channel tag. Equal
|
||||
to the derived revision means the bytes this push would produce are already
|
||||
published, so the build is skipped.
|
||||
|
||||
That makes the revision load-bearing in a way a version string is not — it is
|
||||
compared for equality against a value stamped into a real published artifact.
|
||||
Both ways of getting it wrong are silent:
|
||||
|
||||
* **it does not identify the content** — a revision that moves when the source
|
||||
did not (a HEAD-derived value, say) never matches, nothing is ever skipped,
|
||||
and the mechanism quietly buys nothing while every lane stays green.
|
||||
* **it identifies the wrong content** — a revision that holds still when the
|
||||
source DID change matches a stale label, the build is skipped, and the
|
||||
channel serves bytes that do not correspond to the commit. This is the
|
||||
dangerous direction, and it is what `test_artifact_paths.py` guards from the
|
||||
other side by pinning the path sets.
|
||||
|
||||
This module owns the narrower claim: whatever the path sets say, the revision
|
||||
is genuinely the commit those paths last changed in.
|
||||
|
||||
## version
|
||||
|
||||
`YYYY.MM.DD.HHMM`, zero-padded, UTC — one shape across the family (note #3127
|
||||
§1, rule 148), so the string this project emits is the same string its siblings
|
||||
emit. Two nearly-identical formats are more dangerous than two obviously
|
||||
different ones, and the only thing keeping them identical is a test.
|
||||
|
||||
The identity-TAG tests this file used to hold are gone with the tag. There is
|
||||
no longer a `CHANNELLED` list to drift (the channel is which tag you inspect),
|
||||
and no `identity` subcommand to refuse an unqualified call.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
ARTIFACTS = ("web", "ml", "agent", "extension")
|
||||
|
||||
# 12 hex chars — the prefix build.yml stamps and compares.
|
||||
_REVISION = re.compile(r"^[0-9a-f]{12}$")
|
||||
|
||||
# YYYY.MM.DD.HHMM, every segment zero-padded to its full width.
|
||||
_VERSION = re.compile(r"^\d{4}\.\d{2}\.\d{2}\.\d{4}$")
|
||||
|
||||
|
||||
# Everything here goes through artifacts.sh rather than importing a sibling
|
||||
# test module. That is the interface build.yml actually calls, so the tests
|
||||
# exercise the contract instead of a Python re-implementation of it — and no
|
||||
# other test module in this repo imports another, so a cross-test import would
|
||||
# be a new convention introduced for no gain.
|
||||
def artifacts(*args: str) -> str:
|
||||
return subprocess.run(
|
||||
["sh", str(ROOT / "scripts" / "artifacts.sh"), *args],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout
|
||||
|
||||
|
||||
def revision(artifact: str) -> str:
|
||||
return artifacts("revision", artifact).strip()
|
||||
|
||||
|
||||
def newest_by_commit_time(artifact: str) -> str:
|
||||
"""The full SHA of the newest commit touching this artifact's shipped set.
|
||||
|
||||
Ordered by committer TIME, matching what artifacts.sh means. Deliberately
|
||||
not `git log -1`: git's default order is reverse-chronological only within
|
||||
topological constraints, so on a merged history it can name a different
|
||||
commit than the newest timestamp does. They agree on this repo today, and
|
||||
a test that silently depends on them continuing to agree would be a flake
|
||||
waiting for the branch shape that separates them.
|
||||
"""
|
||||
paths = artifacts("paths", artifact).split()
|
||||
log = subprocess.run(
|
||||
["git", "log", "--format=%ct %H", "HEAD", "--", *paths],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout.split("\n")
|
||||
commits = [line.split(" ", 1) for line in log if line.strip()]
|
||||
assert commits, (
|
||||
f"no commit in this history touches the {artifact} path set — the "
|
||||
f"derivation has nothing to stand on"
|
||||
)
|
||||
return max(commits, key=lambda c: int(c[0]))[1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_revision_is_the_commit_its_own_shipped_files_last_changed_in(artifact):
|
||||
"""The claim the whole skip decision rests on.
|
||||
|
||||
Computed from git rather than asked of the script, so it fails if the
|
||||
derivation ever stops meaning what it says — switching to HEAD, to a build
|
||||
clock, or to a path set it did not actually use. Each of those still
|
||||
produces a plausible 12-hex value, which is why this is worth asserting
|
||||
rather than eyeballing.
|
||||
"""
|
||||
expected = newest_by_commit_time(artifact)
|
||||
got = revision(artifact)
|
||||
assert expected.startswith(got), (
|
||||
f"{artifact} derives {got!r}, but the newest commit touching its "
|
||||
f"shipped files is {expected[:12]!r}. The label stamped into the image "
|
||||
f"would not identify its own content."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_revision_is_a_legal_label_value_and_is_stable(artifact):
|
||||
"""It is stamped as a docker label and compared for string equality, so a
|
||||
stray newline or a varying value breaks the comparison rather than the
|
||||
build — the mechanism would simply stop hitting, silently."""
|
||||
first = revision(artifact)
|
||||
assert _REVISION.match(first), f"{first!r} is not a 12-char hex revision"
|
||||
assert first == revision(artifact), "revision is not stable across calls"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_version_is_zero_padded_calver(artifact):
|
||||
"""The family shape, pinned.
|
||||
|
||||
Padding was stripped until 2026-08-28 on the reasoning that each segment
|
||||
should read as a plain integer — which never held, since comparison strips
|
||||
leading zeros on parse anyway. What it did do was make this project emit
|
||||
`2026.8.28.1432` while a sibling emitted `2026.08.28.1432`: two shapes one
|
||||
character apart, which is the hard kind of difference to notice.
|
||||
|
||||
Also catches the midnight case. A `%H%M` of `0322` must survive as `0322`;
|
||||
the old strip-leading-zeros helper turned it into `322`, silently changing
|
||||
a four-digit field into three.
|
||||
"""
|
||||
value = artifacts("version", artifact).strip()
|
||||
assert _VERSION.match(value), (
|
||||
f"{artifact} derives {value!r}, which is not zero-padded "
|
||||
f"YYYY.MM.DD.HHMM. Note #3127 §1 and rule 148 both specify the padded "
|
||||
f"form, and a release tag is this string with a `v` in front."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", ARTIFACTS)
|
||||
def test_version_and_revision_describe_the_same_commit(artifact):
|
||||
"""They are derived independently and must not be able to disagree.
|
||||
|
||||
A build reports the version and skips on the revision, so a divergence
|
||||
would mean an instance naming one commit while carrying another's bytes —
|
||||
unfalsifiable from outside, since both values look perfectly well-formed.
|
||||
"""
|
||||
sha = newest_by_commit_time(artifact)
|
||||
stamped = subprocess.run(
|
||||
["git", "show", "-s", "--format=%cd", "--date=format-local:%Y.%m.%d.%H%M", sha],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
env={"TZ": "UTC", "PATH": os.environ.get("PATH", "")},
|
||||
).stdout.strip()
|
||||
assert artifacts("version", artifact).strip() == stamped
|
||||
assert sha.startswith(revision(artifact))
|
||||
@@ -0,0 +1,199 @@
|
||||
"""`scripts/artifacts.sh` path sets must match what the Dockerfiles copy.
|
||||
|
||||
Each published artifact's version derives from the newest commit touching its
|
||||
own shipped file set (milestone 313). The whole scheme rests on those sets
|
||||
being right, and both ways of being wrong are silent:
|
||||
|
||||
* **too narrow** — a file ships but is not in the set, so the version does not
|
||||
move when the content does, and a pin serves stale bytes. This is the
|
||||
dangerous direction and the one this module exists for.
|
||||
* **too wide** — a file is in the set but never reaches the image, so the
|
||||
artifact re-versions and rebuilds for a change it does not ship.
|
||||
|
||||
Nothing else notices either. The version still derives, CI still goes green,
|
||||
and the mismatch only surfaces as "I pinned that build and got the wrong
|
||||
bytes". So the Dockerfiles are read here and compared against the declaration.
|
||||
|
||||
The COPY list is not the whole answer, though. A file that DECIDES what an
|
||||
artifact reports belongs in its set even though it is copied into nothing —
|
||||
see DERIVERS below, where the same finding is recorded twice (#3156, #3202).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
# artifact -> (dockerfile, build context relative to the repo root)
|
||||
ARTIFACTS = {
|
||||
"web": ("Dockerfile", ""),
|
||||
"ml": ("Dockerfile.ml", ""),
|
||||
"agent": ("agent/Dockerfile", "agent"),
|
||||
}
|
||||
|
||||
# COPY --from=<stage> copies from an earlier build stage, not from the build
|
||||
# context, so its source is not a repo path and cannot be in a path set.
|
||||
_COPY = re.compile(r"^\s*COPY\s+(?!--from=)(?P<args>.+)$", re.MULTILINE)
|
||||
|
||||
|
||||
def declared_paths(artifact: str) -> list[str]:
|
||||
out = subprocess.run(
|
||||
["sh", str(ROOT / "scripts" / "artifacts.sh"), "paths", artifact],
|
||||
capture_output=True, text=True, check=True, cwd=ROOT,
|
||||
).stdout
|
||||
return out.split()
|
||||
|
||||
|
||||
def includes(artifact: str) -> list[str]:
|
||||
"""The set minus its `:(exclude)…` entries."""
|
||||
return [p for p in declared_paths(artifact) if not p.startswith(":(exclude)")]
|
||||
|
||||
|
||||
def copy_sources(dockerfile: str, context: str) -> list[str]:
|
||||
"""Repo-relative sources of every context COPY in a Dockerfile."""
|
||||
text = (ROOT / dockerfile).read_text()
|
||||
sources: list[str] = []
|
||||
for m in _COPY.finditer(text):
|
||||
args = m.group("args").split()
|
||||
# Last arg is the destination; everything before it is a source.
|
||||
for src in args[:-1]:
|
||||
# `frontend/package-lock.json*` — the glob is an optional-file
|
||||
# idiom; the directory it sits in is what matters for coverage.
|
||||
src = src.rstrip("*")
|
||||
sources.append(f"{context}/{src}" if context else src)
|
||||
return sources
|
||||
|
||||
|
||||
def covered_by(path: str, include: str) -> bool:
|
||||
"""`path` ships if an include names it or one of its ancestors."""
|
||||
path = path.rstrip("/").lstrip("./")
|
||||
include = include.rstrip("/")
|
||||
return path == include or path.startswith(include + "/")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", sorted(ARTIFACTS))
|
||||
def test_every_copied_path_is_in_the_artifacts_path_set(artifact):
|
||||
"""The too-narrow direction — the one that serves stale bytes on a pin."""
|
||||
dockerfile, context = ARTIFACTS[artifact]
|
||||
inc = includes(artifact)
|
||||
for src in copy_sources(dockerfile, context):
|
||||
assert any(covered_by(src, i) for i in inc), (
|
||||
f"{dockerfile} copies {src!r} into the {artifact} image, but no "
|
||||
f"include in scripts/artifacts.sh covers it. The {artifact} "
|
||||
f"version will not move when that file changes, so a pinned build "
|
||||
f"will serve stale bytes. Add it to the path set.\n"
|
||||
f" declared includes: {inc}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("artifact", sorted(ARTIFACTS))
|
||||
def test_the_dockerfile_itself_is_in_the_path_set(artifact):
|
||||
"""Changing a base image or a RUN changes the artifact as surely as
|
||||
changing a source file, so each set must include its own Dockerfile."""
|
||||
dockerfile, _ = ARTIFACTS[artifact]
|
||||
assert any(covered_by(dockerfile, i) for i in includes(artifact)), (
|
||||
f"{dockerfile} is not in the {artifact} path set — a base-image bump "
|
||||
f"would not move the version."
|
||||
)
|
||||
|
||||
|
||||
def test_the_web_image_versions_on_an_extension_change():
|
||||
"""The web image bundles the signed XPI, so the extension's packaged files
|
||||
are part of what it ships. Miss this and `:latest` serves a NEW extension
|
||||
under an unchanged web version — a pin that quietly disagrees with itself.
|
||||
"""
|
||||
inc = includes("web")
|
||||
assert any(covered_by("extension/background/background.js", i) for i in inc), (
|
||||
"the web path set does not cover the extension's packaged files, but "
|
||||
"build.yml downloads the signed XPI into frontend/public/extension/ "
|
||||
"before the docker build"
|
||||
)
|
||||
|
||||
|
||||
# A file that DECIDES an artifact's identity is part of what that artifact is
|
||||
# built from, even though it is copied into no image. Both entries here are the
|
||||
# same finding twice — #3156 for packaging.sh, #3202 for artifacts.sh — and
|
||||
# both were latent for the same reason: the version has no backstop.
|
||||
#
|
||||
# The revision does. Change how a REVISION is computed and the derived value
|
||||
# stops matching the label on the published image, which forces a rebuild; the
|
||||
# mechanism self-corrects because it compares against a string stamped into a
|
||||
# real artifact. Nothing compares a version to anything, so a version-only
|
||||
# derivation change is invisible unless the deriver is in the set.
|
||||
DERIVERS = [
|
||||
# packaging.sh decides the version build.yml stamps into the packaged
|
||||
# manifest.json, so changing it changes the shipped bytes. Left out,
|
||||
# milestone 313 step 4 turns silent: the new version misses the
|
||||
# ext-<version> cache and gets signed, while web's revision has not moved,
|
||||
# so the reuse path republishes the old image and the fresh signature is
|
||||
# orphaned. Guarded for web too, since web bundles what the extension makes.
|
||||
("extension/scripts/packaging.sh", ("extension", "web")),
|
||||
# artifacts.sh decides the FC_VERSION baked into the web image (#3202).
|
||||
# Web only, and deliberately: ml and agent ask this script for `revision`
|
||||
# alone, so they are covered by the self-correcting path above, and the
|
||||
# extension takes its version from packaging.sh. Milestone 318 step 5 is
|
||||
# the worked instance — b3989d0 and 5771fd5 share revision fb2c4d5b80be
|
||||
# while the version moved 2026.8.28.1249 -> 2026.08.28.1249. It was
|
||||
# harmless only because FC_VERSION did not exist until one commit later.
|
||||
("scripts/artifacts.sh", ("web",)),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path, artifacts", DERIVERS, ids=lambda v: str(v))
|
||||
def test_a_version_deriver_is_in_the_set_of_what_it_decides(path, artifacts):
|
||||
for artifact in artifacts:
|
||||
inc = includes(artifact)
|
||||
excluded = [
|
||||
p[len(":(exclude)"):] for p in declared_paths(artifact)
|
||||
if p.startswith(":(exclude)")
|
||||
]
|
||||
assert any(covered_by(path, i) for i in inc), (
|
||||
f"{path} decides the version {artifact} reports, but is not in the "
|
||||
f"{artifact} path set. A change to the derivation would leave the "
|
||||
f"revision untouched, the build skipped, and the published image "
|
||||
f"reporting the old version — with nothing to disagree with it."
|
||||
)
|
||||
assert not any(
|
||||
covered_by(path, e.rstrip("*").rstrip("/")) for e in excluded
|
||||
), (
|
||||
f"{path} is excluded from the {artifact} path set, so a change to "
|
||||
f"how the version is derived would not move the version — and "
|
||||
f"step 4 would reuse the image that carries the old one"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"artifact, path",
|
||||
[
|
||||
# Deliberate exclusions — the too-wide direction. Each of these lives
|
||||
# beside shipped code but never reaches an image, and including it
|
||||
# would re-version the artifact for a change it does not carry.
|
||||
#
|
||||
# "Never reaches an image" is the test, not "is not source": DERIVERS
|
||||
# above are also copied into nothing and DO belong in their sets,
|
||||
# because they decide what the image reports. The line between the two
|
||||
# lists is whether the file has a say in the artifact's identity.
|
||||
("agent", "agent/README.md"),
|
||||
("agent", "agent/ruff.toml"),
|
||||
("agent", "agent/docker-compose.yml"),
|
||||
# vite builds from src/, index.html and public/; it never reads test/,
|
||||
# so a frontend test change cannot reach `dist`.
|
||||
("web", "frontend/test/gallery.spec.js"),
|
||||
],
|
||||
)
|
||||
def test_files_that_never_reach_an_image_do_not_version_it(artifact, path):
|
||||
paths = declared_paths(artifact)
|
||||
excluded = [p[len(":(exclude)"):] for p in paths if p.startswith(":(exclude)")]
|
||||
inc = [p for p in paths if not p.startswith(":(exclude)")]
|
||||
|
||||
included = any(covered_by(path, i) for i in inc)
|
||||
exempted = any(covered_by(path, e.rstrip("*").rstrip("/")) for e in excluded)
|
||||
assert not included or exempted, (
|
||||
f"{path} is in the {artifact} path set but is not copied into the "
|
||||
f"image — it would re-version and rebuild {artifact} for a change it "
|
||||
f"does not ship."
|
||||
)
|
||||
@@ -9,3 +9,63 @@ async def test_health_returns_ok(client):
|
||||
assert response.status_code == 200
|
||||
body = await response.get_json()
|
||||
assert body == {"status": "ok"}
|
||||
|
||||
|
||||
# --- build identity (milestone 318 step 6) --------------------------------
|
||||
#
|
||||
# With no version image tags left, /api/health is the only place an instance
|
||||
# says which build it is. Both fields are OMITTED when unset rather than sent
|
||||
# empty: absence already means "cannot say" — an image predating the field
|
||||
# says exactly that by not having the key — and a second spelling would make
|
||||
# every reader special-case it (note #3127 §7).
|
||||
#
|
||||
# The test above is load-bearing for that: it asserts the body is EXACTLY
|
||||
# {"status": "ok"} when nothing is stamped, so a well-meaning `or ""` default
|
||||
# fails it.
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health_reports_the_build_it_is(client, monkeypatch):
|
||||
from backend.app.api import health
|
||||
|
||||
monkeypatch.setattr(health, "FC_VERSION", "2026.08.28.1249")
|
||||
monkeypatch.setattr(health, "FC_CHANNEL", "dev")
|
||||
|
||||
body = await (await client.get("/api/health")).get_json()
|
||||
assert body == {
|
||||
"status": "ok",
|
||||
"version": "2026.08.28.1249",
|
||||
"channel": "dev",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health_keeps_the_channel_out_of_the_version(client, monkeypatch):
|
||||
"""Rule 149, asserted rather than assumed.
|
||||
|
||||
The tempting shortcut is a `-dev` suffix on the version. The extension's
|
||||
comparator parses each dotted segment with `parseInt`, so a suffixed
|
||||
segment reads as 0 and every dev build compares equal to every other —
|
||||
#2993 exactly. Two separate keys cannot express that mistake.
|
||||
"""
|
||||
from backend.app.api import health
|
||||
|
||||
monkeypatch.setattr(health, "FC_VERSION", "2026.08.28.1249")
|
||||
monkeypatch.setattr(health, "FC_CHANNEL", "dev")
|
||||
|
||||
body = await (await client.get("/api/health")).get_json()
|
||||
assert body["version"] == "2026.08.28.1249"
|
||||
assert "dev" not in body["version"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health_omits_a_channel_it_cannot_name(client, monkeypatch):
|
||||
"""A locally-built image has a version but no channel. It must not gain an
|
||||
empty one — the key's absence is the answer."""
|
||||
from backend.app.api import health
|
||||
|
||||
monkeypatch.setattr(health, "FC_VERSION", "2026.08.28.1249")
|
||||
monkeypatch.setattr(health, "FC_CHANNEL", "")
|
||||
|
||||
body = await (await client.get("/api/health")).get_json()
|
||||
assert body == {"status": "ok", "version": "2026.08.28.1249"}
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
"""What the release changelog promises, and the way it would lie quietly.
|
||||
|
||||
A changelog has no consumer that checks it. If it lists the wrong span nothing
|
||||
fails — the release publishes, reads perfectly, and tells the operator that a
|
||||
month of work landed in a build that never contained it. That is the same
|
||||
silent-and-plausible failure class as a revision that identifies the wrong
|
||||
content (`test_artifact_identity.py` guards the other side of it), so the span
|
||||
selection is asserted rather than eyeballed.
|
||||
|
||||
Everything runs the script the way `release.yml` runs it — as a subprocess,
|
||||
through `--dry-run`. That is the same code path as a real publish right up to
|
||||
the HTTP call, so these exercise the interface CI uses instead of a Python
|
||||
re-implementation of it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SCRIPT = ROOT / "scripts" / "release_notes.py"
|
||||
|
||||
|
||||
def notes(*args: str, cwd: Path | None = None) -> str:
|
||||
return subprocess.run(
|
||||
["python3", str(SCRIPT), "--dry-run", *args],
|
||||
capture_output=True, text=True, check=True, cwd=cwd or ROOT,
|
||||
).stdout
|
||||
|
||||
|
||||
def body_of(out: str) -> str:
|
||||
assert "--- body ---" in out, f"no body was rendered:\n{out}"
|
||||
return out.split("--- body ---", 1)[1]
|
||||
|
||||
|
||||
def git(repo: Path, *args: str) -> str:
|
||||
return subprocess.run(
|
||||
["git", "-c", "user.email=ci@example.invalid", "-c", "user.name=ci",
|
||||
"-c", "commit.gpgsign=false", *args],
|
||||
capture_output=True, text=True, check=True, cwd=repo,
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def shaped_history(tmp_path: Path) -> Path:
|
||||
"""Three releases spanning the rule 148 tag-shape change.
|
||||
|
||||
Ancestry order is `v26.06.04.0` → `v2026.08.28.2208` → `v2026.08.29.1000`,
|
||||
which is the exact arrangement where walking ancestry and sorting a list
|
||||
disagree — see the test below. Synthetic rather than taken from this repo's
|
||||
own tags so it holds whether or not CI's checkout brought the tags along:
|
||||
a span test that quietly skips is the one outcome worse than a failing one.
|
||||
"""
|
||||
repo = tmp_path / "shaped"
|
||||
repo.mkdir()
|
||||
git(repo, "init", "-q", "-b", "main")
|
||||
for i, tag in enumerate(("v26.06.04.0", "v2026.08.28.2208", "v2026.08.29.1000")):
|
||||
(repo / "f.txt").write_text(f"{i}\n")
|
||||
git(repo, "add", "f.txt")
|
||||
git(repo, "commit", "-q", "-m", f"work landing in {tag}")
|
||||
git(repo, "tag", tag)
|
||||
# One more commit and a merge, so the merge-exclusion test has something to
|
||||
# exclude that a first-parent listing would otherwise show.
|
||||
git(repo, "checkout", "-q", "-b", "side")
|
||||
(repo / "g.txt").write_text("side\n")
|
||||
git(repo, "add", "g.txt")
|
||||
git(repo, "commit", "-q", "-m", "feat: work done on the side branch")
|
||||
git(repo, "checkout", "-q", "main")
|
||||
git(repo, "merge", "-q", "--no-ff", "side", "-m", "Merge pull request #999 from side")
|
||||
git(repo, "tag", "v2026.08.30.0900")
|
||||
return repo
|
||||
|
||||
|
||||
def test_the_previous_release_is_found_by_ancestry_not_by_sorting(shaped_history):
|
||||
"""The trap this repo is standing in right now.
|
||||
|
||||
Rule 148 moved the tag shape from `v26.05.22.0` to `v2026.08.28.2208`.
|
||||
Lexicographically `v2026...` sorts BEFORE `v26...` — the third character is
|
||||
`0` against `6` — so a sorted-list implementation reaches back past every
|
||||
new-shape tag to the newest OLD-shape one and emits months of commits as
|
||||
"changes since". It looks entirely correct on any repo whose tags share a
|
||||
single shape, which is every repo until the day the shape changes.
|
||||
|
||||
Here, ancestry says `v2026.08.28.2208` and sorting says `v26.06.04.0`.
|
||||
"""
|
||||
out = notes("v2026.08.29.1000", cwd=shaped_history)
|
||||
assert "previous=v2026.08.28.2208" in out
|
||||
assert "v26.06.04.0" not in out
|
||||
|
||||
|
||||
def test_the_body_names_the_span_it_actually_listed(shaped_history):
|
||||
"""A body whose heading says "since X" over commits computed from Y is
|
||||
unfalsifiable from outside — both halves read fine on their own."""
|
||||
body = body_of(notes("v2026.08.29.1000", cwd=shaped_history))
|
||||
assert "## Changes since v2026.08.28.2208" in body
|
||||
assert "v2026.08.28.2208..v2026.08.29.1000" in body
|
||||
assert "work landing in v2026.08.29.1000" in body
|
||||
assert "work landing in v2026.08.28.2208" not in body
|
||||
|
||||
|
||||
def test_merges_are_excluded_so_the_list_is_the_work(shaped_history):
|
||||
"""Rule 153 merges dev into main with a plain merge commit, so `main`'s
|
||||
first-parent view is nothing but "Merge pull request #N". Including those
|
||||
would publish a changelog of PR numbers over the actual changes."""
|
||||
body = body_of(notes("v2026.08.30.0900", cwd=shaped_history))
|
||||
assert "feat: work done on the side branch" in body
|
||||
assert "Merge pull request #999" not in body
|
||||
|
||||
|
||||
def test_the_first_release_still_renders_with_nothing_behind_it(shaped_history):
|
||||
"""No previous tag is reachable from the oldest one. That is a real state,
|
||||
not an error, and it must not take the release down with it."""
|
||||
out = notes("v26.06.04.0", cwd=shaped_history)
|
||||
assert "previous=<none>" in out
|
||||
assert "## Changes" in body_of(out)
|
||||
|
||||
|
||||
def test_a_non_tag_ref_renders_but_refuses_to_claim_it_published():
|
||||
"""`--dry-run HEAD` is the operator's preview before deciding to cut a tag
|
||||
at all. It must not describe itself as a release that happened."""
|
||||
out = notes("HEAD")
|
||||
assert "which is not a tag" in out
|
||||
body_of(out)
|
||||
|
||||
|
||||
def test_the_rollback_refs_name_all_three_images():
|
||||
"""Rule 145: `:c-<sha>` is the rollback unit, and the three images move
|
||||
together. A release listing only the web image sends an operator into a
|
||||
rollback that leaves ml and agent on the newer build — the exact mismatch
|
||||
build.yml builds all three on every push to avoid."""
|
||||
body = body_of(notes("HEAD"))
|
||||
for image in ("fabledcurator", "fabledcurator-ml", "fabledcurator-agent"):
|
||||
assert f"bvandeusen/{image}:c-" in body, f"{image} missing from the rollback refs"
|
||||
|
||||
|
||||
def test_an_unbounded_span_is_truncated_and_says_so():
|
||||
"""With no reachable previous tag the span is the whole history. Emitting
|
||||
eleven hundred lines would bury the one line explaining why there are
|
||||
eleven hundred of them, so the cap is part of the message, not a silent
|
||||
slice."""
|
||||
out = notes("HEAD")
|
||||
if "previous=<none>" not in out:
|
||||
pytest.skip("a previous tag is reachable from HEAD in this checkout")
|
||||
body = body_of(out)
|
||||
listed = [ln for ln in body.split("\n") if ln.startswith("- ")]
|
||||
assert len(listed) <= 200
|
||||
assert "more than a changelog is for" in body
|
||||
Reference in New Issue
Block a user