Make a digest mean something again, and give the refresh somewhere to stand #247

Merged

bvandeusen merged 4 commits from dev into main 2026-09-02 14:56:12 -04:00

Conversation 0 Commits 4 Files Changed 7

+742 -101

bvandeusen commented 2026-09-02 14:56:06 -04:00

Owner

Copy Link

Copy Source

Four commits: the first public release's notes, and milestone 362 steps 1–2.

c037006 — the first release describes the product, because it has nothing to diff against

Cutting step 7's tag today would have published "changes since v26.06.04.0" over 533 commits, truncated to 200 — a release page whose first screen is three months of internal development, addressed to someone who has never seen the project. And it was not fixable by hand: release.yml PATCHes the body on every tag push, so a written description would have been overwritten by the derivation.

Two causes. previous_tag() matched v*, so ancestry walked back to a tag whose release step 3 deleted — reachable is not comparable. And once the match narrowed to rule 148's shape, the first such tag reaches nothing, where the old fallback was the whole history.

So: match v[0-9][0-9][0-9][0-9].*, and a release with no predecessor renders the product overview instead of a commit list. The overview is read out of README.md between <!-- overview:start --> and <!-- overview:end --> rather than written into the script — same reasoning as the derived changelog, and it means the release page and the repo front page cannot disagree.

Also caught while marking up the README: "What it does" still advertised "Importing — ingests an existing library from disk", the folder-import feature 3590c47 had documented as deliberately retired two sections below.

635138b — pin the build clock (#3265)

The weekly refresh rewrote all three :latest tags on 2026-08-30 with nothing changed in any of them. Not a cache miss: run 4934 shows every content step CACHED and both bases on unchanged pinned digests. buildkit stamps the config with the wall clock, so identical layers republish under a new config blob.

The cost is meaning, not storage — :latest moved on a calendar, so a digest change stopped being evidence of anything. SOURCE_DATE_EPOCH fixes it, sourced from a new artifacts.sh epoch routed through the same newest() that already backs revision and version. Three fields of one lookup cannot drift into naming different commits.

Measured, three consecutive builds of identical source:

image	5235	5238	5241
fabledcurator	574df5638738	574df5638738	574df5638738
fabledcurator-ml	(skipped)	e94a5435cb45	e94a5435cb45
fabledcurator-agent	(skipped)	bae27d34d811	bae27d34d811

b590d25 — the refresh builds a candidate, then names the channel

Structural, so step 3's gate has a moment to occupy between "built" and "published". On a push nothing changes. On a refresh the build writes :refresh-candidate and a promote step points the channel at it.

The promote is a manifest PUT, not imagetools create — that wraps its source in an index, and an indexed channel tag makes fc.revision unreadable, so every later push misses and rebuilds with nothing going red (#3183). Promote is unconditional today; step 4 wraps it in the gate's verdict, so the refresh never spends this milestone publishing nothing.

bfc4f9c — a lever, and one fact instead of five spellings

A weekly cron is testable once a week. refresh joins force_build as a dispatch input on the same reasoning that added that one (#3252). Adding it meant collapsing five spellings of "is this a refresh?" onto one expression beside BUILD_REF.

Why this needs to be on main

A refresh checks out main while running the workflow from the branch that triggered it — the cron registers from the default branch. So dev's workflow builds main's source and cannot depend on anything main's tree lacks. It now does: the reuse step calls artifacts.sh epoch. Until this merges, both a refresh dispatch and the Sunday 06:00 UTC cron fail at that call.

It fails loudly, which is the right failure — tolerating a missing epoch would rebuild #3265 into every refresh invisibly. But it does mean the scheduled path is down until this lands, and steps 2–4 cannot be verified before it.

Verification

CI green on bfc4f9c: lint, extension-version, backend unit (551 passed), frontend build, integration. Build workflow green, and run 5252 confirms the push path is untouched — Promote skipped, main-only guard skipped, reuse hits, repoint ran as always.

Prod is already running 3e4d39b and is unaffected by anything here until the next deploy.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

Four commits: the first public release's notes, and milestone 362 steps 1–2. ## `c037006` — the first release describes the product, because it has nothing to diff against Cutting step 7's tag today would have published *"changes since v26.06.04.0"* over **533 commits**, truncated to 200 — a release page whose first screen is three months of internal development, addressed to someone who has never seen the project. And it was not fixable by hand: `release.yml` PATCHes the body on every tag push, so a written description would have been overwritten by the derivation. Two causes. `previous_tag()` matched `v*`, so ancestry walked back to a tag whose release step 3 deleted — reachable is not comparable. And once the match narrowed to rule 148's shape, the first such tag reaches nothing, where the old fallback was the whole history. So: match `v[0-9][0-9][0-9][0-9].*`, and a release with no predecessor renders the product overview instead of a commit list. The overview is read out of `README.md` between `<!-- overview:start -->` and `<!-- overview:end -->` rather than written into the script — same reasoning as the derived changelog, and it means the release page and the repo front page cannot disagree. Also caught while marking up the README: "What it does" still advertised *"Importing — ingests an existing library from disk"*, the folder-import feature `3590c47` had documented as deliberately retired two sections below. ## `635138b` — pin the build clock (#3265) The weekly refresh rewrote all three `:latest` tags on 2026-08-30 with nothing changed in any of them. Not a cache miss: run 4934 shows every content step CACHED and both bases on unchanged pinned digests. buildkit stamps the config with the wall clock, so identical layers republish under a new config blob. The cost is meaning, not storage — `:latest` moved on a calendar, so a digest change stopped being evidence of anything. `SOURCE_DATE_EPOCH` fixes it, sourced from a new `artifacts.sh epoch` routed through the same `newest()` that already backs `revision` and `version`. Three fields of one lookup cannot drift into naming different commits. **Measured, three consecutive builds of identical source:** | image | 5235 | 5238 | 5241 | |---|---|---|---| | fabledcurator | `574df5638738` | `574df5638738` | `574df5638738` | | fabledcurator-ml | *(skipped)* | `e94a5435cb45` | `e94a5435cb45` | | fabledcurator-agent | *(skipped)* | `bae27d34d811` | `bae27d34d811` | ## `b590d25` — the refresh builds a candidate, then names the channel Structural, so step 3's gate has a moment to occupy between "built" and "published". On a push nothing changes. On a refresh the build writes `:refresh-candidate` and a promote step points the channel at it. The promote is a **manifest PUT, not `imagetools create`** — that wraps its source in an index, and an indexed channel tag makes `fc.revision` unreadable, so every later push misses and rebuilds with nothing going red (#3183). Promote is unconditional today; step 4 wraps it in the gate's verdict, so the refresh never spends this milestone publishing nothing. ## `bfc4f9c` — a lever, and one fact instead of five spellings A weekly cron is testable once a week. `refresh` joins `force_build` as a dispatch input on the same reasoning that added that one (#3252). Adding it meant collapsing five spellings of "is this a refresh?" onto one expression beside `BUILD_REF`. ## Why this needs to be on `main` A refresh checks out `main` while running the workflow from the branch that triggered it — the cron registers from the default branch. So dev's workflow builds main's source and cannot depend on anything main's tree lacks. It now does: the reuse step calls `artifacts.sh epoch`. Until this merges, both a `refresh` dispatch and the Sunday 06:00 UTC cron fail at that call. It fails loudly, which is the right failure — tolerating a missing epoch would rebuild #3265 into every refresh invisibly. But it does mean the scheduled path is down until this lands, and steps 2–4 cannot be verified before it. ## Verification CI green on `bfc4f9c`: lint, extension-version, backend unit (551 passed), frontend build, integration. Build workflow green, and run 5252 confirms the push path is untouched — `Promote` skipped, main-only guard skipped, reuse hits, repoint ran as always. Prod is already running `3e4d39b` and is unaffected by anything here until the next deploy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

bvandeusen added 4 commits 2026-09-02 14:56:06 -04:00

release: the first release describes the product; it has nothing to diff against ... c0370069e0

Step 7 needs a release that reads as "what is FabledCurator and how do I run
it". What the script would actually have published is "changes since
v26.06.04.0" over 533 commits, truncated to 200 — a release page whose first
screen is the internal build-out that milestone 328 exists to stop shipping,
addressed to a reader who has never seen this project.

Two causes, fixed separately.

**A pre-convention tag is history, not a predecessor.** The 28 `v26.*` tags
were kept when their releases were deleted, so `--match v*` walks ancestry
straight back to one of them. Reachable is not comparable: nobody has run
v26.06.04.0 and its release page no longer exists to compare against. The
match is now `v[0-9][0-9][0-9][0-9].*` — rule 148's shape, which is exactly
the set of tags naming a release a reader could have been running.

**With that narrowed, the first rule-148 tag reaches no predecessor**, and the
old fallback — diff against the whole history — is worse than the problem it
replaced. A release with no predecessor now renders the product overview and
no commit list at all.

The overview is READ OUT OF README.md between `<!-- overview:start -->` and
`<!-- overview:end -->`, not written into the script, for the same reason the
changelog is derived: two hand-maintained descriptions of one product drift
and nothing ever catches it. The release page and the repo front page are one
source. Missing markers are reported as a note and publish anyway, on
cross_checks()'s reasoning — the release is still the useful object.

Every later release goes back to being a changelog, which is what note #3127
§5 says a release is for. MAX_COMMITS still guards the case it now guards:
two real releases far enough apart that the list stops being readable.

Also corrected while marking up the README: "Importing — ingests an existing
library from disk" was still advertising the folder-import feature that
3590c47 documented as deliberately retired. Replaced with what FC actually
does with what arrives — content-hash dedup, sidecar metadata, provenance.

Tests: the two that asserted the old no-predecessor behaviour are rewritten
rather than left; synthetic repos now carry their own copy of the script,
since the overview resolves relative to `__file__` (correct in production,
where release.yml checks out the tag) and would otherwise have every fixture
silently quoting FabledCurator's real README.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

ci: pin the build clock to the commit, so an unchanged refresh publishes nothing ... 635138b0d1

Milestone 362 step 1, closing #3265's root cause.

The weekly base refresh rewrote all three `:latest` tags on 2026-08-30 with
nothing changed in any of them. Not a cache miss — run 4934's log shows every
content step CACHED and both bases resolved to unchanged pinned digests.
buildkit stamps the image config with the wall clock of the build, so
identical layers get republished under a new config blob and therefore a new
manifest digest.

The cost is not storage, it is meaning: `:latest` moved on a calendar, so a
digest change stopped being evidence that anything was different. That is the
one thing a digest is any use for, and it is load-bearing here — the reuse
check, the `:c-<sha>` rollback story and any future redeploy signal all rest
on it.

SOURCE_DATE_EPOCH normalises `created` and the history timestamps, so the same
source produces the same config bytes and the same digest, and pushing it is a
registry no-op.

The value is routed through artifacts.sh's existing `newest()` rather than
taken from git separately. `revision`, `version` and now `epoch` are three
fields of ONE lookup, so they cannot drift into naming different commits — a
divergence that would stamp an image reproducibly against one commit while it
reported being another, with both values looking perfectly well-formed. Note
#3127 §2 is the record of what a second clock costs; this adds a view, not a
clock.

Also corrected: the build step comment and ci-requirements.md both described
the churn as current behaviour with the fix as a "likely" future. They now
describe what the file does.

Tests pin the property the fix depends on, not the fix: epoch is the same
commit version names, in both renderings including the extension's unpadded
one, and it does not move between two calls on one checkout. A future
refactor that gave epoch its own `git log` would pass every other test in
that file.

Not yet verified end to end — proving it needs two consecutive refreshes to
land on the same digest, which is the next thing, and is the step #3265 exists
because nobody did last time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

ci: the scheduled refresh builds a candidate, then names the channel ... b590d25f8f

Milestone 362 step 2. Structural: it creates a moment between "built" and
"published" for step 3's gate to occupy. No behaviour change.

A refresh rebuilds against freshly resolved base images, and the web image's
runtime is a line of UNPINNED Debian packages — ffmpeg, libjpeg62-turbo,
libpq5, megatools — re-resolved on every build. Nothing in ci.yml can see
that: its lanes run on ci-python:3.14 and install requirements.txt, and a base
bump changes neither. So refreshed bytes need proving before :latest names
them, and proving needs somewhere to stand.

On a push nothing changes: build_ref IS channel_ref, promote is false, and the
build writes the channel tag directly the way it always has. On the schedule
the build writes :refresh-candidate — one moving ref per image, overwritten in
place, holding a build nobody is told to pull. That is the shape rule 145
already allows for :buildcache, not the per-build tag family 318 withdrew.

Both values are decided in the reuse step beside `hit`, because that step
already owns "what does this job do" (build.yml's own rule, at the force
branch). A promote condition derived somewhere else could disagree with the
tag the build actually wrote.

**The promote is a manifest PUT, not `imagetools create`.** That distinction is
the whole risk in this change. `imagetools create` wraps its source in an
index, and an indexed channel tag is the one thing this pipeline cannot
survive: `.Image.Config.Labels` does not resolve through an index, so the
fc.revision the reuse check reads back would come up empty, every later push
would miss and rebuild, and nothing would go red. That is #3183, observed on
run 4751 — reuse worked exactly once and the only symptom was the bill. The
repoint step already excludes its own source tag for this reason; a promote
that re-introduced the wrap through another door would undo that care.

A manifest PUT is what "make this tag name that image" means at the registry:
same bytes, same media type, identical digest, no layer transfer. It reads the
result back and fails if the tag does not name what was just written — a PUT
that 2xx'd and landed something else is exactly the silent-and-plausible
failure this pipeline keeps producing. Every call carries a deadline (rule
156); a registry that stops answering must fail the step, not hang the weekly
refresh until the job times out.

Promote is UNCONDITIONAL today, deliberately. Gating it before the gate exists
would leave the refresh building something and publishing nothing for as long
as this milestone takes. Step 4 wraps it in the smoke suite's verdict.

Not yet verified on the refresh path — that needs a scheduled run, and the
lever to trigger one on demand is the next commit. This one is verified by the
push path being untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

ci: one fact for "is this a base refresh", and a lever to trigger one ... bfc4f9cec9

Milestone 362, enabling step 2's verification and everything after it.

The weekly refresh was testable once a week. That is not a cadence anything
can be developed against, and milestone 362's whole point is a gate — which
has to be watched rejecting something before anyone can believe it is wired
up. So `refresh` joins `force_build` as a dispatch input, on the same
reasoning that added that one (#3252: confirm #3190 was gone rather than wait
for it to recur).

Adding it meant confronting that "is this a refresh?" was asked in five
places and spelled five ways: `github.event_name == 'schedule'` in an `if:`,
`$GITHUB_EVENT_NAME` in one shell, an `EVENT:` env passed into another, and a
bare expression on `pull:`. Five spellings of one fact is how half of them
come to disagree once somebody adds a sixth trigger — which is precisely what
this commit is. So it is derived once at the top, next to BUILD_REF, which
already exists for exactly this reason on exactly this question.

String comparison, not boolean: Forgejo delivers dispatch inputs as strings,
so `inputs.refresh` is 'true'/'false' and `&&` on it would read the string
'false' as truthy.

**A constraint this makes visible, which pre-dates it.** A refresh checks out
`main` (BUILD_REF) while running the workflow definition from the branch that
triggered it — the cron registers from the default branch. So dev's workflow
builds main's source, and dev's workflow cannot depend on anything main's
tree does not have yet. It does now: the reuse step calls `artifacts.sh
epoch`, which lands on main with this batch. Until then a refresh dispatch
fails loudly at that call, which is the right failure — the alternative is
tolerating a missing epoch and silently rebuilding #3265 into every refresh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA

bvandeusen merged commit adab33694d into main 2026-09-02 14:56:12 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#247