100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusen	b4e7b98390	Lapsed subscriptions stop pulling, a quieter Subscriptions card, and working feed filters (#254)	2026-09-13 22:45:03 -04:00
bvandeusenandClaude Opus 5	3fe9d0a612	fix: the Latest feed's filter dropdowns opened empty ... Operator: "the filters in the latest feed, drop down but don't have values". Two separate causes: - Platform: PostsFilterBar built its items from `platformsStore.platforms`. The platforms store has never had that property; it exposes `list` and `byKey`. The read returned undefined, `|| []` turned that into an empty list, and nothing failed. ArtistsView had copied the same read, so the Browse → Artists platform filter was empty too. Both now read `list` and show platform names rather than raw keys. - Artist: the autocomplete searched the server only after something was typed (autocomplete returns [] for an empty query by design, which its tests pin). Opening the dropdown therefore showed an empty menu. PostsFilterBar now loads every artist once from a new lightweight `GET /api/artists/names` (id, name, slug; alphabetical; no joins) and filters client-side, so the list is there on open. A deep-linked artist_id now also shows the artist's real name instead of "Artist #id". Guard: frontend/test/storeUsage.spec.js scans src for `platformsStore.<name>` and fails on any name the store doesn't define, since the frontend CI has no type-checker to catch this. A positive control shows the shipped `platformsStore.platforms` read is flagged, and a vacuity check confirms the scan really walks the tree. tests/test_api_artists_create.py covers /names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:38:27 -04:00
bvandeusenandClaude Opus 5	7ff8915147	feat: a source stops pulling once its membership ends, and resumes on resubscribe (3995) ... Operator, 2026-09-13: "if I kill a subscription on patreon I would like the pulling to stop on curator as well", with auto-resume chosen. This reverses the 2026-09-11 "report only" decision for lapsed sources. membership_reconcile.apply_membership_lapses runs in sync_memberships right after each platform's successful sync, so it only ever acts on the roster just written. It stops a source (enabled=false, with the same failure-state reset as a manual disable, #1285) only when all of these hold: - the roster is fresh - the source's matched membership says has_paid_access is False (lapsed, or a free follow) - the paid-through date has passed, where the platform gives one (Patreon's member.access_expires_at; SubscribeStar gives none, so it stops at once) - the source is enabled - the operator hasn't chosen to keep it It never acts on absence. A source with no matched membership keeps pulling, because a rename or a never-walked source produces the same absence. An unrecognised status is never a lapse either. It resumes only sources carrying its own `_membership_stopped` marker, once the membership is paid again. The operator outranks the sweep both ways (SourceService.update): - turning a stopped source back on marks it `_membership_kept`, so the next sweep leaves it alone until it's paid again - turning a source off by hand drops the marker, so the sweep never switches it back on Both are `_`-prefixed app-managed config keys, which operator edits already preserve. No migration. The roster/fetch line holds. This is a source-level action by the sweep. No download path reads the roster, and the scheduler still selects on `enabled` alone. test_no_fetch_path_can_read_the_roster is unchanged. UI: SourceRow shows a neutral "Membership ended" chip, with the status and the resume/keep explanation, ahead of the other chips. The sweep's task summary reports stopped/resumed counts. Tests (tests/test_membership_lapses.py): - a lapse stops the source with a clean slate and keeps the id cache - paid-through is honoured - absence, an unknown status and a stale roster never stop anything - a resume touches only what the sweep stopped - a manual on sticks, a manual off drops the marker, and a kept source is released once paid Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:08:06 -04:00
bvandeusenandClaude Opus 5	4d84ab2816	fix: the filter rail goes, and the subscriptions card stops listing what you don't pay for — and can be dismissed ... Operator live review of #253 on the wide window. The images landed well, but: - "the left rail feels like wasted space". The filters and check status move back into one row above the feed (PostsView). On a wide window the row lines up with the feed column, clear of the day gutter, and the container recentres without the 280px rail. The narrow layout is as before, with the status still first. This retires #407 option E; A and D stay. - The "you can't see" link landed on the Subscriptions reconcile card, which listed 9 sources as `former_patron` under "sources your roster doesn't account for". The operator doesn't want sources they no longer pay for listed there, and wants the card dismissable "unless something changes": - MembershipReconcileCard renders only "creators you subscribe to but don't follow here" and the stale-roster warning. - Stopping pulls on lapsed sources is the membership sweep's job instead (#3995, next). - A close button dismisses the card. The dismissal is keyed to a fingerprint of what the card says (offered membership ids plus stale-roster flags), so a new subscription or a roster going stale brings it back. It's stored per browser, which is enough for a single-operator instance. The backend still computes tracked_not_subscribed. #3995 acts on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:04:02 -04:00
bvandeusen	c3dac235a9	Wide-window feed, Discord downloads named and dated correctly, and a repair for the broken ones (#253)	2026-09-13 20:57:06 -04:00
bvandeusenandClaude Opus 5	9b82a95b7e	feat: Settings can re-download the Discord images the None naming broke (3999) ... The operator chose a clean re-download over relinking in place. The ~1,600 broken files can't be relinked reliably: their message ids are gone, and their sidecars collided. Settings → Maintenance → "Repair Discord downloads" previews, then applies: - Deletes every image whose path is `…/discord/None/<8 digits>_None_…`. Both the folder and the name are required, because that pair is only what the bug produced. It reuses cleanup_service.delete_images for the record and file deletes. - Sweeps the leftover collided sidecars from those folders and removes the emptied folders. - Only then clears gallery-dl's archive rows `discord%`, excluding `discordasset_%`. Upstream keys message attachments as `discord{message_id}_{num}`. Since the broken files lost their message ids, per-source forgetting is impossible. Every pre-fix Discord download is broken, and files fetched after the fix still exist on disk, so gallery-dl's `skip` won't re-fetch them. - Arms a fresh backfill on every Discord source. The apply defaults to preview at both the route and the task, runs on maintenance_long, and is never on a beat. The card uses the confirm-dialog pattern of AttachmentReclaimCard. Supporting refactors, with no behaviour change: - gallery_dl.archive_path() is the single definition of the archive location. - source_service.arm_backfill() is the mutation start_backfill already did, now shared with the sync repair. Tests (tests/test_discord_repair.py): - The archive clear leaves other platforms and Discord assets alone, and counting mutates nothing. - Case-twin artist folders are both found. - The folder sweep works. - An integration run shows only the broken image goes. A correctly named Discord file and a `None` folder under Patreon survive, and only Discord sources are re-armed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 20:52:32 -04:00
bvandeusenandClaude Opus 5	dc840feec7	fix: Discord downloads land as None/<date>_None_<name> with no post — name them from the keys gallery-dl really emits ... Operator report: a Discord source showed the wrong post time. Listing the real downloads found every Discord folder on the instance (7 artist folders, ~1,600 media) in a directory named `None`, as `<date>_None_<original name>`, next to sidecars named `<original name>.json`. The real sidecar settles why: - gallery-dl's discord extractor emits `channel` as a plain string, the message as `message_id`, and the attachment position as `num`. It has no `id` key. - The patterns asked for `{channel[name]}` and `{id}`. gallery-dl renders a missing field as "None" and carries on. - The sidecar was named `{filename}.json`, the attachment's ORIGINAL name. find_sidecar never pairs that with `<date>_None_<name>.png`, so no Discord file ever got a Post or a post date. The card fell back to downloaded_at. - Every `image.png` in a channel also overwrote the same `image.json`. Fix (gallery_dl.py): - The directory is `{channel}`. - The filename is `{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}`, unique per attachment. - A discord-scoped metadata postprocessor names the sidecar exactly like the media minus its extension, so find_sidecar's first candidate matches. A per-source filename override re-derives the sidecar name. save_metadata=False drops it. Guard (tests/test_gallery_dl_naming.py) renders the patterns through Python's formatter against a sanitized copy of the real sidecar (same keys and types, invented values). A missing key or a subscript into a string raises, which is the loud failure gallery-dl doesn't give. A positive control shows both shipped patterns fail it. Existing broken downloads are NOT repaired by this. gallery-dl's archive already records them, so a re-run skips them, and their collided sidecars no longer describe them. That repair is a separate, destructive step for the operator to decide on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 20:01:11 -04:00
bvandeusenandClaude Opus 5	98c3b74260	feat: the Latest feed uses a wide window — filmstrip cards, a day gutter and a filter rail (407 A, D, E) ... On the operator's 3432px window the feed was a 900px column, 26% of the width. They picked three options from a to-scale layout study. A — filmstrip card (PostCard.vue): - A card measures itself with a ResizeObserver. At 1100px or wider its hero gets a fixed height, clamp(260px, 34vh, 460px), and the extra images move into a 2-column grid of squares beside it. The grid cells are sized from the hero height, so the grid ends flush with the hero. - The rail cap is 4 cells in this layout (2×2, the last becoming "+N") and 5 in the narrow layout, which is unchanged. - The description clamp drops to 4 lines, because long reads happen in the expanded view. - The hero has a height, not a width, so a wide card can't grow into a full-screen post. That was the operator's constraint. D — day gutter (PostsView.vue): - The normal feed groups consecutive posts by local day (Today, Yesterday, a weekday, or a date), with post and artist counts for what has loaded. - Runs rather than date buckets, because the sort key includes resurfaced_at, which the payload doesn't carry. A resurfaced grouping gets its own heading where it actually appears, instead of being pulled out of order. E — filter rail (PostsView.vue): - At 1600px and wider, the filters and status ribbon stack in a sticky 280px left rail, and each day's heading sits in a sticky 150px gutter beside its posts. - Below 1600px the layout is exactly the old one, including the 900px column. The in-context (post_id) view gets the wide column but no rail or day grouping, so anchor scrolling is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 19:48:46 -04:00
bvandeusen	69d9c14e35	The Patreon roster syncs, the favicon shows, and the logo sits behind every page (#252)	2026-09-13 18:22:00 -04:00
bvandeusenandClaude Opus 5	4b4e532c56	fix: the membership status vocabulary moves beside Membership, so the Patreon client never imports the roster ... 240f11c made PatreonClient._membership import has_paid_access from membership_roster. test_gated_reason::test_no_fetch_path_can_read_the_roster failed on it, correctly: native_ingest_common is a fetch root, patreon_client is reachable from it, and no fetch path may be able to reach the roster. The roster is allowed to explain a skip, never to cause one. MEMBERSHIP_STATUS and has_paid_access are pure platform knowledge with no database behind them. They move to native_ingest_common, next to the Membership type they interpret (the same move C7 made for Membership itself). membership_roster, membership_reconcile, patreon_client and the tests import them from there. There is no re-export from membership_roster. The guard is unchanged. The lapsed-orphan skip from 240f11c stays as it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 16:07:26 -04:00
bvandeusenandClaude Opus 5	240f11c5aa	fix: one lapsed membership to a deleted creator no longer fails the Patreon roster ... The first live sweep failed with "Patreon member resource has no campaign relationship". The operator ran FC's exact /api/members request in a browser and shared the response. - It has 104 rows. The browser's settings page shows 43, because FC sends no membership-type filter and so also gets lapses back to 2016. - Exactly one row has no `campaign` key at all: a former_patron whose membership ended in 2017. Its included reward has no campaign link either, so the creator's page is gone. - Every other row, including the 4 is_active=false campaigns, has a campaign. _membership returns None for a campaign-less row only when has_paid_access says it is definitely not paying (a known lapsed status, or a free member), and iter_memberships skips it. That changes no conclusion. A lapsed membership and an absent one both mean "not paying", and no Source can match a campaign with no id. An active or unrecognised membership without a campaign still raises, because dropping a membership that might be paid would read downstream as a cancellation. Paging still counts the rows the server sent, not the rows kept. A test pins that, so a skip can't re-read an offset or stop a page short. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 16:01:50 -04:00
bvandeusenandClaude Opus 5	57c880a623	feat: the full brand mark as a faint backdrop behind every page ... The operator asked for one large version of the logo as the site background. It is pinned to the viewport behind .fc-content at about 6% strength, so it shows in the gutters and on bare page ground while cards and the nav cover it. The series reader is immersive, skips the shell, and never draws over it. It is a layered background (the page colour at 94% over logo.svg), not an overlay element with opacity. An overlay needs the content z-indexed above it, which turns every page into one stacking context under the nav's z-index 1000 and can trap an in-page overlay beneath the nav. A background changes nothing about stacking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 15:58:40 -04:00
bvandeusenandClaude Opus 5	dd766eb976	fix: the favicon and nav brand mark rendered blank — "--" inside an XML comment ... 7ca6ee0's favicon.svg had a comment naming the `--fc-chrome-rgb` custom property. `--` is illegal inside an XML comment, so the file is not well-formed XML, and a browser renders an SVG-as-image only if it parses. It fails without any error, just a blank image. Because TopNav's brand glyph is the same file, the tab icon and the nav mark both went missing on the #251 deploy. logo.svg parses and was unaffected. The comment now names the property without the hyphens, and says why it has to. tests/test_public_svgs.py parses every SVG under frontend/public. It includes a vacuity guard (the directory really contains the two known files) and a positive control (the exact defect raises ParseError), so the check can actually fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 15:57:19 -04:00
bvandeusen	057bab1011	What you pay for, what Discord drops, and pixiv switched off (#251)	2026-09-13 12:09:28 -04:00
bvandeusenandClaude Opus 5	e3c516d6be	fix: a retired platform leaves the chip maps, and a fixture stops creating pixiv sources (406) ... e3fd8c6 failed two lanes, both on things pixiv's retirement correctly started refusing. Unit - test_fe_be_contract pins platformColor.js's ICONS keys to known_platform_keys(). I had kept pixiv's icon, colour and label "so existing pixiv posts don't look broken". That was wrong, and the file's own header already said why: unknown platforms fall back to a grey chip with the raw key, and that fallback is deliberately what a retired platform hits - it is how a pre-3069 deviantart row already renders. So pixiv leaves all three maps, and its posts show as a grey "pixiv" chip rather than a broken one. The header now says outright that a retired platform is removed, never kept, since the milestone plan itself got this backwards. Integration - four reassign tests built their fixture source through SourceService.create with platform="pixiv", which the validator now rejects. Reassign never reads the platform and never moves files, so any registered platform serves; the fixture uses hentaifoundry. 1294 other integration tests passed on e3fd8c6, so nothing else used pixiv through a validating path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 11:52:06 -04:00
bvandeusenandClaude Opus 5	c2f9e9cc08	docs: stop claiming pixiv, and stop claiming everything gallery-dl supports (406 step 4) ... Ships with the switch-off rather than with the code removal: a doc that promises a platform the product refuses is the Install and Public Surface area's characteristic defect. pixiv comes out of README (twice), SECURITY.md (twice), .env.example and the compose header. The stored-credential warnings now name Patreon and SubscribeStar - the accounts that usually carry a payment method. One correction beyond pixiv. README said FabledCurator follows creators on Patreon, SubscribeStar, Pixiv "and anything gallery-dl supports". That was already false: a platform not in the registry is rejected, however capable gallery-dl is. It now names the real set, which rule 171 records: Patreon, SubscribeStar, Discord and HentaiFoundry. The 3422 docs guards still hold - the key path and bootstrap variable are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 11:45:49 -04:00
bvandeusenandClaude Opus 5	e3fd8c67d4	feat: switch pixiv off — unregistered, unreachable, and refused at dispatch (406 phase 1) ... Milestone 406 retires pixiv (rule 171) in two phases at the operator's explicit ask: switch it off, then later delete its code. This is the switch-off. Steps 2 and 3 ship together because each is a half-state of the other: unregistered but still in the extension, pixiv creator pages would offer a button the backend then refuses. Reachability removed, never gated (rule 22 - no flag, no `if platform == "pixiv"`): - platforms registry: pixiv unregistered, so /api/platforms, the source validator and quick-add all refuse it through their existing unknown-platform paths. - NATIVE_INGESTER_PLATFORMS: pixiv removed. - extension_service: pixiv's quick-add URL pattern removed (the Python half of the JS mirror). - extension: pixiv's host permissions, content-script match, platform entry and artist pattern removed; popup's pixiv branches removed; and the whole pixiv PKCE OAuth flow cut out of background.js. That last one could not wait for phase 2 - a webRequest listener on a host the manifest no longer grants is at best dead and at worst a startup failure for the entire background script. On startup the extension now also removes any pixiv refresh token a browser still holds in storage, for the same reason as the server-side credential cleanup (3980). - frontend: the extension card stops listing pixiv; SourceActions' copy of the native list drops it. platformColor keeps rendering a pixiv key so existing pixiv posts do not look broken. The guard, and why a registry change alone was not enough. A source outlives its platform: the live instance still had one ENABLED pixiv source (step 1). Tracing it: the scheduler only selects enabled rows and every platform lookup uses .get(), so a disabled row is inert - but re-enabling it and pressing Check would have routed pixiv, no longer native, straight into the gallery-dl branch, which still has a pixiv extractor. And a worker can pick up a still-enabled row before a deploy's migration runs. So run_download and verify_source_credential - the two functions every download and credential probe pass through - now refuse any platform not in the registry: an unsupported_url failure for downloads, and an inconclusive (None, not False) verify, since nothing was probed so nothing was rejected. Generic by registration, so it covers deviantart's leftovers too. Positive-controlled: a supported gallery-dl platform must still reach gallery-dl, or a guard that refused everything would pass (rule 167). Migration 0097 disables sources on retired platforms (pixiv, deviantart) and clears their failure state exactly as disabling through the app does (1285), so the stale row stops being scheduled and stops showing as failing. Nothing is deleted: removing a source can collide with uq_post_artist_external_id_null_source on real data, which is phase 2's step 6 to check. No post or image is touched. Tests: the known-platform lists drop pixiv and gain retirement assertions beside deviantart's; pixiv's positive extension cases become negative guards; the pixiv sidecar post-URL test is deleted with the behaviour it tested; quick-add rejects a pixiv URL. The pixiv client/downloader/ingester suites stay - that code stays until phase 2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 11:45:49 -04:00
bvandeusenandClaude Opus 5	0835da8a91	fix: give the roster's column zip an explicit strict=False (387 D1, B905) ... ef91fcf failed ruff's B905 lane on one zip(labels, cells) without strict=. Tests, integration and the frontend were already green on that SHA. strict=False is the deliberate side, not the quiet one. strict=True raises a bare ValueError - not SubscribeStarDriftError - and would fail the whole roster sync over a column mismatch in `details`, which nothing reads yet. That would take down reconciliation and the gated-post reasons over a cosmetic markup change, while creator identity (id, slug) never depended on the columns at all. But a shifted column would mislabel details silently (a price filed under "discord"), so a count mismatch now logs a canary warning, mirroring the feed parser's existing parse canary: diagnosable from the worker log, never fatal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 10:56:18 -04:00
bvandeusenandClaude Opus 5	ef91fcfd26	feat: SubscribeStar joins the membership roster (387 D1) ... The second platform through the seam note 3970 contracted, characterized first from a live capture of the account's /subscriptions page (note 3989). The capture lives in the gitignored captures dir; the committed fixture is hand-built with invented values and was verified tag-for-tag against it - card wrappers, both table heads, and every distinct row shape - before any code depended on it. What the page is, and the three decisions it forced: The table IS the status. SubscribeStar has no per-row status word: a creator is either in the active_subscriptions card or the cancelled_subscriptions one. The card's data-identifier is stored verbatim as Membership.status and mapped in MEMBERSHIP_STATUS, keyed on the identifier rather than the table class because the cancelled table's class names the same list differently (for-unsubscribed_users). The creator's numeric data-user-id is the key, not the slug. A slug re-keys when a creator renames; the old row stops appearing; and a disappearance is exactly what reconciliation reads as a lapse. Keyed on the slug, a rename would have told a paying subscriber they had cancelled. The slug rides as vanity, where the identity join already looks for a handle. Price is kept as text, never parsed into amount_cents. A bare $ names no currency and a page price is not proven to be the charge - 3970 finding 4. Tier names live behind a per-row modal and are not fetched. Refusals, because SubscribeStar offers nothing like Patreon's meta.pagination.total and every conclusion downstream is drawn from absence. The parser raises when: the active card is missing (auth error on a login/age wall, drift otherwise); a row lacks a numeric creator id or a creator link; anything renders after a card's table; or the page carries a page= link. Both cards are paginatable (app#embed_pagination) and the captured account was too small to show what pagination looks like, so possible pagination is a roster FC cannot prove complete. A loud error on a larger account beats a quiet half-list. A missing cancelled card is not drift, and a creator in both tables is reported once, as active. Fetched from subscribestar.adult, not the .art the capture came from: FC's requests never clear the .art age wall with the 18+ cookie (1259, 1284). Whether /subscriptions on .adult authenticates exactly as .art did in the browser is untested - if not, the sweep records a visible error and C6 shows its unavailable rung. The seam leak D1 found. Note 3970 promised a second platform would be one builders line plus the client method. The sweep instead called current_user_id() on every client, which only Patreon's has, so SubscribeStar would have raised AttributeError on the first sweep. roster_user_id probes it with getattr, the same way the sweep already probes iter_memberships. Two existing tests were passing for the wrong reason and now can fail: - "a platform that has never been characterised says nothing" named SubscribeStar, and stayed green only because active_patron is not a SubscribeStar word. Now uses hentaifoundry, with a positive SubscribeStar test beside it. - the freshness test gave SubscribeStar a Patreon word, so the vocabulary excluded it and deleting the freshness gate outright would have left it green. It now uses cancelled_subscriptions, making the gate the only thing that excludes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 10:53:01 -04:00
bvandeusenandClaude Opus 5	529d4bff57	test: tie the install docs to the code they quote (3422 follow-up) ... 3422 was already fixed. Commit 86abaf0 applied option 1 in full and it is on main: .env.example carries the bootstrap section with the backup warning, README explains the refusal and why it is deliberate, docker-compose forwards the variable, and the milestone-362 smoke gate that FOUND the bug now sets it (build.yml:1159) and passes. The issue's premise - "CURATOR_BOOTSTRAP_NEW_KEY appears nowhere outside backend/" - is stale. What was left is the dependency that fix created. README.md and .env.example now both print the literal error text, the literal key path and the variable name, because a stranger greps for the string their terminal showed them. That is the right call and it means two user-facing files now depend on this module's wording with nothing connecting them - the install surface's characteristic defect, one rename away from a README that sends strangers to a path that does not exist. Three guards, all presence checks on both sides. An absence check against prose would pass for the wrong reason the moment a sentence were reworded (snippet 3352): - the raised message still contains the sentence README reproduces, the variable both docs say to set, and the restore-rather-than-mint alternative the whole refusal rests on; - both docs still name _CREDENTIAL_KEY_PATH and the variable, read from the code rather than retyped, so a rename fails here; - compose still forwards the variable - without that line the docs' "set it in .env" is silently inert and fails identically to not setting it. Option 2 (mint when the credential table is empty) is deliberately NOT done. The issue's own guidance is "(1) now, (2) if the friction proves annoying", and the friction has not been reported. Worth recording that its predicate checks out exactly: the Fernet key protects Credential.encrypted_blob and nothing else - no other Fernet user exists - so "no credential rows means nothing can be made undecryptable" is provable rather than probable. The cost is placement: create_app() is sync and constructs the key before any engine exists, so the check cannot live where the failure is. entrypoint.sh, which already runs alembic against the DB, is the natural seam. Only the web role is affected; the Celery roles build the key lazily inside tasks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 21:31:21 -04:00
bvandeusenandClaude Opus 5	eb6e0df858	feat: the empty front door offers to find what you already subscribe to (387 C6) ... B4's on-ramp read "add a credential, add a source" - the manual path, and the one that makes a new installer retype creators they have already told Patreon about. With C4 in place the app can just look them up. This is the step where the milestone's original framing actually lands on screen. Four rungs, chosen by ONE predicate rather than four independent v-ifs. The rungs are mutually exclusive by construction and the rung shown matches what is actually POSSIBLE: no credential -> Add a credential. Discovery is not offered, because a button that cannot work is worse than its absence. credential, never synced -> Find what you already subscribe to. Queues the C3 sweep. synced, unmatched > 0 -> the count, linking into C4's bucket 1. synced, nothing unmatched-> the manual add, because there is genuinely nothing to discover. The fifth state is rule 164's. A sweep that has been ATTEMPTED and never succeeded reads as "couldn't reach patreon", with the error type and the manual path still open - not a spinner, not a crash, not a retry button that will fail identically. An install with no outbound network lands here. That is deliberately narrower than "there is an error": a roster that synced once and failed since is NOT unavailable. It has a roster, just an ageing one, and C4's freshness gate already handles that. Collapsing the two would hide a usable roster behind an error banner. Still offer, never auto-add. An empty front door is exactly where "just add all thirty" is most tempting and most wrong - thirty backfills on first boot - so the rung carries the count and sends them to the picker. No new machinery: three existing stores (credentials, membershipSync, membershipReconcile). Reconcile is fetched only once something has synced, since bucket 1's count is meaningless before that. Every load swallows its failure, because this screen renders on an install that can reach nothing. Ten tests on top of B4's, including the mutual-exclusion property asserted directly. Its phrases are each pinned to a single template line and both sides whitespace-normalised - a phrase spanning a line break would never match, and a mutual-exclusion check whose phrases never match passes vacuously (rule 167). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 20:19:42 -04:00
bvandeusenandClaude Opus 5	4533e036ac	refactor: the membership seam's contract type is the seam's, not Patreon's (387 C7) ... Membership moves from patreon_client to native_ingest_common, beside PostRecordOutcome, for exactly the reason that one lives there: it is the seam's contract rather than the first platform's. Left where it was, D1 would have had to import the shape it implements from the module of the platform it is being mirrored FROM - which inverts the dependency and is how a seam advertised as portable quietly stays Patreon-shaped. Found by C7's own pass, which is the point of running C7 before D1 rather than writing it up afterwards: this is invisible while there is only one implementer and load-bearing the moment there are two. No behaviour change. Three files, no shim (rule 122): patreon_client imports it, the dataclass keeps its docstring, and the test imports from the seam's home. The docstring gains what the contract owes a second platform - that a missing field supplies the empty answer and never a guess: no tiers -> [], no pledge -> None (absent stays distinguishable from zero, since "free" and "we don't know" are different answers), no vanity -> None with identity falling back to the URL tail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 20:05:11 -04:00
bvandeusenandClaude Opus 5	6f5ea5d1d3	fix: the provenance panel hotlinked the CDN for images already on disk (3965) ... Two surfaces render a post's HTML body. get_post did _localize_inline_images(sanitize_post_html(...)); provenance_service._post_dict called sanitize_post_html alone. So opening the Provenance panel fetched images from Patreon's CDN for files FC had already downloaded - the archive reaching out to the platform to display what it had archived, which is the thing 830 Phase 2 set out to stop. The same bodies also break when a CDN URL expires or a post is removed, while the identical local copy sits unused. The fix is the shape, not the call. Those were two separately-callable halves and only the first looked mandatory, so a second caller was always going to do half of it. render_post_body in the new services/post_body.py is the whole pipeline in one call, and sanitizing-without-localizing is no longer a reachable operation. _localize_inline_images moves there verbatim; post_feed_service loses five imports that went with it. _post_dict becomes async and takes the session. Both call sites are already inside async methods, so for_image's list comprehension awaits per entry - fine, because localization issues ZERO queries for a body with no inline <img>, which is most of them. Recorded that early exit in the module so the loop isn't "optimized" into a batch without a measurement. Deliberately NOT fixed: provenance still names the same columns url/title/date where the feed says post_url/post_title/post_date, and its description_translated is full text where the feed truncates to DESCRIPTION_LIMIT. That is 3965's wider half - a breaking payload change for ProvenancePanel with no second reason to spend it today. Noted in _post_dict's docstring so the next reader knows it was seen and left. Four regression tests on the provenance path, covering both entry points plus the two refusals the feed already pins: an uncaptured image stays hotlinked (a broken local path is worse than an intact remote one), and a filehash owned by another artist never leaks in. Reverting render_post_body to a bare sanitize fails the first two. Recorded as snippet 3968. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 19:57:00 -04:00
bvandeusenandClaude Opus 5	2862fadcb1	fix: the roster guard's import walk resolved package __init__ imports wrongly (387 C5) ... Two of C5's three structural tests errored in CI with ValueError: PosixPath('.') has an empty name. The walk special-cased a package's __init__.py, dropping the __init__ component before computing what `from .` refers to — which made `api/__init__.py`'s `from . import health` resolve to the app root instead of to `api`, and `celery_app.py`'s `from . import celery_signals` resolve to the empty string, which is what actually crashed. The special case was never needed: `parts[:-1]` already gives the CONTAINING package for both forms, because `services/foo.py` drops `foo` to leave `services` and `api/__init__.py` drops `__init__` to leave `api` — exactly what `from .` means inside each. The level slice is now clamped at 0 as well; an import climbing past backend/app left the tree, and the unclamped negative index wrapped and resolved to the wrong module rather than to nothing. A module directly under backend/app doing `from . import x` still yields no package prefix, and there the alias alone IS the dotted name - handled explicitly rather than by falling through into a Path built from an empty string. The two positive controls earned their place immediately: they are what failed. Without them the walk would have resolved almost nothing and test_no_fetch_path_can_read_the_roster would have passed on a broken walker, reading as coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-11 23:02:18 -04:00
bvandeusenandClaude Opus 5	aa765f0a72	feat: say why the posts are invisible, without ever deciding they are (387 C5) ... A3 made a tier-gated source say "47 posts you can't see". The roster turns that into a reason: the membership ended, or the tier doesn't reach these posts, or it's a free follow. Rendered under A3's count in the health tooltip, quieter than the count it explains. FREE is a fourth case the step didn't enumerate, and it earns its own sentence. has_paid_access collapses "former patron" and "current free follower" to the same False, so deriving the reason from that boolean would tell a free follower "you're not a patron any more" - a false statement about a state they were never in. gated_reason reads the status axis first, calling has_paid_access with is_free_member forced off, then splits on the free flag. Silence is the default, and there are four ways into it: campaign absent from the roster, roster stale, platform never swept, status word not yet characterised. All four send null and the count stands alone. The frontend has no fallback sentence either - a default would turn "we don't know why" into a reason, which is the one thing this step must not do. The line that must not be crossed is pinned structurally rather than by inspection: test_no_fetch_path_can_read_the_roster walks the transitive first-party imports from the fetch roots and asserts the roster is unreachable. FC runs no local verification (rule 85), so a guard cannot be falsified by hand before it lands - it carries two positive controls instead, proving the walker finds roster imports that ARE there, one direct and one through a hop, so the real assertion can never pass merely because the walk resolved nothing. C4's identity loop moved to membership_roster.pair_sources_with_memberships when C5 became its second caller; two copies would let the Subscriptions row and the reconciliation card disagree about which creator a source IS. Three test files were each building PlatformMembership rows with their own drifting helper - consolidated into tests/roster_builders.py, same family as issue 3109. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-11 22:59:29 -04:00
bvandeusenandClaude Opus 5	de11c14448	refactor: one relative-time formatter, not three ... MembershipRosterCard had grown its own ago(iso) helper, a near-copy of utils/date.js::formatRelative. C4's card was about to become a third copy before a hook caught it. Both now use the shared helper. The hand-rolled copy was also slightly wrong in ways the shared one is not: it floored everything under a minute to '1m ago', and would have rendered NaNm ago for a null timestamp had a caller ever reached it without a v-if guard. Only sub-minute output changes, which no spec exercises - membershipRosterCard.spec.js seeds its rows at exactly 1h and 9d, where both helpers agree, and asserts on literal phrases rather than time strings. Recorded formatRelative as snippet 3959 so the next component is offered it instead of deriving a fourth copy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 22:48:24 -04:00
bvandeusenandClaude Opus 5	005680f234	fix: one source edit no longer wipes the campaign id and the backfill position ... Source.config_overrides carries two unrelated things under one column: the operator's per-source download settings, and state FC writes for itself. update() treated the whole column as operator-owned and assigned it wholesale, so a dialog save discarded patreon_campaign_id and the entire #693 backfill state machine. Not a hand-edited-JSON edge case: SourceFormDialog's structured tab rebuilds the object from two fields, so saving without touching anything was enough. _merged_config now merges: the operator's keys replace wholesale (removing a key must still remove it), FC's keys survive and are applied LAST so a stale echoed cursor cannot roll a walk backwards. App-managed is _-prefixed or *_campaign_id, both matching data already on disk, so no migration. Preserving the id exposed a bug the wipe was MASKING: nothing cleared it when a source's URL changed, and patreon_resolver reads that cache before attempting any lookup. A repointed source would have resolved the old creator forever and 387 C4 would have reported a confident wrong match. So update() drops *_campaign_id when the URL actually changes - but keeps the backfill cursor, which the walk's own stall guard validates and which is expensive to rebuild. test_update_changes_fields was DOCUMENTING the bug: it asserted config_overrides == {videos: False} on a source whose create() had armed _backfill_state, so it could only pass because the state had been destroyed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 21:50:28 -04:00
bvandeusenandClaude Opus 5	fc136006b7	feat: what you pay for, against what FC actually follows (387 C4) ... Reconciliation in Subscriptions, asymmetric on purpose. Subscriptions FC does not follow get a per-row add; sources the roster cannot account for are REPORT ONLY (operator decision) and link to the list on the same page. No one-click disable, so no disabled-reason column and no migration. The membership<->source join lands as a SHARED resolver in membership_roster, not inline here: E4 now uses it as its negative check, so the two features cannot give different answers to 'is this membership already tracked?'. Keys on the exact cached campaign id FIRST and the URL handle only as fallback, because the id is written only after a source has been walked once. Read via any <platform>_campaign_id override rather than naming Patreon's, per rule 169. The report-only bucket is gated on roster freshness and carries a per-row basis, so 'your membership says former patron', 'we know this id and it is absent', and 'we only have a handle' stay three different sentences. has_paid_access None never reads as lapsed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 21:38:03 -04:00
bvandeusenandClaude Opus 5	f8614d437d	fix: a self-contradicting test fixture, and import order (E4 follow-up) ... Two failures on 51e78a3, neither in the shipped logic. **The integration failure was a TEST bug, not a code bug.** `test_a_weak_name_needs_the_declaration` set `display_name` to something deliberately weak but left `_membership`'s DEFAULT vanity, which matched the artist slug exactly. So the name signal was legitimately 1.0 and the matcher was right to propose — my assertion of 0 was asserting the wrong scenario. Both identity fields now have to be weak for the test to mean what it says, and the arithmetic was checked before pushing: 0.39 without the declaration, 0.74 with it. Worth keeping: a fixture whose fields disagree with each other will pass or fail for reasons unrelated to the property under test, and this one was one default away from silently testing nothing. **The lint failure was import order** — `artist_membership_service` sorts before `credential_*` and I inserted it after. Second isort slip this session from patching an import block with a script rather than reading it back; the repo has a rule about exactly this (#102). Everything else passed on that SHA: 1256 tests, the frontend suite, and migration 0096. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 08:02:08 -04:00
bvandeusenandClaude Opus 5	51e78a329b	feat: offer the creator you already track as the one you subscribe to (388 E4) ... **The verification the step asked for came back "not the schema".** `Source.artist_id` is a plain FK so many sources per artist already works; `POST /api/sources` already takes an `artist_id`; the add-source dialog already has an artist autocomplete that attaches to an EXISTING artist; and `SourceService.reassign` already moves a source between artists WITH post and image re-attribution. A sweep for one-source-per-artist assumptions found only `func.count()` calls — the opposite of assuming one. So no parallel association table was built for a relationship the schema already expresses (rule 28). What was missing is FC OFFERING the link, and that is all this adds. **Accepting adds a SOURCE. It never merges two artists.** That asymmetry sets the whole posture: adding a source is trivially undone, while a wrong merge silently mixes two creators' work and corrupts tagging, series and provenance downstream with nothing left to tell them apart by. A test asserts the artist count is unchanged by accepting. The weights encode the judgement rather than a code path doing it — name 0.65, declared 0.35, cut at 0.60 — so that: * an EXACT name match alone proposes (same slug on both sides is strong, and demanding corroboration would propose almost nothing); * a CONTAINMENT match alone does not ("art" sits inside "artgirl"), and short slugs are excluded from containment entirely because a 3-character slug is inside a great many longer ones; * the declaration ALONE never proposes, because a creator may link another creator's Patreon and a link is not a claim of identity. A guard test pins all three against WEIGHTS directly and says not to fix a failure by moving the numbers. Two corrections carried forward from earlier steps rather than rediscovered: * The declaration is NOT read from `ExternalLink`. `SUPPORTED_HOSTS` is file hosts only and `host_for()` returns None for patreon.com, so no row is ever written for one — the same trap that caught E5 for Discord invites. It reads the raw body, because these links live in an `href` and `html_to_plain` discards attributes. * `vanity` is not a column: C1 modelled the roster before any platform was characterised, which is exactly what `details` exists for. `vanity_or_none()` reads it from there and falls back to the URL's last segment, so a row written before the field was understood still resolves. Two fixes during the writing. `accept()` first created a bare `Source()`, skipping the platform/URL validation, duplicate check and #693 backfill-arming that a hand-added source gets — a second, quieter way to create a source is how two paths drift until one is subtly broken; it now goes through `SourceService.create`. And the candidate query used a bare `exists().where()`, which has no FROM to correlate against; now `select(...).exists()`. Chained onto the roster sweep rather than given its own beat entry: a suggestion can only be as good as the roster behind it, so any other cadence would just propose from staler data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 07:56:16 -04:00
bvandeusenandClaude Opus 5	61abd0007c	fix: stdlib imports split by a stray blank line (isort I001) ... My scripted patch inserted the new stdlib from-imports after `import logging` with a blank line between them, which isort reads as a group boundary. Ruff-only failure — every test passed on 751e7dd, including the full integration suite and migration 0095. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 23:46:02 -04:00
bvandeusenandClaude Opus 5	751e7ddb9f	feat: the membership sweep, and the state that makes its failures readable (387 C3) ... A daily sweep that walks each platform's roster into `platform_membership`. Daily because memberships change on a BILLING cycle, not a download cadence. **`membership_sync` is the part that earns its keep.** Without it three very different situations are one indistinguishable state — the account subscribes to nothing, the sweep never ran, the sweep failed — and all three leave zero rows in `platform_membership`. "You are tracking 12 sources you no longer subscribe to" is correct in the first case and an invitation to cancel things the operator is actively paying for in the other two. So C4 gates its CONCLUSIONS on `last_success_at`, not merely its display, and `roster_is_fresh` is computed server-side so no caller can forget to. Two timestamps rather than one: `last_attempt_at` moves every run, `last_success_at` only on a clean walk. The gap between them is the signal — a sweep hammering a broken credential every day must not look healthy because it ran recently, and there is a test for exactly that. Rejected shortcuts, both tempting: `MAX(platform_membership.last_seen_at)` cannot tell "synced fine, found nothing" from "never synced"; `task_run` is worse, since its retention prunes ok rows after 24h and a sweep that last succeeded three days ago would leave no trace at all. **The fetch completes before anything is written.** That ordering is the safety property: a walk that dies mid-pagination writes nothing, so a failure can never leave a roster half this week's and half last week's. `touch_membership` never deletes, so a failure cannot empty the roster either — but "intact" should mean intact, not merely non-empty. Rule 89's four, each where it actually lives: recovery is "run it again" (upsert, no deletes); retention is C1's age-out-never-delete, because disappearing IS the signal; the wall-clock deadline is per-platform and distinct from the per-REQUEST timeout the client already has (rule 156 — a paginated roster answering every page slowly-but-within-timeout would never trip that one and would sit on a worker indefinitely); duration comes from the existing TaskRun signal plumbing. **A bug caught in review, not production:** the broad `except Exception` would have swallowed Celery's SoftTimeLimitExceeded — which is an ORDINARY Exception subclass, not a BaseException — letting the sweep run past the soft limit into the hard one, where it is SIGKILLed mid-transaction. A sweep that cannot be stopped is worse than one that fails. Now re-raised explicitly, with a test that also asserts SoftTimeLimitExceeded is still an Exception, so the re-raise cannot quietly become dead code. Rule 164 is why this ships with UI rather than backend-only: a roster that never synced must be VISIBLE as such. The card says "never synced" in words and states no count at all — rendering it as 0 is the precise conflation the whole step exists to prevent — while a real zero behind a real sync is reported as zero, because that one IS an answer. Pinned in both directions. Three independent gates decide whether a platform is swept — registered here, client exposes `iter_memberships`, credential exists — each silent, so adding SubscribeStar (D1) is one line and nothing else. A missing credential is not an error: recording a failure would light up the UI for a feature never enabled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 23:40:48 -04:00
bvandeusenandClaude Opus 5	afcde8e457	feat: PatreonClient.iter_memberships — the roster seam (milestone 387 step C2) ... Built on C0's real capture (Scribe note #3886), not on API docs — gallery-dl has no membership extractor and Patreon's public v2 API is the CREATOR surface behind OAuth, so the rule-130 reference had to be a characterized response. **The request is deliberately minimal, and that is a privacy decision.** The browser's own include set pulls `latest_pledge.card`, and those card resources come back carrying the ACCOUNT HOLDER'S EMAIL in `merchant_name`; `address` is in there too. Copying the query string wholesale is the obvious move and would have FC fetching payment PII it has no use for and can only mishandle. We ask for `include=campaign,reward` and nothing else, and a test asserts on the params actually sent so nobody widens it back. **We do not send `filter[membership_type]`.** The browser sends the six buckets its settings page displays, which excludes lapsed memberships — and a DISAPPEARANCE is precisely the signal the roster exists to read. Filtering here would manufacture the event C4 acts on. Two corrections the capture forced, both now in code: * **The filter vocabulary is not the status vocabulary.** I had read the six filter words off a screenshot and was about to write them into MEMBERSHIP_STATUS as the enum. The body shows `patron_status` carrying `former_patron` — absent from that filter — on a row the filter selected as `free_member`. So the map is taught exactly the two OBSERVED values, and `declined_patron` stays out despite looking obviously right: believing the filter is the mistake that was just caught. * **Free membership is a boolean, not a status.** `has_paid_access` gains an `is_free_member` axis, because `active_patron` alone would report a free follower as a paying patron and C4 would never offer to clean it up. Honest limit, stated in the docstring: the capture has no ACTIVE free member, so it shows the separation is possible, not that it occurs. C1's tripwire test did its job — it was written to fail the moment anyone populated the status map, and updating it here IS the confirmation step, done with the capture rather than ahead of it. `_fetch`'s retry/backoff/auth-vs-drift/Retry-After logic is extracted to a shared `_request` so the roster rides the same path rather than growing a second copy — two copies would drift, and the half that drifted would be the one that only runs daily. Every error message and log line renders byte-identically for the posts path, so the existing tests pin the refactor. Pagination is driven by `page[offset]` against `meta.pagination.total`, never by `links`: the response's own `links.first` is built WITHOUT the `/api/` prefix the request uses, so following it would hit the web page. An empty page is terminal regardless of what the total claims, so a server reporting more rows than it hands over cannot spin the walk forever. Drift is stricter here than on the posts path, on purpose: a missing `meta.pagination.total` raises rather than returning a short list, because a truncated roster reads downstream as "you cancelled those" — the worst wrong answer this feature can give. `current_user_id()` is marked INFERRED, not characterized: C0 captured /api/members, not /api/current_user, so it relies only on the JSON:API envelope this API demonstrably uses elsewhere, and raises drift rather than returning something plausible if that is wrong. The fixture is derived from the real capture with every piece of account data replaced (the raw capture stays gitignored). Six members, each earning its place: a former patron with a null pledge, an active patron with no tier, an annual cadence, a previous_pledge whose included resource has no `relationships` key at all, and a reward priced in CAD beside a USD charge — the trap that makes reading `reward.amount_cents` report a number the operator was never charged. A leak check caught a free-membership-subscription id and six real campaign launch timestamps before any of it was staged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 22:26:01 -04:00
bvandeusenandClaude Opus 5	533a1ce674	chore: a gitignored home for raw platform captures (milestone 387 C0) ... C0 characterized Patreon's `/api/members` from a live capture of the operator's own session. That capture is worth keeping — re-capturing means re-authenticating by hand, and it is the ground truth the characterization (Scribe note #3886) gets re-checked against when a platform's shape is suspected to have drifted. It cannot be committed. It carries the operator's creator list, pledge amounts, and — inside the `card` resources the web app's include set pulls — the account's own email address. So: a directory that is ignored wholesale rather than by filename, so the next capture is covered by this rule instead of needing a line somebody has to remember to add. Two details that are the point rather than incidental: * The ignore is written as `captures/*` plus a negation for README.md, NOT as `captures/`. Git does not descend into an excluded DIRECTORY, so a negation for a file inside one never takes effect — the README would have been silently ignored along with everything else, and the convention would not have survived a fresh clone. * The README states plainly that SANITIZED fixtures belong in git, elsewhere under tests/fixtures/. The raw capture exists to derive those from and to re-check against; it is not the thing tests should load. The ignore rule landed before the capture file did, deliberately: a payload with an email address in it should never be sitting in the working tree un-ignored, however briefly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 21:07:48 -04:00
bvandeusenandClaude Opus 5	059e2128ee	fix(ci): stop telling readers the runner can't run upload-artifact@v4+ ... build.yml and baseline.yml both explained a missing artifact step by saying act_runner cannot run actions/upload-artifact@v4+ (and baseline.yml said ci-requirements.md records that, which it does not). The runner is now gitea/runner 3.x, which edits the action's GHES refusal out of its bundle, and stock v4+ is proven working on this forge (Scribe spike #3843). Comments only. Neither lane gains an upload step: build-web still reads the signed XPI from the release asset, and the baseline candidate is still printed to the log, because those remain the better channels. The comments now say why for the right reason. Scribe snippet #2271, milestone 395. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DwoKYuw3qJmUUYsJeNherB	2026-09-10 17:43:36 -04:00
bvandeusenandClaude Opus 5	d0b0458d27	feat: the announcement link, on the card and in a review queue (388 E5) ... Rule 27 — E5's other half. The matcher can propose; this is where the operator decides, and where an accepted link actually shows up. **The review queue** (Settings → Ingestion & filters). Each proposal shows the per-signal breakdown, not just the total: "why did it suggest this" is the question the operator actually has, and a lone percentage cannot answer it. So a row reads "72% · timing 95% · says so 60%", and the copy states outright that a pair always needs two reasons — which is the property that stops a busy posting day from producing false pairs. The empty state says so explicitly. Nothing proposed is the EXPECTED state most of the time, and an empty queue that looks like a failure invites turning the threshold down until it produces noise. **On the card**, both directions, and accepted links only: the teaser gets "The full set is in Discord", the drop gets "Announced on Patreon". A pending proposal is a question for the review queue, never a claim to render beside the artwork — that distinction is the whole confirm-only design, so it is asserted in the backend (only `linked` rows reach the payload) and again here. One detail worth the comment it carries: the link's target is `{ query: { post_id } }` with no name or path. In vue-router that means "the current route with these query params", so it works identically from Latest and from Browse — and, more usefully, the card never reaches for `useRoute()`, which it has no other reason to know about and which is not available when it is mounted in a test without a router. Backend CI on 235393c was green: all 13 E5 tests and migration 0094. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:53:17 -04:00
bvandeusenandClaude Opus 5	235393c08b	feat: link the Patreon teaser to the Discord drop it announced (388 step E5) ... The point of the milestone rather than its tail. Two of the operator's artists post a deliberately cropped fragment on Patreon to signal that the real thing has landed in their Discord; this proposes those pairs. Confirm-only, following the FC-6.3 series matcher. A wrongly-asserted association tells the operator two different pieces are one, which is strictly worse than no link: no link leaves them where they already were, a wrong one actively misinforms and then propagates into whatever reads it. So the matcher's job is a SHORT list worth reading, not a long list worth trusting. **The threshold sits above every single signal weight, and that is the design.** Proximity is 0.55, declaration 0.45, the cut 0.60 — so neither signal can carry a pair alone. That makes "time proximity alone is never sufficient" an arithmetic property rather than an aspiration: on a busy day an artist posts several times, and a matcher that could pair on proximity alone would turn every one of those days into false pairs until the review queue got abandoned. A guard test asserts the relationship against WEIGHTS directly, so it survives any refactor of the scorer, and says in its own failure message not to fix it by lowering the assertion. **Crop-to-source matching is HELD, on the plan's instruction** — real work with real false-positive risk, worth building only once signals 1 and 2 are shown insufficient against the operator's actual artists. Worth stating: a naive whole-image SigLIP similarity is NOT that signal. A cropped teaser and its full version are precisely the pair a whole-image comparison handles worst, so adding one as a "bonus" would mostly add noise while looking like progress. Two premises in the plan corrected in the building: * **E4 is not actually a prerequisite.** A Patreon Source and a Discord Source the operator has added under one Artist already share `Post.artist_id`, and the synthetic grouping inherits it. E4 EXTENDS this to creators FC has to learn the association for; it is not needed to represent one FC was told. Same-artist is then a hard filter, not a scored signal — two different creators posting minutes apart is a coincidence, not evidence. * **`link_extract` cannot supply the declaration signal.** It exists, but `SUPPORTED_HOSTS` is file hosts only and `host_for()` returns None for a Discord URL, so no ExternalLink row is ever written for one. The signal reads the post body directly instead. And a bug my own test would have caught: `declared_signal` stripped the HTML before looking for an invite, but `html_to_plain` discards attributes and these creators put the invite in an anchor's `href` — so the strongest form of the signal was being thrown away, leaving only whatever the link text said. The invite now matches the raw body; the bare mention still matches stripped text, so `\bdiscord\b` is tested against prose rather than against markup. Dismissed rows are kept, not deleted: the row is what remembers the rejection, and re-proposing a rejected pair on every scan is the one behaviour that makes a review queue get ignored. Both FKs CASCADE, so E3's one-DELETE reversal cannot leave a proposal pointing at a post that no longer exists. Only ACCEPTED links reach the post payload. A pending proposal is a question for the review queue, not a claim to render beside the artwork. UI (rule 27) follows in the next commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:47:28 -04:00
bvandeusenandClaude Opus 5	ba96ecfb2d	fix: the disabled sweep's shape assertion pinned the pre-E3 payload ... My own E2 test asserted `sweep`'s disabled return by exact equality, and E3 added `images_joined` to it — a rule 90 miss on a consumer I wrote an hour earlier. Every E3 test passed; this was the only failure (1 failed, 1202 passed). Fixed by extending the assertion, NOT by loosening it to a subset check. The exactness is the point: a disabled sweep reports a complete zeroed shape rather than a shorter one, so a caller can read any counter unconditionally, and this assertion is what notices when a new counter skips that path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:35:53 -04:00
bvandeusenandClaude Opus 5	1e45e2c56c	feat: an open grouping — a later drop joins its post (milestone 388 step E3) ... A synthetic post is no longer sealed at creation. A creator who adds two more variants the next day extends the existing post, its body grows with the new messages, and no rival post appears. That is what makes chat capture read as content trickling in rather than as a stream of separate arrivals. The sweep now runs two passes per source and the ORDER is load-bearing: offer new messages to still-open groups BEFORE founding new ones, because whichever runs first claims a message. E3's three named problems, each answered rather than discovered later: **Bridging.** A candidate near two groups joins NEITHER. Nearest-wins would silently make an arbitrary choice between two posts the operator may already have seen; merging them is worse still, because a merge rewrites history and anything pointing at the absorbed post dangles. Leaving it to found its own group is the recoverable failure. AMBIGUITY_MARGIN is a module constant and deliberately not a setting — it is not a quality dial anyone would tune toward a better feed, and exposing it would invite turning it to zero, which is exactly the silent arbitrary choice it prevents. **Re-surfacing without thrashing.** A grouping has two dates, and which one orders the feed is a real decision, so the feed orders by neither directly. Ordering by when the drop STARTED buries a group that grows a week later under a week of other posts — defeating the point of keeping it open. Ordering by every growth lets a group gaining one image a day live permanently at the top, so chat out-competes authored posts for the front page — the opposite of "post pacing stays front and centre". Instead `resurfaced_at` moves only when growth clears BOTH a minimum-images bar and a cooldown, so a drip-feed updates in place and a genuine second wave resurfaces exactly once. It is NULL on every ordinary post, so the sort key COALESCEs through it without moving anything that is not a grouping. **Reopening forever.** Groups close after a quiet period — artists reuse characters for years, and a group left open indefinitely will eventually absorb something it shouldn't. Openness is DERIVED, not stored: a group is open if it grew (or started) within the window. Lowering the setting closes old groups and raising it reopens them, with nothing to repair either way; a stored closed_at would have needed a sweep to set it and a repair path to ever change the policy. Rule 89 is satisfied structurally rather than by a parallel mechanism: celery_signals writes a TaskRun for every task, which already supplies duration, the 5-minute stalled-run recovery, and retention pruning. What this step owed on top of that was a wall-clock limit (present) and idempotence — re-running the joiner adds nothing, asserted directly rather than left to the unique (image, post) constraint to catch. Two bugs fixed in the writing, one of which my own test would have hit: * `assign_to_group` sorted bare (distance, Post) tuples, which falls through to comparing Posts when two distances tie — and a perfectly symmetric bridge, the exact case the function exists for, would have raised TypeError instead of declining to choose. Now keyed on the distance alone. * The cursor was still built from `post_date or downloaded_at` while the ORDER BY had gained `resurfaced_at`. Two expressions that disagree at a page boundary don't error, they silently skip or repeat rows; both sites now go through one `_post_sort_value`, and a test pages through one row at a time to prove the walk matches the whole list. Image linking is now one shared helper rather than written twice, because creation and joining would otherwise be free to drift on exactly the detail (which post owns the image) that makes a grouping reversible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:30:17 -04:00
bvandeusenandClaude Opus 5	7071c87cd6	feat: a grouped post says so, and the operator can tune the grouping (388 E2) ... Rule 27 — E2's other half. The backend can author posts; this is what makes that visible and adjustable. **The honesty marker.** A chip on every synthetic post's card: "grouped by FabledCurator", titled with what it was built from ("Grouped from 4 Discord messages"). This chip is the only thing standing between "FC assembled this" and the card reading as something the artist authored, so it keys off nothing but the flag, and it states the member count rather than just disclosing that grouping happened — a claim you can check beats a claim you're asked to trust. A synthetic post has no title on purpose (inventing one is the one place this feature could put words in a creator's mouth), and the untitled fallback would otherwise have printed the internal key: "Post fc-drop:99887766". It now names the post for what it is. There's a test for that specifically. **The tuning card.** Ingestion & filters gets a Discord-drop-grouping tile: the switch, the distance cut and the drop window, each with the sentence that tells the operator which one to reach for. The window's copy says outright that it is the setting doing most of the work — without it, everything an artist ever drew of one character collapses into a single post. Both directions are pinned in postCard.spec.js, including the one that actually matters: an ordinary post is never marked. Also covered — a post dict composed before these fields existed degrades to unmarked rather than throwing on `synthesis.message_count`. Also fixes the ruff UP017 that failed the lint job on 73eeb7a (timezone.utc → datetime.UTC, the convention everywhere else in this repo). The integration suite on that SHA was green: all 12 grouping tests passed and migration 0092 applied cleanly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:17:56 -04:00
bvandeusenandClaude Opus 5	73eeb7a377	feat: FC authors the post that Discord never wrote (milestone 388 step E2) ... Discord is a delivery channel, not a publisher. One message is not one post, and today every message lands as its own `post` row, so chat lines compete with authored work for the same surface. Rather than demote them into a second-class feed, FC now writes the post itself: one row per DROP, its images the drop's images, its body the messages' text in arrival order. Synthesising a `Post` (rather than inventing a parallel entity) is the whole point — the result is post-shaped by construction, so feed, provenance, translation, attachments and series keep working on it unchanged. The predicate is three axes ANDed, and the time one does the real work: same source AND cosine distance <= threshold AND no gap > window Similarity alone over-groups, and that is the failure that would make this useless: any two pieces of the same character by the same artist sit close in SigLIP space, so a cosine-only rule collapses a month of one character into a single "post". Two details inside the predicate are load-bearing — * distance is measured to the group's SEED, never to the previous member, because chaining lets a group DRIFT: twenty small steps walk from one piece to a completely different one, each hop individually within threshold; * the window is measured between CONSECUTIVE messages, not from the first, so an artist trickling variants out over an evening stays one drop. Why a post-import sweep and not part of ingest. The obvious alternative was to migrate Discord to the native post-first ingester (#1266) and group at capture time. That cannot work: the grouping signal is `siglip_embedding`, which is produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at capture time there is nothing to group on. Grouping is necessarily something that happens once the vectors catch up — hence a re-runnable sweep that skips what it cannot yet place, and an hourly (not daily) cadence. The honesty rule, enforced in the schema. `post.synthesized_by` names the grouper; `synthesis_details` records the members, the count, and the thresholds AS THEY WERE (they are operator-tunable, so without that "why did it group these" is unanswerable a month later). Member posts are absorbed, not destroyed — they remain the images' true origin and the audit trail — and `absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post releases its members back into the feed in one DELETE with no repair step. `post_title` stays NULL deliberately: a synthesised title is the one place this could put words in a creator's mouth. Two guards the first draft would have failed: * the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON forces its own ORDER BY, so the sort now happens outside the subquery; * a cap landing mid-drop would have published a truncated group claiming to be a whole drop, so the last group is left for the next run. And one vacuous test caught before it shipped: the support vector perturbed a single component of an all-ones vector, moving it ~1e-6, so every distance assertion passed regardless of what the predicate did. `_vec` now builds a unit vector at a stated angle, where distance is exactly 1 - cos(delta) — rule 167, a guard has to be able to fail. UI (rule 27) follows in the next commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:15:14 -04:00
bvandeusenandClaude Opus 5	4fe792b61c	feat: platform_membership — the learned roster of what the account pays for (milestone 387 step C1) ... FC knows which creators it was TOLD to follow and nothing about which ones the operator is subscribed to. Those two sets drift both ways and neither drift is currently visible: a subscription FC doesn't track is content the operator believes they're archiving and aren't, and a source walked after the subscription lapsed is requests spent on a wall reported as a creator gone quiet. Sibling of service_seen (milestone 365) and the same insight — an absence is only observable against a record of presence. touch_membership reuses the recorded touch_service shape (snippet 3447): upsert rather than read-modify-write, first_seen_at deliberately outside the update set because it's the one field that makes a later DISAPPEARANCE readable as a lapse rather than as a creator we never knew. Nothing populates it yet, and that's the intended intermediate state. The sweep (C3) needs a client seam (C2) that needs Patreon's real response characterised from a captured sample (C0), which needs the operator's browser session. The table's SHAPE doesn't wait on that, because it's deliberately free-form exactly where C0's findings would otherwise dictate a column. status is an unconstrained String holding the PLATFORM's own word, not a normalised FC value. Rule 36 considered and declined, same reasoning service_seen.kind records: the vocabulary isn't ours to invent, and picking a lowest-common-denominator enum before any platform has been characterised would bake a guess into the schema. The service owns the whitelist and the mapping; the column owns the evidence. MEMBERSHIP_STATUS ships EMPTY, guarded by a test that fails if anyone adds an entry — every one must come from a characterised response, not from API docs. That's rule 130 at the one place it's easiest to break, and the failure message says so. has_paid_access returns None, never False, for a word it hasn't been taught. The difference is load-bearing: False means the operator lost access, which C4 turns into an offer to disable the source, so asserting it from an unrecognised word would tell them to cancel a subscription they're still paying for. Retention decided here rather than deferred (rule 89): a membership that stops appearing is aged out on time, never deleted on absence — deleting would destroy the signal at the moment it became interesting. Rule 90 check, done on the right thing this time: the per-test TRUNCATE teardown derives its table list from Base.metadata.sorted_tables, so the new table is picked up automatically; test_models asserts a subset, so it doesn't break. 0091 follows 0090 on the collapsed baseline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 10:58:25 -04:00
bvandeusenandClaude Opus 5	6b19012bb6	feat: the empty front door is the install's first screen (milestone 387 step B4) ... The front door is now a feed, and a blank feed implies things should be here in a way a blank masonry does not. On a fresh install this is the first screen anyone sees — including someone who is not the operator, which is what milestone 328 is making possible. Tells the two empties apart, which is the point. "No sources yet" gets the on-ramp; "sources configured, nothing landed yet" gets told that the first check takes a while and pointed at Downloads. Telling someone to add a source when they already have three and are mid-backfill reads as the app not knowing its own state. Needs total_sources on schedule-status to distinguish them — deliberately not auto_sources, which counts only what is on a schedule, so a source with auto_check off would have read as "nothing configured". Both exact-shape assertions updated in THIS change rather than after CI caught them, which is the lesson from B3's red push. An absent status falls back to the on-ramp on purpose: it is merely redundant to an established operator, whereas "see what's running" shown to someone with nothing configured is a dead end. A filtered miss is deliberately NOT the onboarding case — the operator has posts, they just narrowed past them. Showing a fresh-install on-ramp there would tell someone with a full library to go set it up. This is where logo.svg lands, as the operator asked. It earns its place on a first-run screen and not on a populated feed, and gives the on-ramp something to compose around instead of prose plus two buttons. Large: the mark stops reading below ~48px, which is why the 22px nav slot has a different one. Pinned by test so a later tidy-up cannot quietly shrink it to a glyph. Also extracts mountWithStore into the shared test support module. Writing the second spec created exactly the copy-paste that open issue 3109 tracks for the backend row factories, so it is consolidated now rather than at copy three, and recorded as snippet 3829 with the two traps it does NOT solve — named slots rendering nothing, and components that fetch on mount. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 23:10:06 -04:00
bvandeusenandClaude Opus 5	3f8306f705	fix: two exact-shape assertions pinned the old schedule-status payload ... B3 added failing_sources / no_access_sources to schedule-status and broke test_schedule_status_shape and test_summary_returns_rollup_shape, both of which assert the payload's EXACT key set. My own tests passed; these two did not, and integration caught it. A rule 90 miss: I grepped for the predicates I changed and not for consumers of the response shape. The shape is the thing I actually changed. Both keys added to the assertions rather than loosening them to a subset check — an exact-set assertion is what catches a key being renamed out from under a consumer, which is precisely the value these two tests just demonstrated. The other readers (PipelineStatusChip, SchedulerStatusBar) pull individual keys, so they were unaffected. SchedulerStatusBar's prop comment documented the old shape and is corrected here; a comment that lies about a contract is worth the same as a doc that does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 23:01:08 -04:00
bvandeusenandClaude Opus 5	a708f5e9db	feat: the front door says whether ingestion is working (milestone 387 step B3) ... The step phase A was building toward. A1 made the gated count true, A2 made it a durable state, A3 made it visible in Subscriptions — but Subscriptions is where you go once you already suspect something. This is the line that reaches someone who wasn't looking. A thin grey strip above the feed, front door only: last check, sources failing, sources you can't see. Only the actionable items take a colour, and nothing renders at zero — a permanent "0 failing" trains you to skip the line, which would hide the real number when it appears. Two predicates, defined once. The ribbon counts and the surfaces it links to have to agree on what "failing" and "no access" MEAN, or the ribbon says 3 and the card shows 4. They live in db_helpers, which exists for exactly this reason (its docstring: divergent copies are how the race bugs crept in). Not in source_service, because scheduler_service needs them too and source_service already imports scheduler_service — the other direction is a cycle. Counting deliberately spans all ENABLED sources rather than the auto_check subset scheduler_status already walks: a source erroring on a manual-only artist is still erroring. Disabled sources count for nothing, which is what makes issue 1285 the real escape hatch for a sub you stopped paying for. Extends the existing schedule-status endpoint rather than adding a parallel aggregate — the store already fetches it. Two scalar COUNTs. The status filter is now URL-addressable, which it had to be for the ribbon's links to land anywhere: a count that drops you on an unfiltered list makes the reader redo the filtering the ribbon just did. Mirrors how artistFilter already reads from route.query. Front-door-only via a route prop, not a route.name check, so the view doesn't need to know what it's mounted as and the router states the intent in one place. Inside Browse's Posts tab you're looking FOR something and the hub is one click away. The fetch is swallowed on mount by design (rule 164): this is an aside, and the feed must render whether or not the status call succeeds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 22:55:14 -04:00
bvandeusenandClaude Opus 5	ecd72015a7	feat: the front door answers "what arrived?" (milestone 387 step B1) ... Moves FRONT_DOOR from /showcase to /latest. Showcase is a random TABLESAMPLE — lean-back, and it can never tell you anything is wrong. The feed is the only view where a failing source surfaces on its own, as a creator who has gone quiet. The per-artist "new since last visit" badges (#597) were a workaround for this view not being the door. Showcase is demoted to a nav entry, not removed. Nothing is being replaced, so rule 22's delete-the-legacy-path does not apply. Deviation from the filed plan, deliberate: the step said write a new LatestView.vue. Rejected — PostsView is ALREADY a self-contained feed (own container, own store, infinite scroll, filters, deep-link anchoring, empty state), and Browse only ever wrapped it in a tab strip. A new view would have duplicated 231 working lines to gain nothing. Mounting PostsView directly at its own route IS the whole difference the promotion was after: a door you arrive at, not a hub you navigate out of. Rule 28. Backend untouched, as scoped — PostFeedService.scroll already does cursor-paginated newest-first. Two things this shook loose: PostsView's deep-link "All posts" button was a hard `{ name: 'posts' }`, which redirects into Browse. Correct while the view only ever rendered inside Browse's tab; from the front door it would have yanked the operator sideways into a different surface. Now returns to the current route minus post_id, so Browse keeps its tab and any active scope. The README claimed "A Showcase front page". That block is the SOURCE the release notes quote (scripts/release_notes.py product_overview), not a generated copy, so it is fixed here — a document contradicting the code is the characteristic defect of the public-surface area. No stickyChrome on the route: unlike Browse/Gallery/Settings this view has no sticky sub-header for the nav to butt against. The router spec pinned FRONT_DOOR to /showcase and now pins /latest, plus that Showcase stayed reachable and in the nav — the demotion is asserted, not assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 22:46:04 -04:00
bvandeusenandClaude Opus 5	7ca6ee0666	feat: new curator brand mark — traced logo + redrawn glyph ... Replaces the placeholder folder glyph with the new logo: a white-gloved hand presenting a framed work, which says what the app is for far better than the old mark did. Two assets, because one cannot serve both jobs. Measured, not assumed: rendered at 22px the full logo is unreadable mush, so favicon.svg stays a separate, much simpler mark. logo.svg — traced from the source raster with potrace, then repainted from the design tokens. Segmentation notes, since this is the part that is easy to get wrong on a re-do: hue does NOT separate the glove from the frame's highlights (both sit near 38 degrees) and neither does saturation alone. The split is a connected-component fill seeded inside the cuff, dilated first so it can cross the dark outline strokes that cut the fingertips off from the palm. The source plate was a warm brown (#1B1105), not the app's cool obsidian (#14171A) — side by side it read as a logo sitting on its own warmer card. It is dropped entirely: the mark is transparent and the frame interior shows whatever surface hosts it. The source gold was #AA7E39, which is within a couple of points of accent.curator #A87338 — so the mark now shares one colour with nav-active text and the wordmark rather than nearly sharing it. The glove goes to text.parchment for the same reason. favicon.svg — hand-drawn rather than traced. At 16px a traced mark carries hundreds of wobble nodes that read as fuzz and can never be tidied. Ring + frame + star merged into a blob at that size, so it keeps two elements: the frame and the star. Frame over a plain ring because it carries the meaning, and it is the full logo's own centrepiece; the glove, cufflink, finials and sparkle rays are deliberately absent rather than drawn and lost. The favicon keeps its obsidian plate so the tab icon is self-contained against any browser chrome; on the nav that plate is invisible because it matches --fc-chrome-rgb exactly. logo.svg has no plate at all. Both files carry a comment explaining why they are shaped this way, so the next edit does not undo the reasoning. The old favicon is one revert away in history. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 22:20:00 -04:00
bvandeusenandClaude Opus 5	6bb18050a4	feat: no-access is visible per source, and findable (milestone 387 step A3) ... A3 of milestone 387, completing phase A. A1 made the count true, A2 made it a durable state; this makes it something the operator can see without going looking. Turned out smaller than filed, because A2 revealed why the existing `tier_limited` palette entry in FailingSourcesCard had never rendered: the chip was being cleared by the same successful run that produced it. The colour was already chosen. Where it surfaces: - SourceHealthDot gains a `no-access` grade. Deliberately its own grade rather than folded into healthy (which hides it) or warning (which sends the operator hunting for a break that isn't there). A source with real failures still grades as failing whether or not it is also gated. - SourceRow gets an info-coloured lock chip in the status cell, which was empty for these sources — they have zero failures. Placed ahead of the backfill states: "we can't see this creator" is the more useful thing to say than which walk phase it is in, and unlike those it does not resolve on its own. - A "No access" status filter, deliberately separate from "Has errors". Without it a gated source is invisible in a long list, because it correctly stays out of the failing rollup. Left OUT of NeedsAttentionCard on purpose. That card's only affordance is Retry, and you cannot retry your way into a subscription tier — issue 1285 already gives the real escape hatch, since disabling a source clears its state. Nothing structural needed changing: the card is fed by consecutive_failures > 0, which a tier-limited source never has. The count lives on the download event, not the source, so `list()` joins it in with one DISTINCT ON query — selecting the run_stats sub-object rather than whole metadata blobs, which carry up to 500KB of truncated stdout each. Scoped to tier-gated rows only, so a healthy library issues no extra query at all. Absent stays None rather than 0, and both UI surfaces phrase the state without a number when it is missing instead of printing a fabricated zero. Also covers A1's live gated count, which shipped untested, and extends the mount helper with slot stubs: SourceHealthDot puts the dot in a NAMED slot, and unresolved Vuetify components render default slots only — so those assertions would have found an empty wrapper and passed vacuously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 21:26:48 -04:00
bvandeusenandClaude Opus 5	7751715b83	feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2) ... A2 of milestone 387. A1 made the gated-post count true; this makes it mean something. A native walk that reached the bottom returned `error_type=None` whether the creator had posted nothing or every post sat behind a tier we don't hold. `source.error_type` stayed NULL and the source read healthy and quiet. gallery-dl has classified this as TIER_LIMITED since the paywall-as-"needs attention" complaint; the native path never did. Two things had to move that the plan didn't foresee, both found by reading the consumers rather than by testing afterwards: The backfill lifecycle's completion test required `error_type is None`. Returning TIER_LIMITED naively would have dropped a fully-paywalled backfill into the not-finished branch — zero downloads means no progress, two strikes marks it "stalled" — so the creator we can see least would become the one we re-walk most. `walk_completed` now admits informational classes. `_update_source_health` only stamps `error_type` on status "error" and CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped by the very run that produced it — which is why FailingSourcesCard's `tier_limited` palette entry has never been reachable. An "ok" run now keeps an informational class while failures stay 0 and last_error stays clear: the run did not fail and must not earn a backoff. Deviation from the plan, deliberate: the filed step said classify only when `downloaded == 0`. gallery-dl doesn't condition on that, and diverging the two backends over the same concept is what rule 169 forbids — so the native path mirrors it. "There is content here you aren't paying for" is equally true in a week we also got the cheap posts. Pinned by a test, since the stricter rule looks more correct. The predicate, the wording and the completion test are defined once in gallery_dl.py and spread into both backends (snippet 3087), rather than re-derived per half — which is exactly how they drifted apart before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 15:40:19 -04:00
bvandeusenandClaude Opus 5	173f4b00aa	fix: the native path never reported tier-gated posts (milestone 387 step A1) ... `make_run_stats` has always declared `tier_gated_count`, and DownloadDetailModal has always rendered it. gallery-dl populated it; `ingest_core._result` did not — it built run_stats with six keys and let the seventh default to 0, while the very same walk counted gated posts into `gated_skipped` and spent the number on a log line. So on Patreon, SubscribeStar and pixiv — the three platforms we now own — the Downloads modal read "Tier-gated: 0" for a walk that skipped N paywalled posts. A creator we've lost access to was indistinguishable from a creator who stopped posting. Migrating Patreon off gallery-dl is what dropped the signal. Pass the count through, and tick it in the live-progress payload too, so a long backfill on an inaccessible creator explains itself while it runs rather than only at finalization. ActiveDownloadsPanel renders it only when non-zero, coloured 'info' to match the severity FailingSourcesCard already assigns tier_limited — this is not a failure. Tests assert the run_stats key the UI actually reads rather than the ingester's internal counter, so the guard tracks the property and not a name. Falsification is structural: `make_run_stats` defaults the key to 0, so both assertions fail against the pre-fix call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-09 12:34:08 -04:00
bvandeusenandClaude Opus 5	ad8392b790	fix: system health is a Settings tab, not a page only the dot reached ... The surface shipped at /system with no nav entry, reachable only by clicking the health dot beside the brand — a target you have to already suspect something is wrong to go looking for. Operator-flagged: it needs a path someone can walk to. Settings is where you go to ask the instance about itself, so the view becomes a tab there, beside Activity — Activity answers "what is the queue doing", System answers "is anything left to do it". - SystemView.vue moves to components/settings/SystemHealthTab.vue; the content is unchanged apart from shedding its own container and h1. - SettingsView adopts useTabQuery (the composable Browse and Subscriptions already use) so a tab can be linked TO. The health dot now points at ?tab=system, and /system redirects there so the previous build's link and any bookmark still land. - The tab drops its own 10s poll. v-window keeps a visited item mounted rather than destroyed, so that timer would have gone on firing behind Maintenance — and TopNav already polls the same store every 15s for the dot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 19:45:54 -04:00
bvandeusen	a8fdbd86bc	Merge pull request 'FabledCurator can now tell you one of its own parts has stopped' (#250) from dev into main	2026-09-02 18:01:38 -04:00
bvandeusenandClaude Opus 5	5084ba666b	feat: the dot beside the brand now means the whole stack (milestone 365 step 4) ... The ask was a surface AND a path. The path is the part that was missing — everything that could answer "is it running" lived inside Settings, which you only open once you already suspect something. **Re-used the indicator that already existed rather than adding a fourth.** There were three partial surfaces: TopNav's health dot, PipelineStatusChip's pulse, and the Settings Activity tab. None answered "is every part alive", and a fourth would have made the question harder to answer, not easier. TopNav's dot read /api/health — a no-DB liveness check proving only that the WEB container is serving. Green there while a worker was dead is exactly what it looked like, and a green dot beside the product name gets read as "everything is fine". It now reflects the whole-stack verdict, and it is a link: the place someone already looks when they suspect something is now also the way to the detail. The tooltip names the actual problem. "Scheduler has not checked in for 6 min" sends someone somewhere; "something is unhealthy" sends them hunting. /system is deliberately NOT in the nav row — TopNav builds that from routes with a meta.title, and a sixth top-level tab for a page visited twice a year costs more attention than it returns. It is reached from the dot. The page lists every learned part with its state as a sentence rather than a chip, and prints the staleness thresholds it was judged by, taken from the endpoint so the UI keeps no second copy of them. PipelineStatusChip still hand-rolls its own 3-minute scheduler window; that is now a duplicate of a threshold the server owns, and worth collapsing once this has been watched working. The stores stay separate on purpose: system.js is "can I reach the API", systemActivity.js is "what is the pipeline doing", systemHealth.js is "is anything broken". Running and alive fail independently — an idle stack with a dead worker looks identical to a healthy one on every activity surface, which is the whole reason this milestone exists. Not yet verified against a real stopped service. Rule 12 keeps a local stack out of it, and frontend CI has no Vue type-check or visual regression, so this needs an operator look rather than a green lane. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 17:20:55 -04:00
bvandeusenandClaude Opus 5	fe4e0f2b71	feat: /api/system/health — one verdict for the whole stack (milestone 365 step 3) ... The single endpoint the nav indicator and the System page will both read. Composing a verdict is this module's job, not the UI's. Two kinds of part, answered differently. LEARNED — celery roles and the GPU agent, out of service_seen, where the question is "how long since it checked in" and the answer can be "it has not". PROBED — Postgres and Redis, always expected, never learned, because a last-seen for them would be actively misleading: that Redis answered thirty seconds ago says nothing about now. **The endpoint must never fail because something it checks has failed.** That inversion is easy to write by accident and it destroys the feature exactly when it is needed — a 500 when Redis is down instead of `redis: down`. Every probe is wrapped, every wait carries a deadline (rule 156), and the roster refresh swallows its own errors. The worst case is a part reported `unknown`, which is a true statement about the system. Postgres is probed first and gates the rest, because if it is unreachable nothing else can be read — and "the database is down" is the most useful single thing this can ever say. The staleness thresholds are the design risk, not the code, and they are deliberately generous: 90s to doubt, 300s to disbelieve. The constraint is a deploy rather than a crash — `docker compose up -d` rolls start-first, so a role is briefly served by two containers and then by neither while the old one drains. Thresholds tight enough to catch a crash in seconds would paint the page red on every update, and an alarm that cries wolf on every deploy is one nobody reads. Tune down only after watching a real deploy pass through. The numbers ship in the response so the UI can explain a `stale` without keeping a second copy of them. States are described in sentences rather than left as chips: "Scheduler has not checked in for 6 min — treat it as stopped" is what someone needs at the moment they are deciding whether to go and open Portainer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 17:17:22 -04:00
bvandeusenandClaude Opus 5	dc8af8b1a7	feat: a learned roster, so a stopped part is observable (milestone 365 steps 1-2) ... Nothing in FabledCurator knew what was SUPPOSED to be running. `celery inspect` reports the workers that ANSWER, so a dead worker was a shorter list rather than a red light, and grep for any notion of expected services returned nothing. That is why Portainer was the only place an operator could see it: Portainer knows the intended set. `service_seen` is the memory that makes an absence observable — every part that has checked in, and when it last did. **Keyed on the queue set, not the worker hostname.** Celery's worker names here are `celery@<container id>`, minted fresh on every deploy. Keyed on those, this table would record a death and a birth every time the stack updates — and a status page that goes red on every deploy is a status page nobody reads, which is worse than not having one. CELERY_QUEUES is assigned per role in compose and survives container replacement, so it is the stable identity. Two replicas of a role are therefore ONE row, which is right: the question is whether the role is served, not how many containers exist. The GPU agent is keyed on agent_id, the identity its lease protocol already uses. gpu.py received it on both lease and heartbeat and threw it away — an idle agent with nothing to lease left no trace and was indistinguishable from one switched off a week ago. Now recorded on the calls that were already happening. **Who observes, corrected from the plan.** The plan said "record from the existing inspect path", which would only run when someone opened the Activity tab. Two other candidates and why they lost: - A beat sweep. If the scheduler dies the sweep stops, every row goes stale, and the page says everything is down when one thing is. An alarm that cannot distinguish "a part died" from "the observer died" is worse than none. - A background task in web. hypercorn runs --workers 4, so that is four concurrent inspect loops per container, forever. Taken instead: refresh on demand, rate-limited by the newest last_seen_at that every process can already see. The observer is then the thing serving the page — if web is down you get a browser error, not a confidently green page — and it self-limits with no coordination, since a race costs one redundant inspect that writes identical values. Migration 0090 is the first written on the collapsed baseline (milestone 328), so it is also the first evidence the chain steps FORWARD from 0089 rather than merely reproducing the schema. No secondary indexes: one row per moving part means every read is a handful of rows, and #3301 is the record of what speculative indexes cost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 17:15:46 -04:00
bvandeusen	0421fd3109	Merge pull request 'The weekly refresh now publishes only what a gate has proven — and a new install can start' (#249) from dev into main	2026-09-02 16:31:55 -04:00
bvandeusenandClaude Opus 5	131237143b	Revert "test: force the smoke gate to fail, to watch it block a publish" ... The gate held. Run 5320, dispatched with the forced failure in place: build-web success (candidate published) build-ml success build-agent success smoke-web FAILED promote skipped run failure And the three channel tags did not move: fabledcurator 33d3d8332f74 -> 33d3d8332f74 fabledcurator-ml e94a5435cb45 -> e94a5435cb45 fabledcurator-agent bae27d34d811 -> bae27d34d811 So a refresh that breaks something now leaves :latest naming the build that works, which is the property milestone 362 exists to establish. The rejected candidate is still published under :refresh-candidate, so whoever reads the red job on Monday can pull the exact image that failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:23:49 -04:00
bvandeusenandClaude Opus 5	59d27ef76e	test: force the smoke gate to fail, to watch it block a publish ... TEMPORARY, reverted in the next commit. Milestone 362's verification section requires the gate to be seen rejecting a build — a gate nobody has watched reject anything is a gate nobody knows is wired up. Every real check passes, so the rejection has to be forced. Under test is the job dependency, not the assertions: a failed smoke-web must skip the promote job, and the three :latest tags must still name the digests they named before the run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:22:01 -04:00
bvandeusenandClaude Opus 5	f630e50e75	ci: the refresh publishes only what the gate passed (#3265 milestone step 4) ... The gate reported a verdict nothing consulted. Now it decides. The promote moved out of the three build jobs into its own `promote` job, because the verdict cannot exist until build-web has finished and the promote used to run inside it. `needs: [build-web, build-ml, build-agent, smoke-web]` is the whole mechanism: a failed smoke skips the promote, so a refresh that broke something leaves :latest naming the build that works. "The refresh failed" and "production is broken" must not be the same event. A SKIPPED smoke also skips it, and that is the case that matters most. On run 5290 the gate silently skipped itself — job-level `if:` cannot read the env context — and a design where only a FAILED gate blocks would have published unverified images while reporting success. Not running is not the same as passing, and today produced two separate bugs of exactly that shape (#3414, and the smoke-web skip). All three images now promote together or not at all. They are one stack: build.yml already refuses to publish a :dev web image beside a stale :dev ml because the mismatch only surfaces as a runtime failure, and a refresh that published ml while withholding web would be that same trap reached through the gate. Stated plainly in the job comment: the gate covers web only, so ml and agent are held to web's verdict rather than their own. That is the conservative direction, not equivalent evidence, and should not be read as if it were. Three near-identical promote steps collapsed into one loop. A partial failure now says which images moved and that the state is inconsistent, rather than leaving that to be inferred — the promote is idempotent and the candidates are still published, so the instruction is simply to re-run. Also removed the now-dead `promote` output from the ml and agent reuse steps. Only build-web's is read (as outputs.candidate); two more copies nothing consults is the kind of thing that reads as load-bearing a year later. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:21:27 -04:00
bvandeusenandClaude Opus 5	86abaf0b94	docs: a new install could not start, and nothing told anyone why (#3422) ... The install path milestone 328 wrote produces a web container that exits on boot. entrypoint.sh runs alembic, then app construction raises: MissingCredentialKey: Fernet key file not found at /images/secrets/credential_key.b64. For first-time setup, set CURATOR_BOOTSTRAP_NEW_KEY=1. That variable appeared in no README, no .env.example and no compose file — only in backend/. So a stranger following the documented steps got an app that does not start and an error with no context. Found by the milestone-362 smoke gate on its first real run (#3422). The product behaviour stays exactly as it is. credential_crypto refuses to mint a key because the 2026-06-02 audit found a partial restore — database back, ./images/secrets lost — silently generating a fresh one and producing a healthy-looking instance where every authenticated download failed AUTH_ERROR. Failing fast is right; not saying so is the bug. So: .env.example carries the variable in its own FIRST BOOT ONLY section with the reasoning and an instruction to delete the line afterwards, and README's First run leads with it, because "the app will not start" belongs before "the ML worker downloads weights". Both say to back up ./images/secrets/ alongside the database, which is the part that costs real data if it is learned late. **compose had to change too, and this is the part that would have shipped a second broken instruction.** A variable in `.env` is only used for ${...} interpolation — it does not reach the container unless the service names it. Telling people to set it in .env, without that, would have documented a step that does nothing. Added to the shared app_env anchor, defaulted to empty so the refusal still stands for everyone who has not opted in. Not taken: auto-bootstrapping when the credential table is empty, which would remove the manual step entirely and keep the audit's protection for restores. That is the better product and it is a code change with a predicate that has to be exactly right; this is the smallest correct fix, and #3422 stays open for the other one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:19:28 -04:00
bvandeusenandClaude Opus 5	4815040d74	ci: the smoke gate found a real one on its first run — and had two bugs of its own ... Run 5296 was `smoke-web`'s first genuine execution. Checks 1 and 2 passed: alembic built the schema from empty inside the image, all five apt binaries resolved, and the application's own Thumbnailer produced JPEG, PNG-with-alpha, WebP and an ffmpeg video frame against the image's libraries. Check 3 failed, and the trap's log dump said exactly why: MissingCredentialKey: Fernet key file not found at /images/secrets/credential_key.b64. For first-time setup, set CURATOR_BOOTSTRAP_NEW_KEY=1. That is the product being right. credential_crypto refuses to mint a key unless someone opts in, because the 2026-06-02 audit found a partial restore (DB back, /images/secrets/ lost) silently generating a fresh one and leaving a working-looking system where every authenticated download failed AUTH_ERROR. It is also a first-run blocker for milestone 328, filed as #3422: the variable appears in no README, no .env.example and no compose file, so the install path that milestone just finished writing produces a container that exits on boot. Not fixed here — the fix trades safety against friction and is the operator's call. Two defects in the gate itself, both surfaced by the same run: - A throwaway CI instance IS first-time setup, so it now passes CURATOR_BOOTSTRAP_NEW_KEY=1. The check was asserting a condition no fresh container can satisfy. - The health loop polled a dead container for 3m35s. Docker had already recycled its IP, so the replies were a baffling mix of connection-refused and 5s timeouts from whatever took the address next. It now checks `.State.Running` each iteration and fails immediately with the container's log. The trap had the real answer the whole time; this stops burying it under four minutes of noise. Also corrected a message claiming a 120s budget: 60 iterations of up to 5s connect plus 2s sleep is nearer seven minutes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 16:00:18 -04:00
bvandeusenandClaude Opus 5	81b7b6f308	ci: smoke-web never ran — a job's if: cannot read the env context ... Run 5290 dispatched a refresh. Everything worked: the guard fired, the build published the candidate, the promote pointed :latest at it. And `smoke-web` reported conclusion "skipped", with no steps and no log. Its condition was `if: env.IS_REFRESH == 'true'`. The env context is available to STEP conditions and step bodies but never to a job's own `if:`, and an unresolvable context there evaluates to empty rather than erroring. So the gate skipped itself, silently, on the one run that existed to exercise it. Second silent-skip of this family today, after #3414. Same shape both times: something evaluated false, nothing failed, and the run reported success. It is worth naming the pattern — on this pipeline, "green" and "ran" are different claims, and the steps' own conclusions are the only place the difference shows. Fixed by keying off a job output rather than re-deriving the trigger: build-web now exposes the reuse step's `promote` decision as `outputs.candidate` and smoke-web consumes it. That is better than duplicating the expression: it is the same single decision the build, the XPI download and the promote all take already — build.yml's own "one decision drives everything downstream" — and it asserts the thing smoke-web actually depends on, that a candidate was published, rather than restating the reason one would be. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 15:53:40 -04:00
bvandeusen	d01f33dea6	Merge pull request 'A gate for the weekly refresh, and the lever that makes it testable' (#248) from dev into main	2026-09-02 15:51:50 -04:00
bvandeusenandClaude Opus 5	bfa9fd678b	ci: smoke the refreshed image against real Postgres and Redis ... Milestone 362 step 3. This is the gate the weekly base refresh never had. `ci.yml` cannot be that gate, and the reason matters more than the fix. Its lanes run on ci-python:3.14 and install requirements.txt — a base refresh changes neither, so all five stay green through a bump that breaks the product. What a refresh re-resolves is the Dockerfile's apt layer: ffmpeg unar libpq5 postgresql-client zstd megatools libjpeg62-turbo libwebp7 libpng16-16 ca-certificates Unpinned, every build, and nothing else in this repo looks at it. That line is the dependency creep; it is also precisely what the test suite structurally cannot observe, since the suite never runs inside the image and the image carries no tests and no pytest. So `smoke-web` runs the CANDIDATE IMAGE against real service containers: 1. `alembic upgrade head` on an empty database — the image's own libpq and psycopg, and the same call entrypoint.sh makes before it serves anything, so a failure here is a failure to boot. 2. The apt binaries, then the application's own `Thumbnailer` — JPEG, PNG with alpha, WebP, and a video frame through ffmpeg. `Thumbnailer` needs no database and no app context, so the check exercises real product code rather than a proxy for it. `ffmpeg -version` exiting 0 would pass while a codec removal broke every thumbnail in the library. 3. The web role boots and answers /api/health. Every failure names the package it implicates. This fires on a Sunday, unattended, about a change nobody made deliberately — "assertion failed" a week later teaches nobody anything. The script is piped over stdin rather than bind-mounted: the workspace is a docker volume belonging to the job's own container, so a host bind of $PWD does not resolve for a sibling. Container logs are dumped only on failure, and the trap re-exits with the real status rather than the status of `docker rm`. Deliberately NOT gating the promote yet — that is step 4. Landing the gate and the thing it gates together would mean the first time anyone saw this job run would also be the first time it could stop a publish. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 15:45:10 -04:00
bvandeusenandClaude Opus 5	24a2b70a5a	ci: a boolean input never equals the string 'true' ... The refresh lever did not work, and the way it did not work is the point. Run 5270 dispatched with refresh=true. Its log: expression '(github.event_name == 'schedule' || github.event.inputs.refresh == 'true') && 'true' || 'false'' evaluated to '%!t(string=false)' trigger: event=workflow_dispatch IS_REFRESH='false' BUILD_REF='refs/heads/dev' trigger: raw inputs refresh='true' force_build='false' The input arrived as true and the comparison still said false. `type: boolean` delivers a real boolean, and GitHub expression semantics cast operands to numbers when their types differ — so `true == 'true'` compares 1 against NaN. My comment on the previous commit asserted the opposite, that Forgejo delivers inputs as strings, and asserted it without checking. The run went GREEN with every step skipped, because a refresh that evaluates false is indistinguishable from an ordinary push. A lever that silently does nothing is worse than no lever: it would have been trusted. Normalised through format(), which is representation-independent — a boolean true and a string 'true' both render 'true'. That is also why force_build was never bitten: it passes its raw value into an env var and compares in the shell, where everything is a string already. format() buys the same thing at expression level, which is where a step `if:` needs the answer. The diagnostic from the previous commit stays. It is what turned this from a guess into a measurement, and it is the only thing that would catch the same class of failure next time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 14:59:48 -04:00
bvandeusenandClaude Opus 5	2c88ad3efb	ci: report the raw and normalised trigger values ... The refresh dispatch on run 5265 went green with every step skipped: the main-only guard did not fire, checkout took dev, and the reuse step read IS_REFRESH as false. So both workflow-level expressions evaluated false while the identical accessor works for force_build, which compares its value in the shell rather than in an expression. That is a guess until it is measured, and the failure is silent by construction — a refresh that evaluates false behaves exactly like an ordinary push and reports success. This prints the raw input beside the normalised value in the step that already exists to say what a run derived, so the two disagreeing is visible rather than inferred. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 14:58:20 -04:00
bvandeusen	adab33694d	Merge pull request 'Make a digest mean something again, and give the refresh somewhere to stand' (#247) from dev into main	2026-09-02 14:56:12 -04:00
bvandeusenandClaude Opus 5	bfc4f9cec9	ci: one fact for "is this a base refresh", and a lever to trigger one ... Milestone 362, enabling step 2's verification and everything after it. The weekly refresh was testable once a week. That is not a cadence anything can be developed against, and milestone 362's whole point is a gate — which has to be watched rejecting something before anyone can believe it is wired up. So `refresh` joins `force_build` as a dispatch input, on the same reasoning that added that one (#3252: confirm #3190 was gone rather than wait for it to recur). Adding it meant confronting that "is this a refresh?" was asked in five places and spelled five ways: `github.event_name == 'schedule'` in an `if:`, `$GITHUB_EVENT_NAME` in one shell, an `EVENT:` env passed into another, and a bare expression on `pull:`. Five spellings of one fact is how half of them come to disagree once somebody adds a sixth trigger — which is precisely what this commit is. So it is derived once at the top, next to BUILD_REF, which already exists for exactly this reason on exactly this question. String comparison, not boolean: Forgejo delivers dispatch inputs as strings, so `inputs.refresh` is 'true'/'false' and `&&` on it would read the string 'false' as truthy. **A constraint this makes visible, which pre-dates it.** A refresh checks out `main` (BUILD_REF) while running the workflow definition from the branch that triggered it — the cron registers from the default branch. So dev's workflow builds main's source, and dev's workflow cannot depend on anything main's tree does not have yet. It does now: the reuse step calls `artifacts.sh epoch`, which lands on main with this batch. Until then a refresh dispatch fails loudly at that call, which is the right failure — the alternative is tolerating a missing epoch and silently rebuilding #3265 into every refresh. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 14:41:33 -04:00
bvandeusenandClaude Opus 5	b590d25f8f	ci: the scheduled refresh builds a candidate, then names the channel ... Milestone 362 step 2. Structural: it creates a moment between "built" and "published" for step 3's gate to occupy. No behaviour change. A refresh rebuilds against freshly resolved base images, and the web image's runtime is a line of UNPINNED Debian packages — ffmpeg, libjpeg62-turbo, libpq5, megatools — re-resolved on every build. Nothing in ci.yml can see that: its lanes run on ci-python:3.14 and install requirements.txt, and a base bump changes neither. So refreshed bytes need proving before :latest names them, and proving needs somewhere to stand. On a push nothing changes: build_ref IS channel_ref, promote is false, and the build writes the channel tag directly the way it always has. On the schedule the build writes :refresh-candidate — one moving ref per image, overwritten in place, holding a build nobody is told to pull. That is the shape rule 145 already allows for :buildcache, not the per-build tag family 318 withdrew. Both values are decided in the reuse step beside `hit`, because that step already owns "what does this job do" (build.yml's own rule, at the force branch). A promote condition derived somewhere else could disagree with the tag the build actually wrote. **The promote is a manifest PUT, not `imagetools create`.** That distinction is the whole risk in this change. `imagetools create` wraps its source in an index, and an indexed channel tag is the one thing this pipeline cannot survive: `.Image.Config.Labels` does not resolve through an index, so the fc.revision the reuse check reads back would come up empty, every later push would miss and rebuild, and nothing would go red. That is #3183, observed on run 4751 — reuse worked exactly once and the only symptom was the bill. The repoint step already excludes its own source tag for this reason; a promote that re-introduced the wrap through another door would undo that care. A manifest PUT is what "make this tag name that image" means at the registry: same bytes, same media type, identical digest, no layer transfer. It reads the result back and fails if the tag does not name what was just written — a PUT that 2xx'd and landed something else is exactly the silent-and-plausible failure this pipeline keeps producing. Every call carries a deadline (rule 156); a registry that stops answering must fail the step, not hang the weekly refresh until the job times out. Promote is UNCONDITIONAL today, deliberately. Gating it before the gate exists would leave the refresh building something and publishing nothing for as long as this milestone takes. Step 4 wraps it in the smoke suite's verdict. Not yet verified on the refresh path — that needs a scheduled run, and the lever to trigger one on demand is the next commit. This one is verified by the push path being untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 14:40:19 -04:00
bvandeusenandClaude Opus 5	635138b0d1	ci: pin the build clock to the commit, so an unchanged refresh publishes nothing ... Milestone 362 step 1, closing #3265's root cause. The weekly base refresh rewrote all three `:latest` tags on 2026-08-30 with nothing changed in any of them. Not a cache miss — run 4934's log shows every content step CACHED and both bases resolved to unchanged pinned digests. buildkit stamps the image config with the wall clock of the build, so identical layers get republished under a new config blob and therefore a new manifest digest. The cost is not storage, it is meaning: `:latest` moved on a calendar, so a digest change stopped being evidence that anything was different. That is the one thing a digest is any use for, and it is load-bearing here — the reuse check, the `:c-<sha>` rollback story and any future redeploy signal all rest on it. SOURCE_DATE_EPOCH normalises `created` and the history timestamps, so the same source produces the same config bytes and the same digest, and pushing it is a registry no-op. The value is routed through artifacts.sh's existing `newest()` rather than taken from git separately. `revision`, `version` and now `epoch` are three fields of ONE lookup, so they cannot drift into naming different commits — a divergence that would stamp an image reproducibly against one commit while it reported being another, with both values looking perfectly well-formed. Note #3127 §2 is the record of what a second clock costs; this adds a view, not a clock. Also corrected: the build step comment and ci-requirements.md both described the churn as current behaviour with the fix as a "likely" future. They now describe what the file does. Tests pin the property the fix depends on, not the fix: epoch is the same commit version names, in both renderings including the extension's unpadded one, and it does not move between two calls on one checkout. A future refactor that gave epoch its own `git log` would pass every other test in that file. Not yet verified end to end — proving it needs two consecutive refreshes to land on the same digest, which is the next thing, and is the step #3265 exists because nobody did last time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 13:46:16 -04:00
bvandeusenandClaude Opus 5	c0370069e0	release: the first release describes the product; it has nothing to diff against ... Step 7 needs a release that reads as "what is FabledCurator and how do I run it". What the script would actually have published is "changes since v26.06.04.0" over 533 commits, truncated to 200 — a release page whose first screen is the internal build-out that milestone 328 exists to stop shipping, addressed to a reader who has never seen this project. Two causes, fixed separately. **A pre-convention tag is history, not a predecessor.** The 28 `v26.*` tags were kept when their releases were deleted, so `--match v*` walks ancestry straight back to one of them. Reachable is not comparable: nobody has run v26.06.04.0 and its release page no longer exists to compare against. The match is now `v[0-9][0-9][0-9][0-9].*` — rule 148's shape, which is exactly the set of tags naming a release a reader could have been running. **With that narrowed, the first rule-148 tag reaches no predecessor**, and the old fallback — diff against the whole history — is worse than the problem it replaced. A release with no predecessor now renders the product overview and no commit list at all. The overview is READ OUT OF README.md between `<!-- overview:start -->` and `<!-- overview:end -->`, not written into the script, for the same reason the changelog is derived: two hand-maintained descriptions of one product drift and nothing ever catches it. The release page and the repo front page are one source. Missing markers are reported as a note and publish anyway, on cross_checks()'s reasoning — the release is still the useful object. Every later release goes back to being a changelog, which is what note #3127 §5 says a release is for. MAX_COMMITS still guards the case it now guards: two real releases far enough apart that the list stops being readable. Also corrected while marking up the README: "Importing — ingests an existing library from disk" was still advertising the folder-import feature that 3590c47 documented as deliberately retired. Replaced with what FC actually does with what arrives — content-hash dedup, sidecar metadata, provenance. Tests: the two that asserted the old no-predecessor behaviour are rewritten rather than left; synthetic repos now carry their own copy of the script, since the overview resolves relative to `__file__` (correct in production, where release.yml checks out the tag) and would otherwise have every fixture silently quoting FabledCurator's real README. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 12:28:48 -04:00
bvandeusen	3e4d39b111	Merge pull request 'The install path a stranger reads, and a CI probe that never probed' (#246) from dev into main	2026-09-02 11:25:28 -04:00
bvandeusenandClaude Opus 5	3590c478f5	docs+ci: folder import stays retired, and fix a readiness probe that never probed ... Two unrelated things, both found while closing out milestone 328. **Folder import (#3367).** The operator's call, this session: the import-from-file surface was abandoned on purpose and is not coming back — "it has its own complexities that we didn't need." The README and the compose comment both described the missing button as a rough edge with a tracking issue, which promised a fix that is not coming. Both now say the retirement is the decision, name Subscriptions as the supported way to fill a new install, and describe /api/import/trigger as an unsupported escape hatch for anyone who wants to script one. **The CI readiness probe.** ci.yml's integration job and baseline.yml both waited for Postgres with `(echo > /dev/tcp/$PG_IP/5432)`. Those steps run under `sh -e` — act's default shell — where /dev/tcp is not a magic path but a filename that does not exist. The probe could therefore never succeed: run 18035, a GREEN run, spends 05:20:53 → 05:22:53 in that loop and exits it by exhaustion, not by connecting. Every integration run has been paying a flat 120s for a check that established nothing, and proceeding regardless. Replaced with a socket connect in python (present in the image, no package needed), and exhausting the budget is now a named failure instead of a silent fall-through — rule 156. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TTjbZZ6JirCMSaJzQV1RhA	2026-09-02 00:51:25 -04:00
bvandeusenandClaude Opus 5	8a4af589f1	docs: write the install path for someone who is not the operator (#3271) ... FC has no login — no User model, no session auth, nothing. That was a deliberate call for a single-operator tool and it stays (operator, this session), but it was nowhere in the docs, and the app stores live Patreon / SubscribeStar / Pixiv session cookies on accounts that carry a payment method. Anyone standing this up from the README could reasonably have put it behind a TLS-terminating proxy and considered it handled. So the no-auth posture is now stated three times, in the three places someone decides where to bind the port: README has a "Before you expose it" section above the install instructions, .env.example explains why there is no auth variable in it, and the compose header says it before the first service. SECURITY.md claimed the opposite. It listed "a multi-user sharing ACL — instances can be shared" among the things worth protecting; there are no accounts to share between. That was rule 47 applied to a codebase that does not implement it, and it would have told a researcher FC holds a boundary it does not. Replaced with the real posture, including that TLS without an authenticating layer in front changes nothing. Also corrected, all of it stale rather than wrong-at-the-time: - EXTENSION_API_KEY was dead config. config.py read it into a field nothing consumed; the real key is generated into app_setting on first use and managed in the UI. Removed from config.py, compose and .env.example. - .env.example pointed at docs/superpowers/specs/… — there is no docs/ dir — and described the extension key as "lands in FC-3", closed 2026-05-21. - The /import mount comment described an FC-5 ImageRepo migration run from "Settings → Maintenance → Legacy migration", a surface with no frontend. - README said the extension installs from Settings → Maintenance. It is on Subscriptions → Settings. README is now split: running FC above the line, developing FC below it, with requirements, first run, the extension, upgrading and troubleshooting on the running side. First run documents the one real gap it found — a new installer with a library on disk has no button to import it, only POST /api/import/trigger, because the manual-scan UI was retired 2026-07-02 when that stopped mattering for an established install. Filed as #3367. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-09-02 00:16:23 -04:00
bvandeusen	b93a5fc5b4	Merge pull request 'Collapse alembic 0001..0089 into one baseline' (#245) from dev into main	2026-09-01 01:30:52 -04:00
bvandeusenandClaude Opus 5	aa71cbbdbf	db: the baseline was missing the three system-tag seeds (#3266) ... Integration caught it: 36 tests failing with NoResultFound, all on _system_tag(db, "banner") and its siblings. 0075 seeds three hygiene system tags — wip, banner, editor screenshot — and the first version of the baseline carried only the two settings singletons. This is the same defect class the baseline's own docstring warns about, which I then walked into anyway. The reason is worth recording: my scan for data statements used a regex requiring INSERT to sit immediately after the opening quote, so it saw op.execute("INSERT INTO ml_settings (id) VALUES (1)") and missed 0075, which builds the statement through sa.text() across several lines with bound parameters. The narrow pattern found two of three seeds and reported itself complete. The wider scan — grep for insert/bulk_insert across every revision in 725bf15 — turns up six data-writing migrations, and they separate mechanically: INSERT ... VALUES (literals) = SEED. Product data. Carry it. 0002 import_settings, 0003 ml_settings, 0075 system tags INSERT ... SELECT ... FROM tbl = BACKFILL. Derives from existing rows, inserts nothing on an empty database, correctly omitted. 0034 artist_visit, 0040 and 0047 series_chapter That rule is now in the docstring, because the next person collapsing a chain needs the rule more than they need the answer. 0075's adopt-before-insert guard is kept as WHERE NOT EXISTS. It cannot fire on the empty database this file runs against — it existed because an operator might already have hand-tagged `wip` — but it makes the statement re-runnable for free. Worth stating plainly: baseline.yml passed on the version without these rows, and would pass again. It compares schema, and a baseline missing every seed still produces a byte-identical schema. The integration suite is what caught this, which is the argument for the first-run check that #3271 should carry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-09-01 01:20:42 -04:00
bvandeusenandClaude Opus 5	973db73221	db: collapse alembic 0001..0089 into one baseline (#3266) ... 89 files and 6,300 lines become one file of 807. Nothing about the resulting schema changes; what goes away is the requirement that a new installation replay our development history to arrive at it. revision = "0089", down_revision = None. That pairing IS the migration strategy for existing installs, not a detail of it: a deployed database already has alembic_version = '0089' from running the real 0089, so alembic reads the version table, sees head reached, and does nothing. No stamp is required — which matters, because `alembic stamp` writes a version string without validating anything about the schema it is writing it against, and a wrong stamp is indistinguishable from a right one until the next migration fails. An empty database runs the file and records 0089. Both paths converge. The next migration is 0090, as it would have been; the numbering is continuous across the collapse on purpose. Autogenerate produced nearly all of this unaided, which was NOT true of the first attempt — that one was reverted because the generator silently dropped eleven indexes and three uniqueness guarantees. #3275 put those on the models first, so the HNSW index with its opclass, the COALESCE expression index, the partial uniques, 107 server_defaults and the enum CHECKs are all emitted now. Doing the reconciliation before the squash, rather than after, is what made this work. Hand-added, because none of it can live in a model: * CREATE EXTENSION vector / tsm_system_rows (0001, 0004) — database objects, not table metadata. * The pgvector import. Autogenerate writes qualified pgvector.sqlalchemy.vector.VECTOR references without importing the package, so its own output cannot run (run 4988). * THE TWO SEED ROWS. 0002 and 0003 did not only build schema — each inserted a settings singleton, and nothing in the app ever creates them: ImportSettings.load() and MLSettings.load() are select(...).scalar_one(), which RAISES NoResultFound rather than returning None. A models-only baseline would leave both tables empty and crash a fresh install on first settings access, while baseline.yml reported a perfect schema match. Only running the app against a new database finds that. Not carried over: 0023's DELETE FROM tag and 0047's series deletes, which are historical cleanups operating on rows an empty database lacks. downgrade() raises. A baseline's downgrade is "drop every table", which is a data-loss event wearing a migration as a disguise; offering it as one invites someone to run it. Restore from a backup. Also removed, per the plan: the 10 test_migration_*.py files (they assert intermediate states and backfills that no longer exist — a test that a column exists is already the model tests' job) and backend/app/utils/artist_backfill.py, whose only importer was 0008. Verified no other consumer anywhere in backend/ or tests/. baseline.yml changes with it. chain_ref now DEFAULTS to 725bf15, since the tree no longer carries a chain to compare against — that pinned commit is the last one that does. And the CheckConstraint repair is removed, because it never fired. I added it claiming autogenerate re-doubles a constraint name on the round trip and asserted in b979062 that it was "still correct and still needed". It is not: autogenerate wraps names in op.f(), which marks them already-formatted and blocks the convention from re-applying. Tested against the real candidate line — the regex matches nothing. What actually fixed the mismatch was 0088's renames alone. The doubling is a real hazard, but of hand-writing a pre-prefixed name, not of the generator; the comment asserting otherwise was worse than the dead code under it. The header comment is rewritten for the same reason — it described 87 revisions, and claimed the HNSW index could not be expressed in a model, which #3275 disproved. It now also states plainly what this check CANNOT see: it compares schema, so a green run means the schema is right, not that the baseline is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-09-01 01:14:37 -04:00
bvandeusen	725bf15f88	Merge pull request 'AGPL-3.0, SECURITY and CONTRIBUTING; the install path pulls :latest, not :dev' (#244) from dev into main	2026-08-31 17:01:20 -04:00
bvandeusenandClaude Opus 5	bc4eba636d	fix: the documented install path pulled :dev, not :latest (#3270) ... docker-compose.yml pinned :dev on all five app services — web, worker, scheduler, maintenance-long, ml-worker. The README documents `docker compose -f docker-compose.yml up -d` as the production path, and -f means "use only this file", skipping the override and its build: directives. So Compose pulled image:, and image: was the rolling development channel. The documented way to install this product shipped development builds. It went unnoticed for a structural reason rather than a careless one: nobody who works on the project takes that path. The operator deploys from a swarm stack file; contributors get docker-compose.override.yml, which sets build: for all five services, and build: wins over image:. The broken path is reachable only by a stranger following the README — which is exactly the audience that did not exist until now. :latest, per rule 147: main IS production. It is also what the agent stack (agent/docker-compose.yml) already pinned, so this makes the two stacks agree rather than introducing a new convention. Both paths verified with `docker compose config`, which merges and prints without starting anything: dev path — build: present on all five, image: not pulled -f production — 0 build: directives, five :latest images resolved Also checked the base file for anything a stranger could not satisfy: no host-absolute volume paths, no operator-specific port bindings, no device mappings. The tag was the only defect in the consumer path. The comment on web.image is deliberately long (rule 32). A line reading :latest inside a file a developer is debugging with is exactly the line someone flips back to :dev to test something and then commits, and the consequence — strangers silently installing bleeding edge — is invisible to everyone who works here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 16:55:43 -04:00
bvandeusenandClaude Opus 5	dbc4e8b0c6	docs: AGPL-3.0, plus SECURITY and CONTRIBUTING (#3269) ... The repo had no LICENSE, which meant all rights reserved by default: nobody could legally run or modify it, and "public availability" was a contradiction no amount of install documentation could fix. This is the hard blocker in milestone 328; everything else in it is quality. AGPL-3.0, at the operator's explicit choice, for the reason the operator gave: this should not become something another party runs as a hosted proprietary service. Section 13 is what makes it fit — for a self-hosted web app, distribution otherwise never happens, so the GPL's obligation would never actually bite. AGPL reaches the case that matters here: running a MODIFIED copy as a service for others. LICENSE is the FSF text fetched from gnu.org and verified byte-identical (34,523 bytes, 661 lines, §13 "Remote Network Interaction" present), not retyped. README's old "Personal project; use at your own discretion" said nothing legally and is replaced with what the licence actually asks — including the part worth being clear about, that running an unmodified copy for yourself carries no obligation whatsoever. SECURITY.md names what this software actually holds, because that is what makes a report serious here: live third-party session cookies for accounts with payment methods attached, the extension API key, the multi-user sharing ACL, and arbitrary downloaded media that gets decoded and fed to models. It also states the plain-HTTP posture up front, so "served over HTTP" and "no HSTS" are understood as the documented design rather than filed as findings. There is no private disclosure channel yet, so the reporting instruction is to open an issue containing NOTHING but the fact that a report exists, and wait for a private contact. Awkward on purpose: an issue tracker is public the moment it is written to, and every self-hosted instance stays vulnerable until its operator can update. Worth replacing with a real contact address — that decision is the operator's, since it publishes one. CONTRIBUTING records the two things that actually catch people: ruff's order-by-type import sorting, and that a model change and its migration belong in the same commit. The second is not style — the models and the chain silently diverged for a long time (#3275) and autogenerate was unsafe as a result. Pre-publication scan, since the repo is about to get attention: .env.example is placeholders only (`changeme_*`), `.env` is gitignored with an `!.env.example` exception, no credential-shaped literals are committed, and there are no private IPs or operator home paths. The only hostnames are the project's own forge, which is public by design. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 14:53:06 -04:00
bvandeusen	b14818303c	Merge pull request 'Schema reconciliation + index hygiene, and the weekly base-image refresh' (#243) from dev into main	2026-08-31 08:34:55 -04:00
bvandeusenandClaude Opus 5	08418d54a3	db: index the seven unindexed FKs, drop the seven redundant ones (#3300, #3301) ... A structural sweep of the deployed schema, run AFTER 0088 got the models and the chain to exact agreement. That agreement is what 0088 achieved, and it is worth naming what it does not prove: a models-vs-chain diff shows the two describe the same schema, not that the schema is right. Everything here was wrong in BOTH. The one that matters: image_tag has PRIMARY KEY (image_record_id, tag_id) and no other index, so tag_id is unindexed. That is the gallery's tag filter (tag_query.py builds `image_tag.c.tag_id == tid`) and the ON DELETE CASCADE from tag, both scanning the largest table in the schema. Six more FKs were unindexed on smaller tables; presentation_review.tag_id also CASCADEs. Dropped, on the other side: ix_image_record_sha256 was an exact duplicate of the index uq_image_record_sha256 already builds — two btrees on the same column of the highest-insert-rate table. The other six are single-column indexes a later composite superseded without the narrow one being retired; a btree on (a,b) already serves lookups on a. 0088 deliberately taught the models to declare BOTH sha256 indexes so they would describe reality. This changes the reality instead, and the models change with it — otherwise the next baseline.yml run reintroduces exactly the drift 0088 removed. CONCURRENTLY throughout, so building the image_tag index does not hold an ACCESS EXCLUSIVE lock over every write for the duration. The cost is that the migration cannot run in a transaction and so is not atomic: every statement is IF NOT EXISTS / IF EXISTS, making a re-run after a partial failure safe. The docstring carries the query for finding an INVALID index left by an interrupted CONCURRENTLY build. What the sweep found clean, for the record: all 43 tables have a primary key; all 51 FKs declare an explicit ON DELETE, so none silently blocks a delete; the three enum CHECKs match the code that writes them (rule 36). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 08:25:41 -04:00
bvandeusenandClaude Opus 5	b1bd2531ad	style: sort JSON first in three sqlalchemy import blocks ... ruff's isort runs with order-by-type, which sorts ALL_CAPS names ahead of CamelCase ones, so `JSON` belongs at the head of the list rather than between `Integer` and `String`. Two of these (backup_run.py, post.py) have been failing lint since 5e1996e — I did not check the push CI after that commit, only the baseline workflow I had dispatched, so ci.yml has been red on dev across 5e1996e, ed2b1ad and d044e93. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 00:38:46 -04:00
bvandeusenandClaude Opus 5	389afe2f7b	db: the doubled CHECK list was six, not four (#3275) ... Run 5029 confirmed the four renames landed and surfaced two I had missed: external_link's host and status CHECKs are doubled the same way. They did not show in run 5026's diff because BOTH sides produced the doubled form back then — external_link.py pre-prefixed its names, so the models matched the chain's mistake. Switching all six models to bare names is what exposed the two the migration did not cover. The list in the file now comes from matching ck_(\w+?)_ck_\1_ against the chain's own pg_dump, rather than from reading migrations by eye. Reading by eye is what missed these, in the same way it earlier missed a UNIQUE constraint sitting two lines above the index being looked at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 00:33:50 -04:00
bvandeusenandClaude Opus 5	b979062dd7	db: rename the four double-prefixed CHECK constraints (#3275) ... Run 5026 got the models-vs-chain diff to 7 lines. Three findings, and one of them reverses an assumption I made in the previous commit. The doubled CHECK names are what the DATABASE has, not what the generator invented. base.py's convention is ck_%(table_name)s_%(constraint_name)s, which — unlike uq/fk/ix — applies even to a constraint that already has a name, so four migrations that passed an already-prefixed name got it prefixed twice: ck_import_settings_ck_import_settings_singleton ck_ml_settings_ck_ml_settings_singleton ck_post_ck_post_translation_override ck_tag_ck_tag_fandom_requires_character The workflow repair added last commit is still correct and still needed — autogenerate really does re-double a name on the round trip — but it was making the MODELS side clean against a chain that is dirty. The comment in ml_settings.py claiming its bare name "matches migration 0003" was simply false; 0003 produces the doubled form. Nothing reads a CHECK constraint by name, so this has never done harm. But it is precisely the development-era residue the collapsed baseline exists to leave behind, and a public schema should not ship it — so 0088 renames the deployed constraints and all six models now declare bare names. RENAME CONSTRAINT is catalog-only: no scan, no rewrite, no revalidation, which is why this is safe on post and tag. Guarded on pg_constraint scoped by conrelid, so it is a no-op on a database built from the models. ix_tag_fandom_id showed as a difference only because chain_ref was pinned to 0a5bbe8, which predates 0088 — the comparison was measuring the models against a chain missing the migration that closes the gap. chain_ref now defaults to blank, meaning "the chain in this ref". Pin it to a commit only after the collapse, when the tree no longer carries the revisions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 00:29:57 -04:00
bvandeusenandClaude Opus 5	573228b9da	db: finish reconciling the models with the deployed schema (#3275) ... Closes the residue the first reconciliation pass left, and corrects a factual error I put into the record. sha256 was NOT missing a uniqueness guarantee. I read `op.create_index("ix_image_record_sha256", ...)` at 0001 line 151 and concluded duplicates were possible, without reading line 149 two lines above it: sa.UniqueConstraint("sha256", name="uq_image_record_sha256"), Uniqueness has held since the initial schema. The database expresses it as a CONSTRAINT plus a separate non-unique lookup index; the model said `unique=True, index=True`, which is one UNIQUE index under a different name. Same guarantee, different objects — which is exactly why the two schemas did not line up. The model now declares both objects. No DDL. 0088's docstring, which repeated the claim, is corrected in place. Two real divergences, both the MODEL over-claiming: * source: uq_source_artist_platform_url (alembic 0010) was declared nowhere in the models — source.py had no __table_args__ at all — so autogenerate would have proposed DROPPING it. * head_metrics_snapshot.tag_id: model said NOT NULL, 0060 created it nullable. Left nullable; the FK already cascades. Seven constraints renamed to what the chain actually created, rather than what base.py's naming convention renders: uq_series_page_image, uq_series_chapter_anchor_page, fk_series_chapter_anchor_page, fk_image_record_artist_id, fk_image_provenance_from_attachment, and the two hand-shortened fk_tsr_* names from 0003. Float server_defaults now mirror their own migration, per column. The chain is MIXED: a plain string renders DEFAULT '0.90'::double precision, sa.text() renders DEFAULT 0.90, and the migrations used both. Seven columns take text(); the rest stay strings. Two literals also disagreed outright — process_{auto_apply,conflict}_threshold said 0.9/0.5 against the migration's 0.90/0.50. baseline.yml gains two things. A repair for a SECOND generator defect in the same class as the missing pgvector import: base.py's ck convention contains %(constraint_name)s, so it applies even to a NAMED CheckConstraint — autogenerate writes the already-rendered name into the migration and running it applies the convention again, yielding ck_ml_settings_ck_ml_settings_singleton. That is round-tripping damage, not a claim the models make, so it is undone rather than counted. And the diff now runs twice. Column ORDER differs permanently between a schema built by 87 ADD COLUMNs and one built in a single shot — the operator's database keeps chain order forever, a fresh install gets model order — so a check that failed on it could never pass. The second pass SORTS column lines within each CREATE TABLE instead of deleting them, which cannot hide a column present on one side only, or one whose type, nullability or default differs. Ordered diff is reported as information; the order-insensitive one is the verdict. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017QHszn9H8VBvx5Ke8x1hvw	2026-08-31 00:24:00 -04:00
bvandeusen	d044e93bdb	ci: repair autogenerate's missing pgvector import before applying (#3275) ... mode: models applies the raw autogenerated candidate, and it cannot run: sa.Column('weights', pgvector.sqlalchemy.vector.VECTOR(dim=1152), ...) NameError: name 'pgvector' is not defined Alembic emits the qualified reference without emitting the import. Observed on run 4988, which turns this from a thing I predicted by reading the candidate into a thing demonstrated by executing it. Repaired in the workflow rather than counted as a schema difference: the comparison asks whether the MODELS describe the schema, and this is a defect in the generator. The same fixup has to be applied by hand to any baseline generated this way, which is why it is item 4 on the collapsed baseline's hand-written list.	2026-08-30 14:47:29 -04:00
bvandeusen	ed2b1adc2e	ci: compare the schema the MODELS produce against the migrations (#3275) ... baseline.yml only ever compared migrations against migrations. The question #3275 exists because nobody had ever asked the other one: does a database built from the MODELS match the one the chain produces? `mode: models` answers it. It applies the candidate autogenerated from the models instead of this tree's revisions, and diffs that against the chain. A clean run means --autogenerate is trustworthy again, which it demonstrably has not been: against the pre-reconciliation models it would have proposed dropping eleven indexes and two uniqueness guarantees. The two extensions are created by hand in that mode. They are database objects rather than table metadata, so no model can carry them — their absence is outside what this comparison asks about, and silently tolerating it is correct rather than a filter that hides a defect. Also declares the HNSW index on the ImageRecord model. SQLAlchemy can express an hnsw access method with an operator class (postgresql_using + postgresql_ops), so there was never a reason for it to live only in 0036. That removes the last item from the list of things a generated baseline cannot reproduce, leaving only the two extensions.	2026-08-30 14:43:26 -04:00
bvandeusen	5e1996e77f	db: reconcile the models with the deployed schema (#3275) ... Milestone 328's acceptance test compared a database built by the real 0001..0087 chain against one built from the models, and found ~130 places where they disagree. This closes them. Almost all were the MODEL being wrong, so almost all of this is model edits with no DDL — the database already had these things, nothing in it changes, and no deploy is needed for this part: * 92 columns gained server_default. The models carried Python-side `default=` only, so the ORM filled the value and the column had no database default. Anything inserting outside the ORM behaved differently from production. * Eleven indexes that existed only in migrations are now declared: the three backup_run reporting indexes, the two date-ordered image_record browse indexes, import_task and presentation_review, and the three task_run history indexes. All use text() for their DESC ordering and postgresql_where for the partial one. * Two UNIQUE indexes that autogenerate silently proposed DROPPING, because neither is expressible as a UniqueConstraint: uq_tag_name_kind_fandom — an EXPRESSION index over (name, kind, COALESCE(fandom_id, 0)) uq_post_artist_external_id_null_source — PARTIAL, WHERE source_id IS NULL post.py already had a comment describing the second one. The comment was right; nothing declared it. * The two external_link enum CHECKs (host, status) — rule 36 territory, and absent from the model entirely. * Two indexes were named explicitly. A bare index=True generated ix_tag_alias_canonical_tag_id where the database has ix_tag_alias_canonical, so autogenerate proposed a drop+create of an index that was already there under another name. Same for tag_suggestion_rejection. Only ONE thing needed DDL, as 0088: tag.fandom_id is declared index=True but no migration ever created that index. Deliberately NOT here: image_record.sha256. The model says unique=True; 0001 created a plain index. Duplicates are possible today and the ORM believes otherwise. The fix depends on whether duplicates already exist — if they do, that is a dedupe decision, not a constraint — so it waits on an answer about live data. The real severity of #3275 is not the squash. It is that --autogenerate has been unsafe on this project: run against the old models it would have proposed dropping eleven indexes and two uniqueness guarantees.	2026-08-30 14:42:30 -04:00
bvandeusen	98b56330d0	ci: emit the chain schema dump for local reconciliation work (#3275) ... Reconciling the models against the deployed schema needs the actual pg_dump, not an inference from the unified diff. Parsing table context out of diff hunks drops every table whose CREATE TABLE line falls outside a hunk — it under-reported 81 columns across 13 tables when the real figure spans more, missing artist, gpu_job, download_event and external_link entirely. Same checksummed-base64 transport as the candidate baseline, for the same reason: a plain cat of a file this size was silently truncated mid-line by the runner on run 4964.	2026-08-30 14:35:02 -04:00
bvandeusen	6959e1220c	Revert "db: collapse alembic 0001..0087 into one baseline" ... This reverts 2529b51. Not a retreat — a reordering, on the operator's call, and the better sequence. The squash's acceptance test (run 4971) found ~130 places where the ORM models do not describe the deployed schema (#3275), including a unique=True the database never had and two UNIQUE indexes that exist only in migrations. Collapsing now would have baked all of that into the one file a public installer starts from. So: fix the drift first as ordinary migrations on the intact chain, let the operator deploy so their database moves to the corrected head, and only then collapse. The baseline is then generated from reconciled models and reproduces a schema worth reproducing. Nothing is lost by reverting. The baseline was never deployed, and regenerating it after the fixes is strictly better than patching this copy — it will come out of autogenerate correct rather than needing the same hand-finishing twice.	2026-08-30 14:34:28 -04:00
bvandeusen	2529b516e6	db: collapse alembic 0001..0087 into one baseline (milestone 328 step 1) ... 87 revisions narrating this project's build-out become one file that creates the schema in a single step. They cost nothing at runtime — all 86 upgrade steps ran in 0.2s (note #3260) — so this is a presentation change, not a performance one: a new installer should not inherit our development history to stand up a database. Deleted: 87 revisions (6,052 lines), the 10 tests/test_migration_*.py files (483 lines) that asserted intermediate states and backfills which no longer exist, and backend/app/utils/artist_backfill.py — the only live module a migration imported, with no other consumer anywhere. That last one satisfies the operator's separate request to inline it into 0008 and delete the module; the squash removes both outright. THE REVISION ID IS "0087", NOT "0001", ON PURPOSE. It is the id of the last revision collapsed, so an existing database is already at head and `alembic upgrade head` does nothing. The alternative is `alembic stamp` against live data, and stamp validates NOTHING — it writes a version string whether or not the schema matches, so a wrong baseline surfaces later, via the next real migration, with no clean way back. This removes that operation rather than making it safe. Future revisions run from 0088. Four things are hand-written because SQLAlchemy metadata does not carry them, and none fail at generation time: 1. CREATE EXTENSION vector — the VECTOR columns cannot be created without it, so it is ordered first in upgrade(). 2. CREATE EXTENSION tsm_system_rows — surfaces only when the random sample query runs. 3. the HNSW index on image_record.siglip_embedding, raw SQL because create_index cannot express USING hnsw (... vector_cosine_ops). The quietest of the four: everything works, similarity search just stops using an index. 4. import pgvector.sqlalchemy.vector — autogenerate EMITS pgvector.sqlalchemy.vector.VECTOR references without importing it, so the generated file dies with NameError on first run. The candidate came out of CI (run 4967) as checksummed base64 rather than a plain cat, because run 4964's cat was truncated mid-line inside a column definition with the step still green — 29 tables instead of 42, and it looked entirely plausible. Verified here: 56,582 bytes, sha256 471acfca69c0…, 42 tables, 66 indexes, 42 drops. NOT YET PROVEN against the old chain. baseline.yml does that, and it is step 2's gate; this commit does not claim the schemas match.	2026-08-30 13:45:35 -04:00
bvandeusen	8f1ac0c96a	ci: transport the candidate baseline as verifiable base64 ... Run 4964 passed the control (1121 normalised lines, schemas identical) but its candidate print was silently truncated. `cat` of the ~33KB generated file stopped mid-line inside sa.Column('mime', sa.String(length=128) and the runner carried straight on to the next traced command with the step still green. The captured text was 484 lines and 29 tables, and looked entirely plausible — which is exactly what makes it dangerous: a schema definition cut in half is still syntactically suggestive, and nothing in the log says it was cut. Now emitted as base64 at a fixed 120-column width, followed by a sha256, a byte count and a base64 line count. Short lines instead of long ones, and more importantly the receiving end can PROVE it got the whole file rather than trusting that it did. Also found in that output, and the reason the candidate could never have been committed as-is: it references pgvector.sqlalchemy.vector.VECTOR(dim=1152) for head_training_run.weights and image_record.siglip_embedding, but autogenerate does not add the corresponding import. The file would die with NameError on the first run. That is the fourth item on the list of things the generator cannot be trusted with, alongside the two CREATE EXTENSIONs and the HNSW index.	2026-08-30 13:40:15 -04:00
bvandeusen	5fd171a544	ci: fix two things the baseline control run found ... Run 4960 was the control — the chain compared against itself, which must come back clean before a clean diff after the squash means anything. It did its job and failed on both counts. 1. The harness is sound. Both dumps came back 1123 normalised lines and differed on EXACTLY two, the \restrict / \unrestrict pair that newer pg_dump emits to fence a dump against injection during restore. It is a fresh random nonce per invocation, so it differs by construction and is noise by definition. Now filtered — and the control is what licenses that filter: it was OBSERVED to be the only false positive rather than assumed to be one, which matters for a check whose whole value is that its normalisation does not hide a real difference. 2. The candidate-baseline step never ran. `if: github.event.inputs .generate == 'true'` on a `type: boolean` input silently evaluated false — no diagnostic, step skipped, job carried on. The same `github.event.inputs` typing quirk build.yml already works around for force_build. Rather than fight the input typing, the gate is now the tree itself: skip if alembic/versions holds one file. That is the real question anyway — there is nothing to generate once the chain is collapsed — and it cannot be silently wrong the way an unevaluated expression can. Worth noting what the control also proved incidentally: the two schemas were byte-identical across 1123 lines despite being built by separate alembic runs into separate databases, so pg_dump's object ordering is stable enough to diff directly and no sort normalisation is needed.	2026-08-30 13:35:26 -04:00
bvandeusen	62583791d8	ci: a workflow that proves a collapsed alembic chain matches the old one ... Milestone 328 step 1 needs a baseline generated from the models, and step 2 must not stamp the operator's live database until that baseline is proven to reproduce what the 87-revision chain produced. `alembic stamp` validates nothing, so an unproven baseline fails silently now and loudly later, on real data. There is no local Python environment and rules 10/12 point away from standing one up, so the comparison runs in CI, where a pgvector Postgres is already built from the chain on every integration run and nothing is at risk. It builds two databases and diffs their pg_dump --schema-only output: one from `alembic upgrade head` on the revisions read out of git at `chain_ref`, one from the current tree. Reading the chain from git via a worktree — rather than from the working tree — is what keeps this usable AFTER the old revisions are deleted, so it is the proof for step 1 and the pre-flight for step 2 rather than a one-shot script. Both sides use `alembic upgrade head`, never metadata.create_all, per rule 82 — and that rule's reasoning is exactly the hazard here. `create_all` emits plain CREATE TABLE and skips everything else, which is why the optional autogenerated candidate CANNOT be trusted as the answer. Three things in this schema are invisible to SQLAlchemy metadata: CREATE EXTENSION vector (0001) CREATE EXTENSION tsm_system_rows (0004) the HNSW index on image_record.siglip_embedding, raw SQL because alembic's create_index cannot express USING hnsw (...) (0036) plus any CHECK constraint or server_default a migration added without the model declaring it — 4 model files declare CheckConstraints against 6 migrations that touch them. The candidate is a starting point to hand finish; the diff is what proves nothing was missed. Results are printed to the job log rather than uploaded: ci-requirements records that this runner cannot do actions/upload-artifact@v4+, and the repo dropped the action entirely in 2026-05. Run it first with the chain still present, as a control — the diff compares the chain against itself and must come back clean. A clean diff after the squash only means something if the harness was shown to be capable of producing one beforehand. Temporary. Delete once the baseline is stamped.	2026-08-30 13:31:16 -04:00
bvandeusen	0a5bbe81dc	docs: the scheduled refresh does NOT republish nothing (#3265) ... Step 4 asserted that when the base has not moved the refresh is "a ~13s no-op that republishes nothing", and that this no-op was the point. The first half is false and was written without being tested. Run 4934, the first real fire: every content step reported CACHED and both bases resolved to unchanged pinned digests, yet all three :latest tags took a new manifest digest anyway. fabledcurator 4ea5265ba017 -> 380e504de0fa fabledcurator-ml 6e7cfc0c09fd -> 6b2eefc301d8 fabledcurator-agent 44920e0af1f3 -> 54accbeb52ed buildkit mints a fresh image config per run, so identical layers get republished under a new config blob. Storage cost is trivial; the cost that matters is that a :latest digest change stops meaning "something is different", and :c-<sha> is handed a new manifest to diverge from every Sunday for no reason. Corrects the workflow comment (x3) and ci-requirements.md to say what actually happens. Filed as #3265 with the candidate fixes; the likely one is a deterministic SOURCE_DATE_EPOCH off the value artifacts.sh already derives, which would make "same source, same version" into "same source, same bytes". The rest of step 4 verified clean on the same run: the guard passed (HEAD is main (499720d), `git checkout -B main`) — so this runner DOES evaluate the env context inside `with:` — the tag list was :latest alone with no :c-<sha>, and the repoint step correctly found nothing to write.	2026-08-30 12:57:06 -04:00
bvandeusen	6663e06aa6	ci: assert the scheduled refresh actually checked out main ... BUILD_REF is read through the `env` context inside `with:`, which this runner is not known to evaluate. `${{ steps.* }}` and `${{ secrets.* }}` in `with:`/`env:` are proven here; `env` is not, and run 4915's checkout log (`git checkout -B dev refs/remotes/origin/dev`) cannot tell an honoured `refs/heads/dev` from an empty value falling back to the same place — the two are indistinguishable on every path except the one that matters. If it does resolve empty, the weekly refresh checks out dev and pushes its source to :latest, which is production. Every lane stays green and the first symptom is production running code that was never merged. So each of the four jobs now asserts its own checkout before doing anything, gated on `github.event_name` — the `github` context is demonstrably evaluated in `if:`, so the guard cannot be disabled by the same uncertainty it covers. A red weekly job is an acceptable outcome; shipping dev to production is not.	2026-08-29 22:57:41 -04:00
bvandeusen	63e0a423d7	ci: a weekly base-image refresh on the channel tags (milestone 326 step 4) ... Skip-if-exists is keyed on our own source, so an artifact whose source stops moving stops picking up base-image updates. `agent/` last changed 2026-07-17; every push since has correctly declined to rebuild it, which also means it will serve that day's nvidia/cuda layers indefinitely. A `schedule:` trigger, Sunday 06:00 UTC, away from CI-runner's Monday security sweep so the two are never diagnosing each other. #3154's blocking open question is dissolved rather than answered. It was written when the identity was a `r-<revision>` TAG, and asked how the next ordinary push could avoid repointing :latest back off the refresh. Milestone 318 replaced that tag with a LABEL, and #3183 made the repoint step exclude its source tag so the label stays readable. Excluding the source is what also keeps a refresh from being undone: on the next main push the reuse check hits, :latest is not rewritten, and the new :c-<sha> is written FROM the refreshed :latest. To be verified by digest, not by this argument. Four decisions, each commented where it lives: * It builds `main`, not the branch that triggered it. Forgejo registers a cron from the default branch — `dev` here — so a scheduled run arrives with github.ref on dev, and a refresh of :dev would be refreshing the one channel that is rebuilt constantly anyway. The ref is decided once in a top-level `env: BUILD_REF` that all four checkouts take. Deriving it per job would let the halves disagree: sign-extension would derive dev's extension version while build-web bundled main's, and the release download would 404 on a version that exists perfectly well. * It publishes only the channel tag. :c-<sha> for main's HEAD already names the bytes that commit built; re-pushing it over refreshed layers would break the one tag rule 145 makes immutable, and it is the rollback unit — so the breakage would surface on the day somebody needed it. The repoint step needs no schedule case: the tag list is the channel tag alone, SOURCE is the only entry, it is excluded as always, and the step correctly does nothing. * It bypasses reuse by construction, since it rebuilds the same source and fc.revision always matches. Checked in the reuse step beside force_build, so one decision still drives both the build and the repoint. * `pull: true`, on the scheduled path only, is the actual mechanism. A moved base tag changes the FROM layer's cache key and everything above it rebuilds; an unmoved one is satisfied by the registry cache and the refresh is a ~13s no-op that republishes nothing. That no-op is the point — :latest should change when there is something new in it, not every Sunday. The known lag, left deliberately: an apt package update while the base tag stands still is not caught, and closing it needs no-cache: true, which buys weekly churn for it.	2026-08-29 22:53:48 -04:00
bvandeusen	499720d87e	Merge pull request #242 from dev ... ci: buildx driver, registry layer cache, and a force_build hatch (milestone 326 steps 1-3)	2026-08-29 22:48:47 -04:00
bvandeusenandClaude Opus 5	5e72076298	ci: registry-backed layer cache for all three images (milestone 326 step 2) ... `cache-from`/`cache-to` on `<image>:buildcache`, `mode=max`, on all three build steps. Closes the half of the driver change that step 1 left open. Step 1 measured worse, not better, and that was expected but is worth stating with numbers. Run 4896, first builds after moving to `docker-container`: build-web 3m44s (cold baseline 2m23s) build-ml 3m49s (cold baseline 3m20s) build-agent 11m12s (cold baseline 9m26s) The container driver gets a FRESH buildkit instance per job, so it has no local layer store to fall back on — where the old docker driver at least reused whatever the runner's dockerd happened to hold. That is why a registry cache is the only cache this driver can have, and why step 1 on its own is a regression rather than a win. `mode=max` so intermediate stages cache too. The agent's two ~150s pip layers and web's frontend-builder stage are the entire cost, and a min-mode cache would drop exactly those. A `:buildcache` tag is not the withdrawn tag scheme returning. Rule 145 narrowed against names NOTHING reads; this one is read by every build that runs, is one moving ref per image rather than one per build, holds cache blobs rather than a shippable artifact, and is overwritten in place rather than accumulating. Closer to `:dev` than to the `:2026.8.28` tags 318 deleted — and said so in the workflow, so it is not "cleaned up" by a later reader. Expect the next build to be slower again, once: it is still cold AND now pays the cache export. The measurement that matters is the one after that. Scribe #3114. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 18:56:13 -04:00