1245 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	a9c1b421a7	feat: change a lane's slots on a running system, over the broker (4292) ... Milestone 422 step 2. `GET /api/system/workers` reports every lane joined to its live pool; `POST /api/system/workers/<name>` changes it. NO DOCKER SOCKET. Milestone 365 deferred "acting on the state" because restarting a dead worker needs a socket the web container deliberately does not have. That holds for restarting a CONTAINER; it does not hold for changing how much work a RUNNING worker does. celery's pool_grow / pool_shrink / add_consumer / cancel_consumer send a message over the Redis the app already uses, and the worker resizes itself. No new privilege, no new surface, and the security question that deferred this is never raised. PERSIST AND PUSH, in one call, in that order. pool_grow is not durable — a restart drops every lane to its env concurrency — so a UI that only pushed would lose the setting on the next deploy with nothing to show for it (lesson #4202). Storing alone would describe nothing until something restarted. A failed PUSH is not a failed setting: 200 with `applied: false` and a reason, so the UI says "saved, not yet live" rather than "that didn't work". Step 3's reconcile carries it when the lane answers again. PER-REPLICA DELTAS. `pool_grow(n, destination=[...])` adds n to EACH destination, so while `worker` runs `replicas: 2` a single delta from an aggregate is wrong for both. `slots` therefore means what CELERY_CONCURRENCY means — one process's pool — and each replica is driven to it from its OWN current size, so replicas that drifted apart converge rather than moving in lockstep. I wrote this wrong first: the docstring claimed per-replica while the code computed one delta from the max across replicas. LaneLiveState now carries `pools` per hostname and exposes `pool` as a property. A replica already at the target is sent nothing at all — the reachable fixed point step 3's periodic reconcile needs, or it re-issues a grow of zero every tick forever (lesson #4183). A replica that answered inspect but not stats is NAMED in the error rather than skipped silently, since otherwise it would run at a size the UI claims it does not. `present=False` is not "zero slots", it is "nothing answered" — kept distinct throughout, because step 3 skips an absent lane rather than correcting it. /workers now also reports pool size (from `insp.stats()`) and RESERVED count. Celery prefetches, so tasks that have left the Redis list but not started are invisible to LLEN: a lane can read depth 0 with thirty tasks held in worker memory. `pending` is depth + reserved. The UI is misleading without this and step 7's autoscaler would be simply wrong. Also kills the THIRD copy of the queue list: system_activity's _QUEUE_NAMES, whose own comment admitted the coupling ("must match celery_app.task_routes") and which sat alongside task_routes and the ROLE_NAMES copy step 1 collapsed. Now derived from LANES. The rendered order changes to lane grouping, which is the better shape for a lane-oriented UI. Separate blueprint rather than folding into system_activity, which states in its first line that it is read-only and answers a different question — its /workers is keyed on celery HOSTNAME and reports which nodes answered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-22 08:01:23 -04:00
bvandeusenandClaude Opus 5	5974a1bfbc	fix: two errors in the worker-lane tests (4291) ... Both mine, both in tests/test_worker_lanes.py, neither in the code under test. Run 7242. RUFF I001 — two blank lines between the import block and the first module-level comment. Rule 102 names this exact trap ("exactly ONE blank line between imports and a module-level constant/comment/pytestmark") and I was pointed at that rule repeatedly before opening it. SIX FAILURES in test_worker_lane_check_constraints — the test asserted bare constraint names, but Base.metadata's naming_convention has already applied the `ck_worker_lane_` prefix by the time __table__.constraints is read. The failure output is worth keeping: it shows the model emits exactly the three intended constraints, prefixed once — ck_worker_lane_slots_non_negative slots >= 0 ck_worker_lane_cap_non_negative slots_cap >= 0 ck_worker_lane_slots_within_cap slots <= slots_cap — which is the model behaving correctly, and confirms the migration's op.f() names match what the ORM produces. The assertion is now an equality against the prefixed names plus an explicit check for a doubled prefix. That is strictly more valuable than what I wrote: a bare-name assertion would have passed just as happily against `ck_worker_lane_ck_worker_lane_slots_within_cap`, which is the defect alembic 0088 had to rename four constraints for (#3275). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-22 07:53:30 -04:00
bvandeusenandClaude Opus 5	84f13135ce	feat: worker lanes become rows — slots, a settable cap, a derived ceiling (4291) ... Milestone 422 step 1. The data model the rest of the milestone reads. No behaviour change: nothing consumes these rows yet, and every lane still boots at its CELERY_CONCURRENCY env value. Three numbers, not two, per the operator's distinction — the derived value is a cap ON the cap: slots <= slots_cap <= derived_ceiling (live) (operator) (computed) They can always lower their own cap; they cannot raise it past what the container can hold. The ceiling is never stored, so a row written on a 32GB host and later run in a 4GB container is bounded by the 4GB. `services/worker_lanes.py` is the one place that knows the lane set. `models/worker_lane.py` holds only what an operator may change. Two deviations from the step as written, both deliberate: QUEUES ARE NOT A COLUMN. The step body said the row carries its `-Q` list, but a lane's queues are decided by celery_app's task_routes, not by preference — an operator cannot move a backup off maintenance_long. Storing them would create a row that can contradict the routing table, with nothing to notice until a queue had no consumer. So queues are code, slots are data. `test_every_routed_queue_has_a_lane_that_serves_it` reads the real routing table and fails if a route is ever added without a lane. ROLE_NAMES IS NOW DERIVED, not left alone. It was a hand-kept second copy of "queue set -> display name" and had already drifted: maintenance_long is a live lane with four task routes and a dedicated worker in the operator's stack, and the roster did not know its name — so the System tab labelled it `Worker (maintenance_long)`. Adding a lane table beside it would have made three copies. The ceiling honours cgroup limits rather than the host's. `os.cpu_count()` reports the HOST's cores from inside a container, so a 4-core quota on a 32-core host would otherwise offer 32 slots — and the operator's own stack sets `cpus: '4.0'` on ml-worker, so that is real configuration, not a hypothetical. Memory reads cgroup v2 then v1, and recognises v1's PAGE_SIZE-aligned LONG_MAX sentinel by magnitude rather than treating it as petabytes. Every uncertain case fails LOW. An unreadable limit yields UNKNOWN_CEILING, never unlimited — not knowing how much memory there is must not read as plenty. A box too small to hold one model beside the web process gets an ML ceiling of 0 rather than a floor of 1: offering a slot that OOMs the container the first time it is used is exactly what this exists to prevent. ML_BYTES_PER_SLOT is 4 GiB and is UNMEASURED — flagged as such in the code, with the method for replacing it with a real figure. It decides whether a stranger's server survives enabling tagging, so it errs toward refusing a slot that would have fitted. Seeded one-of-each with ml at 0 and disabled (alembic 0103). ML off is step 6's requirement arriving early: enabling the lane is what triggers the SigLIP download, and rule 164 permits a runtime fetch only for a feature that is optional and clearly off. The seed values are literals rather than an import of LANES — a migration is a statement about one moment, and importing the live defaults would silently change what this revision does on a fresh database in 2027. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-22 07:48:25 -04:00
bvandeusenandClaude Opus 5	aa2bb665b9	fix: serialise builds per branch, and pin :c-<sha> to the digest this run built (4290) ... Two independent defects from one missing mechanism. `grep -rn concurrency .forgejo/workflows/` returned nothing, so two pushes to a branch ran build.yml in full parallel with no ordering. Both jobs read `fc.revision` off the channel tag before either has pushed, so both miss the reuse check and both build. Whichever finishes LAST owns the tag. Half 1 — the rolling tag. A slower older build leaves `:dev` carrying content older than the commit that moved it. Family rule 146 says a rolling channel refreshes itself; this is the case where it quietly does not. Self-healing on the next push. Half 2, and the reason this is not filed low — `:c-<sha>`. The repoint step wrote every non-channel tag by copying the channel tag BY NAME, so the immutable rollback tag named whatever `:latest` pointed at when that step happened to run. Lose the race and `:c-<shaA>` names the OTHER run's bytes. Rule 145 makes that tag the rollback unit and immutable, so this does not break immutability — it makes the tag wrong from birth, and immutability then guarantees nobody ever corrects it. Nothing goes red; it surfaces the day someone rolls back and gets a commit they did not choose. Not self-healing. Two fixes, deliberately both: A. A workflow-level `concurrency` group keyed on `github.ref`, so dev and main never block each other. `cancel-in-progress: false` — queue, never cancel: cancelling could kill sign-extension mid-AMO-upload, leaving the version registered at AMO with no cached asset, which is the unrecoverable stuck state that job's rollback trap exists to prevent, reached by another door. Not keyed on BUILD_REF because the group is evaluated before any job starts and cannot read the `env` context. B. Each build-push-action step gains `id: build`, and the repoint step copies from `$IMAGE@${{ steps.build.outputs.digest }}` — the manifest THIS run pushed — rather than from the channel tag by name. On a reuse hit there is no digest and the channel tag remains correct by definition: "hit" means that tag already carries this commit's fc.revision. B is not redundant with A. A depends on this Gitea honouring a key whose failure mode is silent, and this file has been burned by exactly that before (the `format()` note records `true == 'true'` evaluating FALSE on run 5270, every lane green, the feature simply not happening). B holds the :c-<sha> correctness property whether or not A is honoured. The loop's exclusion is now keyed on CHANNEL_REF rather than on SOURCE, and that is load-bearing: SOURCE may now be a digest ref, which never equals a tag string, so testing against it would stop excluding the channel tag. imagetools would index-wrap it and `.Image.Config.Labels` would stop resolving, killing the reuse label permanently — #3183 arriving again. Neither half is verifiable on a dev push: dev's tag list is the channel tag alone, so the repoint correctly does nothing there. B is observable on the next merge to main (:c-<sha>'s digest must equal the build step's), and A by pushing twice in quick succession and reading the run list for a queued second run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 22:07:53 -04:00
bvandeusenandClaude Opus 5	bd92fb46b3	test: pin the JS<->Py artist-pattern mirror with a shared sample table (3093) ... `PLATFORM_ARTIST_PATTERNS` (extension/lib/platforms.js) and `_PLATFORM_PATTERNS` (extension_service.py) are two hand-kept copies of one table whose only guard was the comment "keep in sync by hand; reviewers catch drift" — the same guarantee manifest.json had before #3069, where deviantart sat in the manifest for seven weeks after the product dropped it. Drift here is worse than the manifest case, because the two copies gate opposite halves of ONE interaction: the JS copy decides whether the "Add to FC" button appears, the Python copy decides whether the resulting POST is accepted. JS looser than Py shows a button that 400s; Py looser than JS silently never offers a button for a URL the backend would take. #1485 (the Patreon /c/ and /cw/ shapes) was the second of those, and its fix had to be applied to both files by hand. The two-runtimes objection to a shared SOURCE file is fair, so this tests the invariant instead of the source. `extension/test/artist-url-samples.json` is one table of 25 URL samples — match (with the expected slug) and no_match, each with a `why` — read by BOTH suites and asserted against each one's own copy of the patterns. Neither runtime imports the other; a change to one copy alone turns the other runtime's suite red. Both halves also assert their own coverage: the sample platforms must equal the platforms that actually have an artist pattern, and every platform must have samples in both directions. Without that, deleting a platform's samples would make the guard pass by testing less. Discord is deliberately in neither table — it is channel-based and has no creator page to put a button on. The no_match half asserts `_derive` RAISES rather than merely missing the platform: it tries every pattern in turn, so a nav page some other platform's pattern happened to swallow would still be accepted by the backend — the same defect wearing a different platform name. Samples live under extension/test/ because that path is excluded from both the XPI file set and the extension version derivation (packaging.sh: NOT_PACKAGED_TRACKED and NOT_VERSION_RELEVANT both carry `test/**`), so adding samples ships no bytes and forces no re-sign. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 21:04:53 -04:00
bvandeusenandClaude Opus 5	24b10d0ffa	feat: retire pixiv entirely — delete its code, its ledgers, its credential (3977, 3978, 3979) ... Milestone #406 phase 2, with issue #3980 folded in. Phase 1 (2026-09-13) unregistered pixiv so nothing could reach it; the code has sat in the tree uncalled since. DeviantArt is why the second half is not left for later — #3069 retired it in code on 2026-08-27 and its stored session was still in the database seven weeks on. Step 5 — the code. Deletes pixiv_client, pixiv_downloader, pixiv_ingester, platforms/pixiv and their three test modules and fixture, then edits out every remaining reference: the dispatch entry, the campaign-id and verify branches in download_backends, the display-name branch in extension_service, and the comments that still described pixiv as live. The consolidation check the step asked for comes back negative: native_ingest_common has seven non-pixiv callers (patreon, subscribestar, membership_reconcile, membership_roster, ingest_core), so nothing there drops to a single user. Step 6 — the data, alembic 0102. Drops pixiv_seen_media and pixiv_failed_media, and deletes credential rows whose platform is not registered. Written as "not registered" rather than "pixiv" at the step's explicit ask, which is what makes one migration cover two retirements: the pixiv OAuth refresh token and DeviantArt's leftover session (#3980). It is also the only way either row can go — the credentials UI renders one card per platform from /api/platforms and looks the credential up by key, so an unregistered platform's row has no card and no Remove button. Pixiv's Source rows are KEPT, changing the milestone's original data table on the operator's call. `platform` is stored only on Source; neither Post nor ImageRecord carries it. Both FKs are ON DELETE SET NULL, so a delete would not lose the art — but it would drop every pixiv image into the gallery's __unsourced__ bucket and strip the platform chip off every pixiv post. The rows stay disabled (0097) and unregistered, so nothing schedules or downloads through them. Keeping them costs nothing and keeps the attribution that "the art already downloaded from pixiv stays" is about. Step 7 — the guard. test_pixiv_code_and_tables_are_gone asserts absence from the module table and from Base.metadata, not from prose (snippet #3352's trap). The extension and registry negative assertions were already in place from phase 1. The final sweep found one real residue step 4 missed: extension/README.md still advertised pixiv support and carried a "Pixiv OAuth" manual-test item. Also replaces the two deleted dispatch tests with one over the whole NATIVE_INGESTER_PLATFORMS set, so adding a platform and forgetting its ingester class now fails at unit level rather than as a mid-download KeyError. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 20:46:15 -04:00
bvandeusenandClaude Opus 5	5aa8e3d81b	fix: a stopped source is not a failing one, and cannot be deep-scanned (4279) ... Ebi77 sat in the "1 source is failing" banner for six days with no action available, reading `stranded by recovery sweep (no terminal status after time_limit)`. Four things lined up: 1. The membership sweep did its job — saw `former_patron`, disabled the source, cleared its failure state. Clean at 02:50. 2. Twenty minutes later a deep scan was armed on it. `/backfill` had a credential pre-flight but NO `enabled` guard, while `/check` has carried one all along. The two trigger endpoints disagreed, and the ungated one is the one that arms the long walk. 3. Without a membership the walk cannot finish, never reaches a terminal status, and the recovery sweep strands it with consecutive_failures = 1. 4. Nothing could clear that. A disabled source is never scheduled, so no successful run resets the count; `SourceService.update` clears only on an explicit disable and it was already disabled; and the banner's Retry routes to `/check`, which refuses a disabled source. The card offered a button structurally incapable of acting on the only source it was showing. `failing_sources_clause()` now means "enabled AND erroring". That also settles a disagreement its two callers already had: the scheduler's count paired it with `enabled.is_(True)` and `SourceService.list(failing=True)` did not, so one counted Ebi77 and the other did not — exactly the drift the note above that function warns about, which is why the test belongs IN the predicate rather than beside it. The scheduler's now-duplicate clause is dropped so one place decides. `/backfill` gains the guard for start/recover/recapture. `stop` stays open on a disabled source, or arming becomes a one-way door. Migration 0101 clears failure state on sources that are already disabled — the predicate fixes what the surfaces report, not what the rows carry, and the rows are why the operator had no way out (lesson #4202). It matches what `update` already does on an explicit disable, so rows disabled by any other path come into line. Enabled sources are untouched: a real failure on a live source must keep showing, which the second new test pins. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 19:25:42 -04:00
bvandeusenandClaude Opus 5	11a01a9686	revert: remove the placement reconciler — it manufactured the problem it solved ... Milestone #421 built a sweep that compared each image's `artist_id` to the name of the directory holding its file, and called every mismatch a misplaced image. It reported 33,789 of 63,605 as wrongly filed. That number described the comparison, not the library. What it actually was: 32,475 (97.1%) one artist's own folder, spelled differently — Telepurte/ vs telepurte/. Same artist, same art. 657 ( 2.0%) loose at the images root 328 ( 1.0%) in a folder named after a different artist And the 1% did not mean what the tool assumed either. `ImageProvenance` records the post and source every file was downloaded from — the authoritative answer, which the tool never consulted. Querying it for all 328: 144 provenance agrees with the record (move would be right) 87 provenance agrees with the FOLDER (the record is wrong; move wrong) 53 provenance names SEVERAL artists (no single correct folder) 41 no provenance at all 3 agrees with neither So the sweep would have misfiled or arbitrarily picked for ~41% of the only set it was really needed for. The system already knew where each file came from; the tool inferred it from a column and a directory name instead. Operator, 2026-09-21: *"the current system consistently records where items are and where they came from this is just complicating something works and doesn't need fixing."* Correct on both counts. Removed: the service, the tasks, the model and migration 0099's table, the /api/cleanup/layout and /placement/* endpoints, the Maintenance card and its store actions, and the tests. 0100 drops the table (rule #22 — no legacy). KEPT deliberately, per the operator: - `utils.paths.canonical_subdir` — new filesystem imports derive their directory from the artist's slug, matching what the downloader always did. Not part of this tool; removing it would be churn that fixes nothing. - The 327 files run 1 moved (InsoUwu/ -> insouwu/). Same artist either way, and the gallery renders them correctly. - Everything from #4223 (three-gate dedup, 256-bit pHash) and #4234 (backup credential exclusion). Those fixed problems that were actually reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 18:17:13 -04:00
bvandeusenandClaude Opus 5	2dd9b956d5	feat: File placement card — survey, plan per artist, apply, put back (4246, slice 3c) ... The UI half of the reconciler, in Maintenance. Survey shows how many images sit in the wrong artist's folder and which folders they are in; each artist gets its own Plan button; each run can be reviewed, applied, and put back. Deliberately NOT using useMaintenanceTask. That composable stashes a task id in localStorage so a result survives navigate-away, which is the right answer when the only record is a Celery result. Here the runs are database rows — so a reload, another machine, or coming back tomorrow simply shows the same state, because the state IS the row. The card polls the runs endpoint instead. Copy avoids the vocabulary this work has been tripping over: "folder", "put back", "in the wrong folder" rather than artist_id, revert and canonical. The one thing the operator most needs to know — nothing here changes who an image belongs to — is what the blurb says first. Three things I had assumed and checked instead: MaintenanceTile lives in common/ not settings/; there is no generic ConfirmDialog (BackupCard uses a purpose-built modal), so this uses a plain v-dialog; and `loadArtistNames` did not exist — it does now, mapping id to name so a run row reads "Conto" rather than "#47". The name deliberately is not denormalised into the run: it belongs to the artist and would go stale on a rename. Also extracted `stubFetch` to frontend/test/stubFetch.js. The shape ledger flagged it as a byte-identical duplicate across five specs and this would have been the sixth; the others keep their copies until each is next touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 14:32:51 -04:00
bvandeusenandClaude Opus 5	a4bdbcaca4	test: import the placement task module so its names reach celery.tasks (4246) ... Three registration assertions failed: a task name only enters `celery.tasks` when its module is imported, and nothing in the test process imported `library_placement`. The API routes import it lazily inside the handlers, and the registration test runs before any handler test triggers that. `include=[...]` is what gets the module imported in a real WORKER, so production registration was never in question — the test was asserting something only observable after an import it never performed. test_tasks_admin already carries the convention verbatim (`import backend.app.tasks.admin # noqa: F401 — register tasks`); I wrote the assertion from what I meant instead of copying the idiom next to it. Same mistake shape as the commit-vs-flush bounce one commit ago: the pattern was already in the suite both times. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 14:21:20 -04:00
bvandeusenandClaude Opus 5	db63377754	test: placement API tests must commit, not flush (4246) ... Five of the new tests failed in CI: every one that created a LibraryPlacementRun and then read it back through the client. The ones that touched no rows passed. A flush stays inside the test's own transaction, and the app under test runs on a separate session and connection — so the endpoint queried a database where the row did not exist yet and got its 404 / empty list honestly. `_seed_runs` in test_api_system_backup already commits for this reason; the idiom was there to copy and I did not look first. Recorded in the helper's docstring rather than just fixed, since the next person writing a create-then-fetch API test will reach for flush too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 14:15:52 -04:00
bvandeusenandClaude Opus 5	abe449b4f2	feat: placement reconciler tasks + API (4246, slice 3b) ... Three Celery tasks wrapping the 3a service, and the endpoints that drive them. Routed to `maintenance_long` alongside backups: 33k renames on NFS have no business in the quick lane where the self-healing sweeps live (the 2026-06-07 starvation). A durability bug in 3a, found by thinking about what a crash costs rather than by a failing test: `apply_run` wrote its ledger only at the end, so a worker dying at row 30,000 of 33,789 would have taken the undo information for the first 29,999 with it — and that ledger is the ONLY record of where those files came from. It now persists every 200 moves. Two things fell out of writing that: - `_persist` reassigns `run.moves`, so the loop had to snapshot the plan first rather than iterate the attribute it rewrites. - the reassignment is itself load-bearing: SQLAlchemy does not track in-place mutation of a JSONB list, so an `.append()` alone would never reach the database and the ledger would have stayed silently empty. Re-running a partially-applied plan is safe — the moved rows no longer match their `from` and refuse as "row moved since planning" — but `apply_placement` deliberately has NO autoretry: re-entering a half-applied plan should be the operator's call after reading what happened, not the queue's. Endpoints gate on run state as well as the service does, so a stray POST cannot re-apply an applied run. The list response omits `moves` (an applied whole-library run carries tens of thousands of entries); the detail endpoint includes them, because that detail IS the preview read before agreeing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 14:13:25 -04:00
bvandeusenandClaude Opus 5	9ccc460c69	feat: placement reconciler — plan, apply, revert (4246, slice 3a) ... Milestone #421 step 3, reframed on the operator's steer: not a one-off migration but the system that keeps the tree true. The 33,789 misplaced rows the survey found are just its first run. The placement half was already done, verified by reading each writer rather than assuming: downloads have always written `<root>/<slug>/<platform>/` (gallery_dl.py:523), attach_in_place leaves files where the downloader put them, and `_copy_to_library` / `_supersede` became canonical in #4244. So nothing is written off-canon today; what remains is the backlog and a standing check for future drift. `LibraryPlacementRun` (migration 0099) holds the plan as JSONB, and that one structure does three jobs: it is the PREVIEW the operator reads, the list the APPLY executes (rather than re-deriving the set, so the two cannot disagree), and — because `from` is retained — the UNDO. The undo is the point. It makes a 33,789-file operation something to do one artist at a time, look at in the gallery, and reverse if it reads wrong. That settles whether artist_id or the folder held the truth (spike #4257) by doing rather than by arguing it from a 50-row sample. An applied run is therefore HISTORY, not state — lesson #4226's trap, since it is the only record of where those files used to be. The model and the migration both say so: any future retention here may prune ready/cancelled/ error runs, never an applied one. Everything fails closed. The apply re-checks each row against what the plan recorded — source still there, destination still free, row still pointing where the plan said — because a download or a supersede can land in between. A refusal is recorded with its reason and the run continues; one stale row is not a reason to abandon the other 33,788. The row is updated only after its rename lands, so a failed move can never leave `path` naming a file that is not there. Writing the collision test caught the code disagreeing with its own comment: it claimed the first of two rows wanting one destination and skipped the second, silently picking a winner by iteration order. Now it counts first and filters after, so genuinely neither is planned. Thumbnails are sha-addressed, not path-keyed, so they do not move — pinned by a test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 12:46:06 -04:00
bvandeusenandClaude Opus 5	9728407539	docs: record why the misplaced-rows LIKE needs no escaping (4245) ... `startswith` compiles to LIKE, where `_` and `%` are wildcards, and the call does not escape them. That is safe only because `slugify` reduces a slug to [a-z0-9-] — an invariant living in a different module, which is exactly the kind of thing that gets widened later without anyone connecting the two. Worth naming because `poch4n_art` is a real directory here: if slugs ever carried underscores, that prefix would start matching `poch4nXart` and the sweep would quietly mis-file one artist's rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 11:05:37 -04:00
bvandeusenandClaude Opus 5	fc982f74b9	feat: survey which image rows sit outside their artist's canonical directory (4245) ... Step 2 of milestone #421. The disk survey counted FOLDERS; this counts ROWS, which is the number that matters — every move in step 3 is a row update, and `ImageRecord.path` is the only pointer at the bytes. `library_layout.py` holds the decision in two shared pieces, and both halves of the consolidation spread them rather than restating them (rule 93, the _x_conditions shape from snippet #3087): - `_misplaced_conditions(root, artist_id, slug)` — rows of one artist whose file is not under that artist's directory. The prefix carries a trailing separator deliberately: without it `ara` matches everything under `arbuzbudesh/`, and one artist reads as fully placed while another's rows are silently skipped. Both are real artists here, hence the test. - `destination_for(path, root, slug)` — where a row's file belongs, or None when it must not be moved: outside the images root, or under one of the reserved stores (`thumbs`, `attachments`, `cookies`, `secrets`, `_backups`, `_quarantine`). Relocating those would move the thumbnail cache or the credential key into an artist folder. `destination_for` diverges from `canonical_subdir` in exactly one case, and the docstring says why: a file at the images ROOT with a known artist moves under that artist here, where the import-time helper leaves it alone. The two answer different questions — an empty subdir at import means no artist was resolved, while a row that already carries an artist_id is an anomaly with a known correct home. The 660 unattributed files have no artist_id at all, so no predicate reaches them; they are counted and left for task #4247. `GET /api/cleanup/layout` exposes it. `?check_disk=1` additionally stats every destination for collisions and missing sources — the conditions the apply refuses on — but it is off by default so the count-only pass answers "how big is this" in seconds instead of timing the request out on NFS. Nothing here writes; a test asserts that against both the row and the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 11:03:03 -04:00
bvandeusenandClaude Opus 5	01c906e6ad	fix: ruff I001 — canonical_subdir sorts before derive_subdir ... Rule 102's import-order trap, hit by inserting into the block rather than re-sorting it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 08:49:38 -04:00
bvandeusenandClaude Opus 5	30337a6c11	fix: library paths follow the artist's slug, not the import folder's name (4244) ... Step 1 of milestone #421. The images tree has 57 directory families for what the database says are single artists — `Conto`/`conto`, `InCaseArt`/`incaseart`, `StickySpoodge`/`Stickyspoodge`/`stickyspoodge`, and so on down to a four-way split for Pocket Ace Games. There was never a duplicate Artist row. `/api/artists/names` returns exactly one per artist. The files simply get written to two places for one row: `_copy_to_library` built its destination from `derive_subdir`, which mirrors the IMPORT tree's folder name verbatim, while the download path leaves files where the ingester wrote them — under the slug. Two writers, two conventions, one artist. This is the half that stops it re-growing, and it has to land before anything moves existing files: consolidate first and the next filesystem import out of a capitalised folder re-creates the directory that was just emptied. `canonical_subdir` replaces the top-level segment with the artist's slug and leaves everything below it alone — the post hierarchy is the downloader's business. Two deliberate pass-throughs: no resolved artist (nothing authoritative to canonicalise against) and an empty subdir (a file at the images root, whose fate is task #4247, not a side effect of this helper). `_supersede` resolves the KEPT row's artist for the same reason — a supersede rewrites `existing.path`, so writing it anywhere else would move a row back out of the tree being consolidated. ImageRecord carries `artist_id` with no relationship attribute, so that is a session lookup rather than an attribute. test_import_one_happy_path pinned the old `Alice/` destination and now pins `alice/` (rule 90). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 08:48:46 -04:00
bvandeusenandClaude Opus 5	6915cbbbe1	fix: the images backup carried the key to the accounts it backs up (4234) ... Listing a 2026-05 tarball while investigating the 4.3T `_backups` pile showed its second and third entries: images/secrets/ images/secrets/credential_key.b64 That is the key that decrypts the stored Patreon/SubscribeStar session credentials, and `cookies/` sat beside it — both unexcluded, so this was true of every images backup taken today, not just the old ones. An images tarball is supposed to be a media archive; one that carries the operator's account keys is a credential leak wearing a backup's name, in a single file that is easy to copy to another disk or restore somewhere less protected. Encryption at rest buys nothing when the key travels in the same archive. `secrets` and `cookies` join `_backups` and `_quarantine` in one named tuple, each with its reason recorded — the recursion that produced 4.3T of nested tarballs is the cautionary tale for why the list is worth explaining rather than just listing. A restore no longer re-establishes credentials. You sign in again, which is the correct outcome for a media backup. Tests cover both new names and that every exclude stays root-relative — a bare `secrets` would also match an artist folder of that name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 08:35:35 -04:00
bvandeusenandClaude Opus 5	3313c3b10a	feat: a report that shows what the near-dup gates decide about real artwork (4223) ... The three pixel constants added with the #4223 fix were chosen without ever measuring real files — CI only has synthetic split/solid fixtures, and FC verifies nowhere else. This prints the measurements they should have been chosen from: per pair, the hash distance, the mean drift, the changed-pixel fraction, the verdict, and which gate produced it. It drives the real find_similar with the real confirm rather than restating the decision, so it cannot drift from what the importer does. Read-only: opens files, touches no database. Also splits fingerprint_diff out of fingerprints_match — same computation, now returning the numbers instead of only the boolean, so the report can show how far a pair sat from a limit rather than which side of it it fell on. Runs inside the published :dev image (PIL + imagehash already there, no local env needed) with the art folder mounted read-only — rule 147's channel, so nothing has to reach main to be tried. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 06:30:54 -04:00
bvandeusenandClaude Opus 5	84f873eb1b	fix: ruff UP037 — unquote the fingerprint_path return annotation ... PIL's Image is imported at module scope, so the annotation never needed to be a string. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 00:17:15 -04:00
bvandeusenandClaude Opus 5	a05cb66635	fix: variant artwork was dropped as a near-duplicate even at threshold 0 (4223) ... The operator reported a 15-image variant pack landing as 3 records, then reported variants STILL being dropped with phash_threshold at 0 — the floor of the dial. No setting could have fixed it: at hash_size=8 a pHash is 64 bits of coarse light/dark layout, so two variants sharing a composition produce the SAME bits. Distance 0 meant "identical hash", not "identical image", and the dial was simultaneously too coarse to keep variants and too tight to catch a re-encoded rescale. The hash no longer decides a merge on its own. find_similar now runs three gates, cheapest first: the threshold proposes candidates, aspect ratio (ASPECT_TOL, matching the tier-1 video path) rejects crops and re-canvases, and a pixel-level confirm on the two files accepts. Every gate fails closed — unknown dimensions, an unreadable candidate, a hash of the wrong width all mean "not a duplicate", because too strict keeps a redundant copy the operator can see while too loose deletes artwork only a source re-walk returns. - utils/phash.py: HASH_SIZE 8 -> 16 (256-bit, what ImageRepo always used); aspect_matches, fingerprint/fingerprint_path/fingerprints_match (PIL-only, mean drift + changed-pixel fraction), find_similar gains `confirm`. - importer: _pixel_confirmer supplies gate 3 on both dedup sites, lazily and cached, so a non-matching import costs no extra I/O. - 0098: widens image_record.phash to 64 chars and NULLs every value — a stored 64-bit hash cannot be compared to a 256-bit one, and backfill_phash is NULL-only, keyset-paginated and now on the daily beat, so the library re-hashes itself. Dedup degrades to sha256 until it finishes. - phash_threshold counts bits and the denominator went 64 -> 256, so the setting is reset to the new default of 24 (there is no honest carry-over) and the slider is rescaled to 0-64. - gallery_service dup_threshold 8 -> 32: the same fraction of the hash, so the Explore rail keeps the variance the operator tuned in on 2026-07-01. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR	2026-09-21 00:16:14 -04:00
bvandeusenandClaude Opus 5	3fe9d0a612	fix: the Latest feed's filter dropdowns opened empty ... Operator: "the filters in the latest feed, drop down but don't have values". Two separate causes: - Platform: PostsFilterBar built its items from `platformsStore.platforms`. The platforms store has never had that property; it exposes `list` and `byKey`. The read returned undefined, `|| []` turned that into an empty list, and nothing failed. ArtistsView had copied the same read, so the Browse → Artists platform filter was empty too. Both now read `list` and show platform names rather than raw keys. - Artist: the autocomplete searched the server only after something was typed (autocomplete returns [] for an empty query by design, which its tests pin). Opening the dropdown therefore showed an empty menu. PostsFilterBar now loads every artist once from a new lightweight `GET /api/artists/names` (id, name, slug; alphabetical; no joins) and filters client-side, so the list is there on open. A deep-linked artist_id now also shows the artist's real name instead of "Artist #id". Guard: frontend/test/storeUsage.spec.js scans src for `platformsStore.<name>` and fails on any name the store doesn't define, since the frontend CI has no type-checker to catch this. A positive control shows the shipped `platformsStore.platforms` read is flagged, and a vacuity check confirms the scan really walks the tree. tests/test_api_artists_create.py covers /names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:38:27 -04:00
bvandeusenandClaude Opus 5	7ff8915147	feat: a source stops pulling once its membership ends, and resumes on resubscribe (3995) ... Operator, 2026-09-13: "if I kill a subscription on patreon I would like the pulling to stop on curator as well", with auto-resume chosen. This reverses the 2026-09-11 "report only" decision for lapsed sources. membership_reconcile.apply_membership_lapses runs in sync_memberships right after each platform's successful sync, so it only ever acts on the roster just written. It stops a source (enabled=false, with the same failure-state reset as a manual disable, #1285) only when all of these hold: - the roster is fresh - the source's matched membership says has_paid_access is False (lapsed, or a free follow) - the paid-through date has passed, where the platform gives one (Patreon's member.access_expires_at; SubscribeStar gives none, so it stops at once) - the source is enabled - the operator hasn't chosen to keep it It never acts on absence. A source with no matched membership keeps pulling, because a rename or a never-walked source produces the same absence. An unrecognised status is never a lapse either. It resumes only sources carrying its own `_membership_stopped` marker, once the membership is paid again. The operator outranks the sweep both ways (SourceService.update): - turning a stopped source back on marks it `_membership_kept`, so the next sweep leaves it alone until it's paid again - turning a source off by hand drops the marker, so the sweep never switches it back on Both are `_`-prefixed app-managed config keys, which operator edits already preserve. No migration. The roster/fetch line holds. This is a source-level action by the sweep. No download path reads the roster, and the scheduler still selects on `enabled` alone. test_no_fetch_path_can_read_the_roster is unchanged. UI: SourceRow shows a neutral "Membership ended" chip, with the status and the resume/keep explanation, ahead of the other chips. The sweep's task summary reports stopped/resumed counts. Tests (tests/test_membership_lapses.py): - a lapse stops the source with a clean slate and keeps the id cache - paid-through is honoured - absence, an unknown status and a stale roster never stop anything - a resume touches only what the sweep stopped - a manual on sticks, a manual off drops the marker, and a kept source is released once paid Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:08:06 -04:00
bvandeusenandClaude Opus 5	4d84ab2816	fix: the filter rail goes, and the subscriptions card stops listing what you don't pay for — and can be dismissed ... Operator live review of #253 on the wide window. The images landed well, but: - "the left rail feels like wasted space". The filters and check status move back into one row above the feed (PostsView). On a wide window the row lines up with the feed column, clear of the day gutter, and the container recentres without the 280px rail. The narrow layout is as before, with the status still first. This retires #407 option E; A and D stay. - The "you can't see" link landed on the Subscriptions reconcile card, which listed 9 sources as `former_patron` under "sources your roster doesn't account for". The operator doesn't want sources they no longer pay for listed there, and wants the card dismissable "unless something changes": - MembershipReconcileCard renders only "creators you subscribe to but don't follow here" and the stale-roster warning. - Stopping pulls on lapsed sources is the membership sweep's job instead (#3995, next). - A close button dismisses the card. The dismissal is keyed to a fingerprint of what the card says (offered membership ids plus stale-roster flags), so a new subscription or a roster going stale brings it back. It's stored per browser, which is enough for a single-operator instance. The backend still computes tracked_not_subscribed. #3995 acts on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 22:04:02 -04:00
bvandeusenandClaude Opus 5	9b82a95b7e	feat: Settings can re-download the Discord images the None naming broke (3999) ... The operator chose a clean re-download over relinking in place. The ~1,600 broken files can't be relinked reliably: their message ids are gone, and their sidecars collided. Settings → Maintenance → "Repair Discord downloads" previews, then applies: - Deletes every image whose path is `…/discord/None/<8 digits>_None_…`. Both the folder and the name are required, because that pair is only what the bug produced. It reuses cleanup_service.delete_images for the record and file deletes. - Sweeps the leftover collided sidecars from those folders and removes the emptied folders. - Only then clears gallery-dl's archive rows `discord%`, excluding `discordasset_%`. Upstream keys message attachments as `discord{message_id}_{num}`. Since the broken files lost their message ids, per-source forgetting is impossible. Every pre-fix Discord download is broken, and files fetched after the fix still exist on disk, so gallery-dl's `skip` won't re-fetch them. - Arms a fresh backfill on every Discord source. The apply defaults to preview at both the route and the task, runs on maintenance_long, and is never on a beat. The card uses the confirm-dialog pattern of AttachmentReclaimCard. Supporting refactors, with no behaviour change: - gallery_dl.archive_path() is the single definition of the archive location. - source_service.arm_backfill() is the mutation start_backfill already did, now shared with the sync repair. Tests (tests/test_discord_repair.py): - The archive clear leaves other platforms and Discord assets alone, and counting mutates nothing. - Case-twin artist folders are both found. - The folder sweep works. - An integration run shows only the broken image goes. A correctly named Discord file and a `None` folder under Patreon survive, and only Discord sources are re-armed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 20:52:32 -04:00
bvandeusenandClaude Opus 5	dc840feec7	fix: Discord downloads land as None/<date>_None_<name> with no post — name them from the keys gallery-dl really emits ... Operator report: a Discord source showed the wrong post time. Listing the real downloads found every Discord folder on the instance (7 artist folders, ~1,600 media) in a directory named `None`, as `<date>_None_<original name>`, next to sidecars named `<original name>.json`. The real sidecar settles why: - gallery-dl's discord extractor emits `channel` as a plain string, the message as `message_id`, and the attachment position as `num`. It has no `id` key. - The patterns asked for `{channel[name]}` and `{id}`. gallery-dl renders a missing field as "None" and carries on. - The sidecar was named `{filename}.json`, the attachment's ORIGINAL name. find_sidecar never pairs that with `<date>_None_<name>.png`, so no Discord file ever got a Post or a post date. The card fell back to downloaded_at. - Every `image.png` in a channel also overwrote the same `image.json`. Fix (gallery_dl.py): - The directory is `{channel}`. - The filename is `{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}`, unique per attachment. - A discord-scoped metadata postprocessor names the sidecar exactly like the media minus its extension, so find_sidecar's first candidate matches. A per-source filename override re-derives the sidecar name. save_metadata=False drops it. Guard (tests/test_gallery_dl_naming.py) renders the patterns through Python's formatter against a sanitized copy of the real sidecar (same keys and types, invented values). A missing key or a subscript into a string raises, which is the loud failure gallery-dl doesn't give. A positive control shows both shipped patterns fail it. Existing broken downloads are NOT repaired by this. gallery-dl's archive already records them, so a re-run skips them, and their collided sidecars no longer describe them. That repair is a separate, destructive step for the operator to decide on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 20:01:11 -04:00
bvandeusenandClaude Opus 5	98c3b74260	feat: the Latest feed uses a wide window — filmstrip cards, a day gutter and a filter rail (407 A, D, E) ... On the operator's 3432px window the feed was a 900px column, 26% of the width. They picked three options from a to-scale layout study. A — filmstrip card (PostCard.vue): - A card measures itself with a ResizeObserver. At 1100px or wider its hero gets a fixed height, clamp(260px, 34vh, 460px), and the extra images move into a 2-column grid of squares beside it. The grid cells are sized from the hero height, so the grid ends flush with the hero. - The rail cap is 4 cells in this layout (2×2, the last becoming "+N") and 5 in the narrow layout, which is unchanged. - The description clamp drops to 4 lines, because long reads happen in the expanded view. - The hero has a height, not a width, so a wide card can't grow into a full-screen post. That was the operator's constraint. D — day gutter (PostsView.vue): - The normal feed groups consecutive posts by local day (Today, Yesterday, a weekday, or a date), with post and artist counts for what has loaded. - Runs rather than date buckets, because the sort key includes resurfaced_at, which the payload doesn't carry. A resurfaced grouping gets its own heading where it actually appears, instead of being pulled out of order. E — filter rail (PostsView.vue): - At 1600px and wider, the filters and status ribbon stack in a sticky 280px left rail, and each day's heading sits in a sticky 150px gutter beside its posts. - Below 1600px the layout is exactly the old one, including the 900px column. The in-context (post_id) view gets the wide column but no rail or day grouping, so anchor scrolling is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 19:48:46 -04:00
bvandeusenandClaude Opus 5	4b4e532c56	fix: the membership status vocabulary moves beside Membership, so the Patreon client never imports the roster ... 240f11c made PatreonClient._membership import has_paid_access from membership_roster. test_gated_reason::test_no_fetch_path_can_read_the_roster failed on it, correctly: native_ingest_common is a fetch root, patreon_client is reachable from it, and no fetch path may be able to reach the roster. The roster is allowed to explain a skip, never to cause one. MEMBERSHIP_STATUS and has_paid_access are pure platform knowledge with no database behind them. They move to native_ingest_common, next to the Membership type they interpret (the same move C7 made for Membership itself). membership_roster, membership_reconcile, patreon_client and the tests import them from there. There is no re-export from membership_roster. The guard is unchanged. The lapsed-orphan skip from 240f11c stays as it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 16:07:26 -04:00
bvandeusenandClaude Opus 5	240f11c5aa	fix: one lapsed membership to a deleted creator no longer fails the Patreon roster ... The first live sweep failed with "Patreon member resource has no campaign relationship". The operator ran FC's exact /api/members request in a browser and shared the response. - It has 104 rows. The browser's settings page shows 43, because FC sends no membership-type filter and so also gets lapses back to 2016. - Exactly one row has no `campaign` key at all: a former_patron whose membership ended in 2017. Its included reward has no campaign link either, so the creator's page is gone. - Every other row, including the 4 is_active=false campaigns, has a campaign. _membership returns None for a campaign-less row only when has_paid_access says it is definitely not paying (a known lapsed status, or a free member), and iter_memberships skips it. That changes no conclusion. A lapsed membership and an absent one both mean "not paying", and no Source can match a campaign with no id. An active or unrecognised membership without a campaign still raises, because dropping a membership that might be paid would read downstream as a cancellation. Paging still counts the rows the server sent, not the rows kept. A test pins that, so a skip can't re-read an offset or stop a page short. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 16:01:50 -04:00
bvandeusenandClaude Opus 5	57c880a623	feat: the full brand mark as a faint backdrop behind every page ... The operator asked for one large version of the logo as the site background. It is pinned to the viewport behind .fc-content at about 6% strength, so it shows in the gutters and on bare page ground while cards and the nav cover it. The series reader is immersive, skips the shell, and never draws over it. It is a layered background (the page colour at 94% over logo.svg), not an overlay element with opacity. An overlay needs the content z-indexed above it, which turns every page into one stacking context under the nav's z-index 1000 and can trap an in-page overlay beneath the nav. A background changes nothing about stacking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 15:58:40 -04:00
bvandeusenandClaude Opus 5	dd766eb976	fix: the favicon and nav brand mark rendered blank — "--" inside an XML comment ... 7ca6ee0's favicon.svg had a comment naming the `--fc-chrome-rgb` custom property. `--` is illegal inside an XML comment, so the file is not well-formed XML, and a browser renders an SVG-as-image only if it parses. It fails without any error, just a blank image. Because TopNav's brand glyph is the same file, the tab icon and the nav mark both went missing on the #251 deploy. logo.svg parses and was unaffected. The comment now names the property without the hyphens, and says why it has to. tests/test_public_svgs.py parses every SVG under frontend/public. It includes a vacuity guard (the directory really contains the two known files) and a positive control (the exact defect raises ParseError), so the check can actually fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 15:57:19 -04:00
bvandeusenandClaude Opus 5	e3c516d6be	fix: a retired platform leaves the chip maps, and a fixture stops creating pixiv sources (406) ... e3fd8c6 failed two lanes, both on things pixiv's retirement correctly started refusing. Unit - test_fe_be_contract pins platformColor.js's ICONS keys to known_platform_keys(). I had kept pixiv's icon, colour and label "so existing pixiv posts don't look broken". That was wrong, and the file's own header already said why: unknown platforms fall back to a grey chip with the raw key, and that fallback is deliberately what a retired platform hits - it is how a pre-3069 deviantart row already renders. So pixiv leaves all three maps, and its posts show as a grey "pixiv" chip rather than a broken one. The header now says outright that a retired platform is removed, never kept, since the milestone plan itself got this backwards. Integration - four reassign tests built their fixture source through SourceService.create with platform="pixiv", which the validator now rejects. Reassign never reads the platform and never moves files, so any registered platform serves; the fixture uses hentaifoundry. 1294 other integration tests passed on e3fd8c6, so nothing else used pixiv through a validating path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 11:52:06 -04:00
bvandeusenandClaude Opus 5	c2f9e9cc08	docs: stop claiming pixiv, and stop claiming everything gallery-dl supports (406 step 4) ... Ships with the switch-off rather than with the code removal: a doc that promises a platform the product refuses is the Install and Public Surface area's characteristic defect. pixiv comes out of README (twice), SECURITY.md (twice), .env.example and the compose header. The stored-credential warnings now name Patreon and SubscribeStar - the accounts that usually carry a payment method. One correction beyond pixiv. README said FabledCurator follows creators on Patreon, SubscribeStar, Pixiv "and anything gallery-dl supports". That was already false: a platform not in the registry is rejected, however capable gallery-dl is. It now names the real set, which rule 171 records: Patreon, SubscribeStar, Discord and HentaiFoundry. The 3422 docs guards still hold - the key path and bootstrap variable are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9 ext-2026.9.13.1545	2026-09-13 11:45:49 -04:00
bvandeusenandClaude Opus 5	e3fd8c67d4	feat: switch pixiv off — unregistered, unreachable, and refused at dispatch (406 phase 1) ... Milestone 406 retires pixiv (rule 171) in two phases at the operator's explicit ask: switch it off, then later delete its code. This is the switch-off. Steps 2 and 3 ship together because each is a half-state of the other: unregistered but still in the extension, pixiv creator pages would offer a button the backend then refuses. Reachability removed, never gated (rule 22 - no flag, no `if platform == "pixiv"`): - platforms registry: pixiv unregistered, so /api/platforms, the source validator and quick-add all refuse it through their existing unknown-platform paths. - NATIVE_INGESTER_PLATFORMS: pixiv removed. - extension_service: pixiv's quick-add URL pattern removed (the Python half of the JS mirror). - extension: pixiv's host permissions, content-script match, platform entry and artist pattern removed; popup's pixiv branches removed; and the whole pixiv PKCE OAuth flow cut out of background.js. That last one could not wait for phase 2 - a webRequest listener on a host the manifest no longer grants is at best dead and at worst a startup failure for the entire background script. On startup the extension now also removes any pixiv refresh token a browser still holds in storage, for the same reason as the server-side credential cleanup (3980). - frontend: the extension card stops listing pixiv; SourceActions' copy of the native list drops it. platformColor keeps rendering a pixiv key so existing pixiv posts do not look broken. The guard, and why a registry change alone was not enough. A source outlives its platform: the live instance still had one ENABLED pixiv source (step 1). Tracing it: the scheduler only selects enabled rows and every platform lookup uses .get(), so a disabled row is inert - but re-enabling it and pressing Check would have routed pixiv, no longer native, straight into the gallery-dl branch, which still has a pixiv extractor. And a worker can pick up a still-enabled row before a deploy's migration runs. So run_download and verify_source_credential - the two functions every download and credential probe pass through - now refuse any platform not in the registry: an unsupported_url failure for downloads, and an inconclusive (None, not False) verify, since nothing was probed so nothing was rejected. Generic by registration, so it covers deviantart's leftovers too. Positive-controlled: a supported gallery-dl platform must still reach gallery-dl, or a guard that refused everything would pass (rule 167). Migration 0097 disables sources on retired platforms (pixiv, deviantart) and clears their failure state exactly as disabling through the app does (1285), so the stale row stops being scheduled and stops showing as failing. Nothing is deleted: removing a source can collide with uq_post_artist_external_id_null_source on real data, which is phase 2's step 6 to check. No post or image is touched. Tests: the known-platform lists drop pixiv and gain retirement assertions beside deviantart's; pixiv's positive extension cases become negative guards; the pixiv sidecar post-URL test is deleted with the behaviour it tested; quick-add rejects a pixiv URL. The pixiv client/downloader/ingester suites stay - that code stays until phase 2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 11:45:49 -04:00
bvandeusenandClaude Opus 5	0835da8a91	fix: give the roster's column zip an explicit strict=False (387 D1, B905) ... ef91fcf failed ruff's B905 lane on one zip(labels, cells) without strict=. Tests, integration and the frontend were already green on that SHA. strict=False is the deliberate side, not the quiet one. strict=True raises a bare ValueError - not SubscribeStarDriftError - and would fail the whole roster sync over a column mismatch in `details`, which nothing reads yet. That would take down reconciliation and the gated-post reasons over a cosmetic markup change, while creator identity (id, slug) never depended on the columns at all. But a shifted column would mislabel details silently (a price filed under "discord"), so a count mismatch now logs a canary warning, mirroring the feed parser's existing parse canary: diagnosable from the worker log, never fatal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 10:56:18 -04:00
bvandeusenandClaude Opus 5	ef91fcfd26	feat: SubscribeStar joins the membership roster (387 D1) ... The second platform through the seam note 3970 contracted, characterized first from a live capture of the account's /subscriptions page (note 3989). The capture lives in the gitignored captures dir; the committed fixture is hand-built with invented values and was verified tag-for-tag against it - card wrappers, both table heads, and every distinct row shape - before any code depended on it. What the page is, and the three decisions it forced: The table IS the status. SubscribeStar has no per-row status word: a creator is either in the active_subscriptions card or the cancelled_subscriptions one. The card's data-identifier is stored verbatim as Membership.status and mapped in MEMBERSHIP_STATUS, keyed on the identifier rather than the table class because the cancelled table's class names the same list differently (for-unsubscribed_users). The creator's numeric data-user-id is the key, not the slug. A slug re-keys when a creator renames; the old row stops appearing; and a disappearance is exactly what reconciliation reads as a lapse. Keyed on the slug, a rename would have told a paying subscriber they had cancelled. The slug rides as vanity, where the identity join already looks for a handle. Price is kept as text, never parsed into amount_cents. A bare $ names no currency and a page price is not proven to be the charge - 3970 finding 4. Tier names live behind a per-row modal and are not fetched. Refusals, because SubscribeStar offers nothing like Patreon's meta.pagination.total and every conclusion downstream is drawn from absence. The parser raises when: the active card is missing (auth error on a login/age wall, drift otherwise); a row lacks a numeric creator id or a creator link; anything renders after a card's table; or the page carries a page= link. Both cards are paginatable (app#embed_pagination) and the captured account was too small to show what pagination looks like, so possible pagination is a roster FC cannot prove complete. A loud error on a larger account beats a quiet half-list. A missing cancelled card is not drift, and a creator in both tables is reported once, as active. Fetched from subscribestar.adult, not the .art the capture came from: FC's requests never clear the .art age wall with the 18+ cookie (1259, 1284). Whether /subscriptions on .adult authenticates exactly as .art did in the browser is untested - if not, the sweep records a visible error and C6 shows its unavailable rung. The seam leak D1 found. Note 3970 promised a second platform would be one builders line plus the client method. The sweep instead called current_user_id() on every client, which only Patreon's has, so SubscribeStar would have raised AttributeError on the first sweep. roster_user_id probes it with getattr, the same way the sweep already probes iter_memberships. Two existing tests were passing for the wrong reason and now can fail: - "a platform that has never been characterised says nothing" named SubscribeStar, and stayed green only because active_patron is not a SubscribeStar word. Now uses hentaifoundry, with a positive SubscribeStar test beside it. - the freshness test gave SubscribeStar a Patreon word, so the vocabulary excluded it and deleting the freshness gate outright would have left it green. It now uses cancelled_subscriptions, making the gate the only thing that excludes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 10:53:01 -04:00
bvandeusenandClaude Opus 5	529d4bff57	test: tie the install docs to the code they quote (3422 follow-up) ... 3422 was already fixed. Commit 86abaf0 applied option 1 in full and it is on main: .env.example carries the bootstrap section with the backup warning, README explains the refusal and why it is deliberate, docker-compose forwards the variable, and the milestone-362 smoke gate that FOUND the bug now sets it (build.yml:1159) and passes. The issue's premise - "CURATOR_BOOTSTRAP_NEW_KEY appears nowhere outside backend/" - is stale. What was left is the dependency that fix created. README.md and .env.example now both print the literal error text, the literal key path and the variable name, because a stranger greps for the string their terminal showed them. That is the right call and it means two user-facing files now depend on this module's wording with nothing connecting them - the install surface's characteristic defect, one rename away from a README that sends strangers to a path that does not exist. Three guards, all presence checks on both sides. An absence check against prose would pass for the wrong reason the moment a sentence were reworded (snippet 3352): - the raised message still contains the sentence README reproduces, the variable both docs say to set, and the restore-rather-than-mint alternative the whole refusal rests on; - both docs still name _CREDENTIAL_KEY_PATH and the variable, read from the code rather than retyped, so a rename fails here; - compose still forwards the variable - without that line the docs' "set it in .env" is silently inert and fails identically to not setting it. Option 2 (mint when the credential table is empty) is deliberately NOT done. The issue's own guidance is "(1) now, (2) if the friction proves annoying", and the friction has not been reported. Worth recording that its predicate checks out exactly: the Fernet key protects Credential.encrypted_blob and nothing else - no other Fernet user exists - so "no credential rows means nothing can be made undecryptable" is provable rather than probable. The cost is placement: create_app() is sync and constructs the key before any engine exists, so the check cannot live where the failure is. entrypoint.sh, which already runs alembic against the DB, is the natural seam. Only the web role is affected; the Celery roles build the key lazily inside tasks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 21:31:21 -04:00
bvandeusenandClaude Opus 5	eb6e0df858	feat: the empty front door offers to find what you already subscribe to (387 C6) ... B4's on-ramp read "add a credential, add a source" - the manual path, and the one that makes a new installer retype creators they have already told Patreon about. With C4 in place the app can just look them up. This is the step where the milestone's original framing actually lands on screen. Four rungs, chosen by ONE predicate rather than four independent v-ifs. The rungs are mutually exclusive by construction and the rung shown matches what is actually POSSIBLE: no credential -> Add a credential. Discovery is not offered, because a button that cannot work is worse than its absence. credential, never synced -> Find what you already subscribe to. Queues the C3 sweep. synced, unmatched > 0 -> the count, linking into C4's bucket 1. synced, nothing unmatched-> the manual add, because there is genuinely nothing to discover. The fifth state is rule 164's. A sweep that has been ATTEMPTED and never succeeded reads as "couldn't reach patreon", with the error type and the manual path still open - not a spinner, not a crash, not a retry button that will fail identically. An install with no outbound network lands here. That is deliberately narrower than "there is an error": a roster that synced once and failed since is NOT unavailable. It has a roster, just an ageing one, and C4's freshness gate already handles that. Collapsing the two would hide a usable roster behind an error banner. Still offer, never auto-add. An empty front door is exactly where "just add all thirty" is most tempting and most wrong - thirty backfills on first boot - so the rung carries the count and sends them to the picker. No new machinery: three existing stores (credentials, membershipSync, membershipReconcile). Reconcile is fetched only once something has synced, since bucket 1's count is meaningless before that. Every load swallows its failure, because this screen renders on an install that can reach nothing. Ten tests on top of B4's, including the mutual-exclusion property asserted directly. Its phrases are each pinned to a single template line and both sides whitespace-normalised - a phrase spanning a line break would never match, and a mutual-exclusion check whose phrases never match passes vacuously (rule 167). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 20:19:42 -04:00
bvandeusenandClaude Opus 5	4533e036ac	refactor: the membership seam's contract type is the seam's, not Patreon's (387 C7) ... Membership moves from patreon_client to native_ingest_common, beside PostRecordOutcome, for exactly the reason that one lives there: it is the seam's contract rather than the first platform's. Left where it was, D1 would have had to import the shape it implements from the module of the platform it is being mirrored FROM - which inverts the dependency and is how a seam advertised as portable quietly stays Patreon-shaped. Found by C7's own pass, which is the point of running C7 before D1 rather than writing it up afterwards: this is invisible while there is only one implementer and load-bearing the moment there are two. No behaviour change. Three files, no shim (rule 122): patreon_client imports it, the dataclass keeps its docstring, and the test imports from the seam's home. The docstring gains what the contract owes a second platform - that a missing field supplies the empty answer and never a guess: no tiers -> [], no pledge -> None (absent stays distinguishable from zero, since "free" and "we don't know" are different answers), no vanity -> None with identity falling back to the URL tail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 20:05:11 -04:00
bvandeusenandClaude Opus 5	6f5ea5d1d3	fix: the provenance panel hotlinked the CDN for images already on disk (3965) ... Two surfaces render a post's HTML body. get_post did _localize_inline_images(sanitize_post_html(...)); provenance_service._post_dict called sanitize_post_html alone. So opening the Provenance panel fetched images from Patreon's CDN for files FC had already downloaded - the archive reaching out to the platform to display what it had archived, which is the thing 830 Phase 2 set out to stop. The same bodies also break when a CDN URL expires or a post is removed, while the identical local copy sits unused. The fix is the shape, not the call. Those were two separately-callable halves and only the first looked mandatory, so a second caller was always going to do half of it. render_post_body in the new services/post_body.py is the whole pipeline in one call, and sanitizing-without-localizing is no longer a reachable operation. _localize_inline_images moves there verbatim; post_feed_service loses five imports that went with it. _post_dict becomes async and takes the session. Both call sites are already inside async methods, so for_image's list comprehension awaits per entry - fine, because localization issues ZERO queries for a body with no inline <img>, which is most of them. Recorded that early exit in the module so the loop isn't "optimized" into a batch without a measurement. Deliberately NOT fixed: provenance still names the same columns url/title/date where the feed says post_url/post_title/post_date, and its description_translated is full text where the feed truncates to DESCRIPTION_LIMIT. That is 3965's wider half - a breaking payload change for ProvenancePanel with no second reason to spend it today. Noted in _post_dict's docstring so the next reader knows it was seen and left. Four regression tests on the provenance path, covering both entry points plus the two refusals the feed already pins: an uncaptured image stays hotlinked (a broken local path is worse than an intact remote one), and a filehash owned by another artist never leaks in. Reverting render_post_body to a bare sanitize fails the first two. Recorded as snippet 3968. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-12 19:57:00 -04:00
bvandeusenandClaude Opus 5	2862fadcb1	fix: the roster guard's import walk resolved package __init__ imports wrongly (387 C5) ... Two of C5's three structural tests errored in CI with ValueError: PosixPath('.') has an empty name. The walk special-cased a package's __init__.py, dropping the __init__ component before computing what `from .` refers to — which made `api/__init__.py`'s `from . import health` resolve to the app root instead of to `api`, and `celery_app.py`'s `from . import celery_signals` resolve to the empty string, which is what actually crashed. The special case was never needed: `parts[:-1]` already gives the CONTAINING package for both forms, because `services/foo.py` drops `foo` to leave `services` and `api/__init__.py` drops `__init__` to leave `api` — exactly what `from .` means inside each. The level slice is now clamped at 0 as well; an import climbing past backend/app left the tree, and the unclamped negative index wrapped and resolved to the wrong module rather than to nothing. A module directly under backend/app doing `from . import x` still yields no package prefix, and there the alias alone IS the dotted name - handled explicitly rather than by falling through into a Path built from an empty string. The two positive controls earned their place immediately: they are what failed. Without them the walk would have resolved almost nothing and test_no_fetch_path_can_read_the_roster would have passed on a broken walker, reading as coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-11 23:02:18 -04:00
bvandeusenandClaude Opus 5	aa765f0a72	feat: say why the posts are invisible, without ever deciding they are (387 C5) ... A3 made a tier-gated source say "47 posts you can't see". The roster turns that into a reason: the membership ended, or the tier doesn't reach these posts, or it's a free follow. Rendered under A3's count in the health tooltip, quieter than the count it explains. FREE is a fourth case the step didn't enumerate, and it earns its own sentence. has_paid_access collapses "former patron" and "current free follower" to the same False, so deriving the reason from that boolean would tell a free follower "you're not a patron any more" - a false statement about a state they were never in. gated_reason reads the status axis first, calling has_paid_access with is_free_member forced off, then splits on the free flag. Silence is the default, and there are four ways into it: campaign absent from the roster, roster stale, platform never swept, status word not yet characterised. All four send null and the count stands alone. The frontend has no fallback sentence either - a default would turn "we don't know why" into a reason, which is the one thing this step must not do. The line that must not be crossed is pinned structurally rather than by inspection: test_no_fetch_path_can_read_the_roster walks the transitive first-party imports from the fetch roots and asserts the roster is unreachable. FC runs no local verification (rule 85), so a guard cannot be falsified by hand before it lands - it carries two positive controls instead, proving the walker finds roster imports that ARE there, one direct and one through a hop, so the real assertion can never pass merely because the walk resolved nothing. C4's identity loop moved to membership_roster.pair_sources_with_memberships when C5 became its second caller; two copies would let the Subscriptions row and the reconciliation card disagree about which creator a source IS. Three test files were each building PlatformMembership rows with their own drifting helper - consolidated into tests/roster_builders.py, same family as issue 3109. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-11 22:59:29 -04:00
bvandeusenandClaude Opus 5	de11c14448	refactor: one relative-time formatter, not three ... MembershipRosterCard had grown its own ago(iso) helper, a near-copy of utils/date.js::formatRelative. C4's card was about to become a third copy before a hook caught it. Both now use the shared helper. The hand-rolled copy was also slightly wrong in ways the shared one is not: it floored everything under a minute to '1m ago', and would have rendered NaNm ago for a null timestamp had a caller ever reached it without a v-if guard. Only sub-minute output changes, which no spec exercises - membershipRosterCard.spec.js seeds its rows at exactly 1h and 9d, where both helpers agree, and asserts on literal phrases rather than time strings. Recorded formatRelative as snippet 3959 so the next component is offered it instead of deriving a fourth copy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 22:48:24 -04:00
bvandeusenandClaude Opus 5	005680f234	fix: one source edit no longer wipes the campaign id and the backfill position ... Source.config_overrides carries two unrelated things under one column: the operator's per-source download settings, and state FC writes for itself. update() treated the whole column as operator-owned and assigned it wholesale, so a dialog save discarded patreon_campaign_id and the entire #693 backfill state machine. Not a hand-edited-JSON edge case: SourceFormDialog's structured tab rebuilds the object from two fields, so saving without touching anything was enough. _merged_config now merges: the operator's keys replace wholesale (removing a key must still remove it), FC's keys survive and are applied LAST so a stale echoed cursor cannot roll a walk backwards. App-managed is _-prefixed or *_campaign_id, both matching data already on disk, so no migration. Preserving the id exposed a bug the wipe was MASKING: nothing cleared it when a source's URL changed, and patreon_resolver reads that cache before attempting any lookup. A repointed source would have resolved the old creator forever and 387 C4 would have reported a confident wrong match. So update() drops *_campaign_id when the URL actually changes - but keeps the backfill cursor, which the walk's own stall guard validates and which is expensive to rebuild. test_update_changes_fields was DOCUMENTING the bug: it asserted config_overrides == {videos: False} on a source whose create() had armed _backfill_state, so it could only pass because the state had been destroyed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 21:50:28 -04:00
bvandeusenandClaude Opus 5	fc136006b7	feat: what you pay for, against what FC actually follows (387 C4) ... Reconciliation in Subscriptions, asymmetric on purpose. Subscriptions FC does not follow get a per-row add; sources the roster cannot account for are REPORT ONLY (operator decision) and link to the list on the same page. No one-click disable, so no disabled-reason column and no migration. The membership<->source join lands as a SHARED resolver in membership_roster, not inline here: E4 now uses it as its negative check, so the two features cannot give different answers to 'is this membership already tracked?'. Keys on the exact cached campaign id FIRST and the URL handle only as fallback, because the id is written only after a source has been walked once. Read via any <platform>_campaign_id override rather than naming Patreon's, per rule 169. The report-only bucket is gated on roster freshness and carries a per-row basis, so 'your membership says former patron', 'we know this id and it is absent', and 'we only have a handle' stay three different sentences. has_paid_access None never reads as lapsed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 21:38:03 -04:00
bvandeusenandClaude Opus 5	f8614d437d	fix: a self-contradicting test fixture, and import order (E4 follow-up) ... Two failures on 51e78a3, neither in the shipped logic. **The integration failure was a TEST bug, not a code bug.** `test_a_weak_name_needs_the_declaration` set `display_name` to something deliberately weak but left `_membership`'s DEFAULT vanity, which matched the artist slug exactly. So the name signal was legitimately 1.0 and the matcher was right to propose — my assertion of 0 was asserting the wrong scenario. Both identity fields now have to be weak for the test to mean what it says, and the arithmetic was checked before pushing: 0.39 without the declaration, 0.74 with it. Worth keeping: a fixture whose fields disagree with each other will pass or fail for reasons unrelated to the property under test, and this one was one default away from silently testing nothing. **The lint failure was import order** — `artist_membership_service` sorts before `credential_*` and I inserted it after. Second isort slip this session from patching an import block with a script rather than reading it back; the repo has a rule about exactly this (#102). Everything else passed on that SHA: 1256 tests, the frontend suite, and migration 0096. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 08:02:08 -04:00
bvandeusenandClaude Opus 5	51e78a329b	feat: offer the creator you already track as the one you subscribe to (388 E4) ... **The verification the step asked for came back "not the schema".** `Source.artist_id` is a plain FK so many sources per artist already works; `POST /api/sources` already takes an `artist_id`; the add-source dialog already has an artist autocomplete that attaches to an EXISTING artist; and `SourceService.reassign` already moves a source between artists WITH post and image re-attribution. A sweep for one-source-per-artist assumptions found only `func.count()` calls — the opposite of assuming one. So no parallel association table was built for a relationship the schema already expresses (rule 28). What was missing is FC OFFERING the link, and that is all this adds. **Accepting adds a SOURCE. It never merges two artists.** That asymmetry sets the whole posture: adding a source is trivially undone, while a wrong merge silently mixes two creators' work and corrupts tagging, series and provenance downstream with nothing left to tell them apart by. A test asserts the artist count is unchanged by accepting. The weights encode the judgement rather than a code path doing it — name 0.65, declared 0.35, cut at 0.60 — so that: * an EXACT name match alone proposes (same slug on both sides is strong, and demanding corroboration would propose almost nothing); * a CONTAINMENT match alone does not ("art" sits inside "artgirl"), and short slugs are excluded from containment entirely because a 3-character slug is inside a great many longer ones; * the declaration ALONE never proposes, because a creator may link another creator's Patreon and a link is not a claim of identity. A guard test pins all three against WEIGHTS directly and says not to fix a failure by moving the numbers. Two corrections carried forward from earlier steps rather than rediscovered: * The declaration is NOT read from `ExternalLink`. `SUPPORTED_HOSTS` is file hosts only and `host_for()` returns None for patreon.com, so no row is ever written for one — the same trap that caught E5 for Discord invites. It reads the raw body, because these links live in an `href` and `html_to_plain` discards attributes. * `vanity` is not a column: C1 modelled the roster before any platform was characterised, which is exactly what `details` exists for. `vanity_or_none()` reads it from there and falls back to the URL's last segment, so a row written before the field was understood still resolves. Two fixes during the writing. `accept()` first created a bare `Source()`, skipping the platform/URL validation, duplicate check and #693 backfill-arming that a hand-added source gets — a second, quieter way to create a source is how two paths drift until one is subtly broken; it now goes through `SourceService.create`. And the candidate query used a bare `exists().where()`, which has no FROM to correlate against; now `select(...).exists()`. Chained onto the roster sweep rather than given its own beat entry: a suggestion can only be as good as the roster behind it, so any other cadence would just propose from staler data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 07:56:16 -04:00
bvandeusenandClaude Opus 5	61abd0007c	fix: stdlib imports split by a stray blank line (isort I001) ... My scripted patch inserted the new stdlib from-imports after `import logging` with a blank line between them, which isort reads as a group boundary. Ruff-only failure — every test passed on 751e7dd, including the full integration suite and migration 0095. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 23:46:02 -04:00
bvandeusenandClaude Opus 5	751e7ddb9f	feat: the membership sweep, and the state that makes its failures readable (387 C3) ... A daily sweep that walks each platform's roster into `platform_membership`. Daily because memberships change on a BILLING cycle, not a download cadence. **`membership_sync` is the part that earns its keep.** Without it three very different situations are one indistinguishable state — the account subscribes to nothing, the sweep never ran, the sweep failed — and all three leave zero rows in `platform_membership`. "You are tracking 12 sources you no longer subscribe to" is correct in the first case and an invitation to cancel things the operator is actively paying for in the other two. So C4 gates its CONCLUSIONS on `last_success_at`, not merely its display, and `roster_is_fresh` is computed server-side so no caller can forget to. Two timestamps rather than one: `last_attempt_at` moves every run, `last_success_at` only on a clean walk. The gap between them is the signal — a sweep hammering a broken credential every day must not look healthy because it ran recently, and there is a test for exactly that. Rejected shortcuts, both tempting: `MAX(platform_membership.last_seen_at)` cannot tell "synced fine, found nothing" from "never synced"; `task_run` is worse, since its retention prunes ok rows after 24h and a sweep that last succeeded three days ago would leave no trace at all. **The fetch completes before anything is written.** That ordering is the safety property: a walk that dies mid-pagination writes nothing, so a failure can never leave a roster half this week's and half last week's. `touch_membership` never deletes, so a failure cannot empty the roster either — but "intact" should mean intact, not merely non-empty. Rule 89's four, each where it actually lives: recovery is "run it again" (upsert, no deletes); retention is C1's age-out-never-delete, because disappearing IS the signal; the wall-clock deadline is per-platform and distinct from the per-REQUEST timeout the client already has (rule 156 — a paginated roster answering every page slowly-but-within-timeout would never trip that one and would sit on a worker indefinitely); duration comes from the existing TaskRun signal plumbing. **A bug caught in review, not production:** the broad `except Exception` would have swallowed Celery's SoftTimeLimitExceeded — which is an ORDINARY Exception subclass, not a BaseException — letting the sweep run past the soft limit into the hard one, where it is SIGKILLed mid-transaction. A sweep that cannot be stopped is worse than one that fails. Now re-raised explicitly, with a test that also asserts SoftTimeLimitExceeded is still an Exception, so the re-raise cannot quietly become dead code. Rule 164 is why this ships with UI rather than backend-only: a roster that never synced must be VISIBLE as such. The card says "never synced" in words and states no count at all — rendering it as 0 is the precise conflation the whole step exists to prevent — while a real zero behind a real sync is reported as zero, because that one IS an answer. Pinned in both directions. Three independent gates decide whether a platform is swept — registered here, client exposes `iter_memberships`, credential exists — each silent, so adding SubscribeStar (D1) is one line and nothing else. A missing credential is not an error: recording a failure would light up the UI for a feature never enabled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 23:40:48 -04:00
bvandeusenandClaude Opus 5	afcde8e457	feat: PatreonClient.iter_memberships — the roster seam (milestone 387 step C2) ... Built on C0's real capture (Scribe note #3886), not on API docs — gallery-dl has no membership extractor and Patreon's public v2 API is the CREATOR surface behind OAuth, so the rule-130 reference had to be a characterized response. **The request is deliberately minimal, and that is a privacy decision.** The browser's own include set pulls `latest_pledge.card`, and those card resources come back carrying the ACCOUNT HOLDER'S EMAIL in `merchant_name`; `address` is in there too. Copying the query string wholesale is the obvious move and would have FC fetching payment PII it has no use for and can only mishandle. We ask for `include=campaign,reward` and nothing else, and a test asserts on the params actually sent so nobody widens it back. **We do not send `filter[membership_type]`.** The browser sends the six buckets its settings page displays, which excludes lapsed memberships — and a DISAPPEARANCE is precisely the signal the roster exists to read. Filtering here would manufacture the event C4 acts on. Two corrections the capture forced, both now in code: * **The filter vocabulary is not the status vocabulary.** I had read the six filter words off a screenshot and was about to write them into MEMBERSHIP_STATUS as the enum. The body shows `patron_status` carrying `former_patron` — absent from that filter — on a row the filter selected as `free_member`. So the map is taught exactly the two OBSERVED values, and `declined_patron` stays out despite looking obviously right: believing the filter is the mistake that was just caught. * **Free membership is a boolean, not a status.** `has_paid_access` gains an `is_free_member` axis, because `active_patron` alone would report a free follower as a paying patron and C4 would never offer to clean it up. Honest limit, stated in the docstring: the capture has no ACTIVE free member, so it shows the separation is possible, not that it occurs. C1's tripwire test did its job — it was written to fail the moment anyone populated the status map, and updating it here IS the confirmation step, done with the capture rather than ahead of it. `_fetch`'s retry/backoff/auth-vs-drift/Retry-After logic is extracted to a shared `_request` so the roster rides the same path rather than growing a second copy — two copies would drift, and the half that drifted would be the one that only runs daily. Every error message and log line renders byte-identically for the posts path, so the existing tests pin the refactor. Pagination is driven by `page[offset]` against `meta.pagination.total`, never by `links`: the response's own `links.first` is built WITHOUT the `/api/` prefix the request uses, so following it would hit the web page. An empty page is terminal regardless of what the total claims, so a server reporting more rows than it hands over cannot spin the walk forever. Drift is stricter here than on the posts path, on purpose: a missing `meta.pagination.total` raises rather than returning a short list, because a truncated roster reads downstream as "you cancelled those" — the worst wrong answer this feature can give. `current_user_id()` is marked INFERRED, not characterized: C0 captured /api/members, not /api/current_user, so it relies only on the JSON:API envelope this API demonstrably uses elsewhere, and raises drift rather than returning something plausible if that is wrong. The fixture is derived from the real capture with every piece of account data replaced (the raw capture stays gitignored). Six members, each earning its place: a former patron with a null pledge, an active patron with no tier, an annual cadence, a previous_pledge whose included resource has no `relationships` key at all, and a reward priced in CAD beside a USD charge — the trap that makes reading `reward.amount_cents` report a number the operator was never charged. A leak check caught a free-membership-subscription id and six real campaign launch timestamps before any of it was staged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 22:26:01 -04:00