PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or
artist left the row behind rather than taking it. Nothing ever pruned those
rows, and nothing in the repo had ever unlinked a file under the attachment
store — so both rows and bytes accumulated permanently, invisible to every
existing diagnostic.
Why a disk->DB reconciliation rather than a row sweep: the store is
sha-addressed and idempotent, so ONE blob backs MANY rows. Deleting a row
does not free its blob, and since the artist cascade (#3066) now deletes
its attachment rows outright, a freed blob has no DB pointer left to find
it by. Walking the store and asking "does any row still reference this
sha?" catches orphans from every cause, including ones no future delete
path will think to report.
Preview and apply share `_orphan_attachment_conditions` (rule 93). The
dry-run derives its surviving-sha set by NEGATING that same predicate, so
it is honest about blobs the delete would free rather than counting them as
still-referenced — the one place this was easy to get backwards, so it has
its own parity test.
Guards, each with a reason:
- A blob is written before its row commits, so a just-stored file legitimately
has no referencing row. Files under 6h are never judged — same guard and
reasoning as ORPHAN_TEMP_MIN_AGE_HOURS.
- `.partial` staging files belong to cleanup_orphaned_temp_files; skipped
rather than raced.
- The sha is parsed as the first 64 chars, not via Path.stem: store() takes
the extension from the source filename, and a URL-encoded basename yields a
multi-dot suffix that would make stem eat part of the sha.
- A 900s walk budget reports partial=True instead of running to the task's
hard limit (rule 89).
- TASK_STUCK_THRESHOLD_MINUTES override at 30 (= time_limit 25 + 5). Without
it a healthy 20-minute walk is phantom-flagged 'RecoverySweep' at the bare
5-min default — the #883 failure class; its invariant test is mirrored here.
Defaults to the safe preview at both the task and the route, unlike the other
maintenance triggers: this apply unlinks files. Operator-triggered only,
never on a beat.
Ships with its UI (rule 27): AttachmentReclaimCard in Cleanup → Duplicates &
leftovers, built on the existing useMaintenanceTask/MaintenanceTile shapes, so
a run survives navigating away. Surfaces files_failed and partial explicitly,
since both change what the numbers mean.
Also promotes humanBytes to utils/bytes.js — it was byte-identical in
VideoDedupCard and GatedPurgeCard and this card would have been the third
copy. The three divergent `formatBytes` helpers are deliberately left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four remaining run-1895 failures were stale expectations, not
predicate bugs — prune/reset returned the right counts, but these tests
verified no-deletion by counting the ENTIRE tag table (or asserting the
full kind set), which now includes the three seeded hygiene tags that
survive prunes and resets by design. Filter is_system=false with a
pointer to #128 so future system tags cannot re-break them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
Operator-flagged: the recovered defective files live DEEP in their artists'
back-catalogues — the normal download cadence (by design, via the seen-gates)
will never re-walk them, so recovery's source re-check alone can't bring them
back. The durable per-post handle is the ExternalLink row, which survives the
image delete:
- services/external_links.refetch_links_for_post: reset settled links to
pending (fresh attempt budget, in-flight left alone) + dispatch their
fetches; sha-dedupe at import discards payload files that still exist, so
only the missing file lands.
- recover_defective_image now captures the image's post ids BEFORE the delete
cascades provenance away and resets those posts' links — future recoveries
are surgical automatically (response gains links_reset; source re-check
stays for gallery-dl-native files within walk reach).
- POST /api/admin/posts/refetch-external {external_post_id, source_id?} — the
manual tool for the three files recovered before this fix existed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
Autoscaler (agent 2026-07-02.5): the buffer-occupancy signal alone would peg
downloaders at DL_MAX while the bandwidth CAP — not concurrency — is the real
constraint (8 streams sharing 8 MB/s move no more data than 4). Growth is now
gated on the pipe having headroom (net < 85% of cap) and a pipe pinned at the
cap (>= 95%) sheds streams down to 3; dead band prevents flapping. The UI hint
says 'holding at the bandwidth cap' and /status reports bw_capped, so the
behavior is legible without tests that need the ML stack.
Reset content tagging: stays a FULL-instance reset (operator's call), but now
lives in a fenced 'Danger zone' section on Cleanup and the apply is gated by a
preview-derived confirm token (mirrors the Tier-C bulk-delete pattern — stale
counts are rejected server-side). Copy no longer claims suggestions repopulate:
it says plainly the heads' training examples are deleted and re-tagging starts
fresh. Moved out of TagMaintenanceCard into DangerZoneCard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
- ml-backfill-daily: the CPU tag_and_embed backfill raced the GPU agent's
daily embed backfill for the same NULL-embedding images at ~100x the cost
(B1 audit verdict, milestone #124). The backfill TASK stays — the manual
/api/ml/backfill button remains the deliberate CPU fallback pending B3.
- purge-legacy: one-time IR-migration cleanup, dry-run verified 0 targets on
the live library before removal (A2 audit, milestone #123). Fully retired
per rule 22: tile, store action, route, service fn, tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
Cluster B, milestone #99. TagService.merge_preview(source, target) computes the
same counts the apply produces (rule 93 parity) without mutating: images_moving
(source links the apply UPDATEs), images_already_on_target (links it drops),
source_total, series_pages, will_alias (_keep_as_alias), a kind/fandom
compatible flag (surfaced, not raised, so the UI can warn), and up to 6
thumbnails of the moving images. The admin /tags/<dest>/merge route gains a
dry_run flag returning the preview JSON.
Tests: preview moving-count == apply merged_count (parity), incompatible flagged
without raising, self/missing raise, admin dry_run returns preview + no mutation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX
DRY pass follow-up (note #1026). Five handlers returned the identical
jsonify({task_id, status:queued}), 202 shape; extract _queued(async_result).
Consumers routed through it: tags_normalize (live branch), trigger_reextract_archives,
trigger_prune_missing_files, trigger_dedup_videos, trigger_purge_gated_previews.
trigger_vacuum stays bespoke (returns no task_id — the UI doesn't poll it).
Added route-level tests for all five consumers (these trigger endpoints had no
route coverage before): 202 + task_id via _queued, and the dry_run flag threading
through to dedup/purge-gated. Behavior unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DRY pass on the cleanup/admin destructive-ops surface (task #753, hardened
process #594). Five Tier-A endpoints repeated the same get_json -> dry_run ->
run_sync(service_fn) -> jsonify block verbatim. Extract _run_dry_run_op(service_fn,
**kwargs); the five route handlers now delegate. reconcile keeps its source_id
validation and passes it through **kwargs.
The cleanup_service predicates were already shared between preview and apply
(find_*_conditions / find_duplicate_post_groups) — the post-data-loss fix — so no
backend-logic change; this is purely the HTTP-handler boilerplate.
Consumers (all routed through the helper, verified no copy left behind):
prune_unused_tags, prune_bare_posts, reconcile_duplicate_posts (+source_id),
purge_legacy_tags, reset_content_tagging.
Added route-level tests for prune-bare (apply) and reconcile (apply + source_id
passthrough + invalid-source_id 400) — the two helper consumers that previously
had only service-level coverage, so every consumer is exercised at the route.
Findings B (queued-response helper) and C (store dry-run POST helper) identified
but not applied this pass (operator scoped to A). The card preview->commit state
machine is deferred to a frontend pattern-consistency sweep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wipe every general + character tag so the operator can re-tag from scratch via
the Camie auto-suggest, while PRESERVING fandoms, series (+ series_page order),
and each image's stored tagger_predictions (so suggestions repopulate
immediately). One set-based DELETE FROM tag WHERE kind IN ('general','character')
— the five tag-referencing tables all cascade, so applications + aliases +
allowlist + rejections + centroids clear automatically; series tags aren't
deleted so series survive; Tag.fandom_id is SET NULL so fandoms are untouched.
Reuses the established dry-run-preview -> confirm pattern: cleanup_service.
reset_content_tagging() + POST /api/admin/tags/reset-content +
TagMaintenanceCard section with a backup-first warning and a red confirm
showing exact counts (tags by kind + image applications). Irreversible except
via DB backup restore; the wipe only fires when the operator confirms.
Tests: service dry-run counts + live delete preserves fandom/series/series_page
while content tags + their image_tag cascade away; API dry-run wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The TABLESAMPLE showcase reads physical blocks (bloat-sensitive), and the
periodic prune/backfill/recovery tasks churn dead tuples faster than
autovacuum always keeps up — so explicit maintenance earns its keep here.
- tasks.maintenance.vacuum_analyze: VACUUM (ANALYZE) over high-churn tables
(VACUUM_TABLES) on an AUTOCOMMIT connection (VACUUM can't run in a txn).
Scheduled weekly via Beat; also operator-triggerable.
- _sync_engine.get_sync_engine(): expose the process engine for the
autocommit connection.
- GET /api/admin/maintenance/db-stats: per-table n_live/n_dead/dead_pct +
last (auto)vacuum/analyze from pg_stat_user_tables — visibility, not a
black box.
- POST /api/admin/maintenance/vacuum: enqueue the task on demand.
Tests: vacuum task runs + reports tables; db-stats shape; trigger queues.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed the app/client fixtures duplicated across 36 test files (two
variants: separate app + client(app), and a self-contained client() that
called create_app inline) and the now-unused create_app imports. Both
fixtures now live once in conftest.py. test_suggestions_bulk keeps its
import (builds the app inline in two tests); test_health drops its local
client + unused pytest_asyncio.
Net -415 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-confirmed 2026-05-28: the BlenderKnight:* tags are `archive`
kind (caught by the kind purge), but the `source:patreon`-style tags
are IR's old `source` kind that fell back to `general` during migration
(FC's enum has no `source` kind) — so they can't be matched by kind.
Broadened the purge to a two-rule match and renamed it for accuracy
(all dev-only, unreleased):
- cleanup_service.purge_tags_by_kind → purge_legacy_tags. Predicate is
now `kind IN (archive, post, artist) OR name LIKE 'source:%'`
(LEGACY_NAME_PREFIXES). Preview classifies each row into by_kind OR
by_prefix (source:* counts once under the prefix bucket regardless of
its general kind).
- endpoint /tags/purge-retired-kinds → /tags/purge-legacy. dry-run
returns by_kind + by_prefix + count + sample.
- store purgeRetiredKindTags → purgeLegacyTags.
- Tag Maintenance card copy + breakdown updated to show both buckets;
button reads "Preview/Delete legacy tags".
Tests updated + extended: dry-run reports by_kind {archive,post,artist}
AND by_prefix {source:*}, plain general/character tags survive; commit
deletes both the kind-matched and source:*-matched rows and leaves the
rest.
Operator-flagged 2026-05-28, two asks.
**1. Provenance posts ate the panel.** Each post card rendered its full
description (180px scroll box) inline, so a few posts pushed everything
else off-screen. ProvenancePanel now collapses the description by
default behind a per-post "Show description ▾ / Hide ▴" toggle (state
keyed by provenance_id, reset when the viewed image changes). Cards stay
compact — platform/date/title/meta/actions — and the operator expands
only the descriptions they want.
**2. Purge tags of retired/system kinds.** The IR migration left
`archive`/`post`/`artist`-kind tags (e.g. `BlenderKnight:Hannah_BJ_Loops`)
that FC no longer creates — the tag input only makes
character/fandom/series/general, and provenance + artists are their own
systems now. (meta/rating were already hard-deleted by alembic 0023.)
- cleanup_service.purge_tags_by_kind(kinds=PURGEABLE_TAG_KINDS) — counts
(dry_run) or deletes tags whose kind ∈ (archive, post, artist).
CASCADE clears the image_tag / alias / allowlist / etc. rows.
- POST /api/admin/tags/purge-retired-kinds (Tier-A, dry-run preview
returns per-kind counts + sample names — the preview IS the
verification of exactly what'll be deleted before committing).
- Tag Maintenance card gets a second section: "Preview retired-kind
tags" → per-kind breakdown + sample → "Delete N retired-kind tag(s)".
Tests: dry-run counts by kind (general survives), commit deletes only
the retired kinds (general + character survive, retired count → 0).
NOTE: a dry-run preview will show exactly which kinds/counts are
present. If the operator's noisy tags turn out to be `general` (e.g. an
IR `source:patreon` that fell back to general during migration), they
won't be caught by the kind purge — the preview makes that visible so
we can decide on a name-based pass separately.
Operator-flagged 2026-05-27: walk the whole project for the same shape
as the min-dim Delete-button silent failure (crypto.subtle TypeError
on plain HTTP). FC runs over plain HTTP per the homelab posture;
Secure-Context-gated browser APIs are undefined on the production
origin.
**Audit results across `frontend/src/`:**
crypto.subtle.digest — 2 sites:
- MinDimensionCard (fixed 2026-05-27)
- BulkEditorPanel (THIS FIX)
navigator.clipboard — 1 site, already guarded:
- utils/clipboard.js writeText with execCommand fallback
serviceWorker / mediaDevices / Push / Web USB|HID|Bluetooth|Serial /
cookieStore / queryLocalFonts / WebAuthn / geolocation
— NOT USED, nothing to fix
Extension scripts (background.js) use crypto.subtle but run from
moz-extension:// which IS a Secure Context — left as-is.
**BulkEditorPanel double bug**
The bulk-delete UI on the gallery selection had been broken since
FC-3k shipped, in two ways:
1. `crypto.subtle.digest` swallowed TypeError on plain HTTP — modal
never opened. Same symptom as min-dim.
2. Even on HTTPS, the modal's `kind="images-selection"` produced
`delete-images-selection-<sha8>` while the backend expected
`delete-images-<sha8>`. The two would never match.
Fix:
- Backend `/api/admin/images/bulk-delete` dry-run response now returns
`confirm_token` (the canonical `delete-images-<sha8>` string).
Integration test `test_bulk_delete_dry_run_returns_counts` pinned to
assert the new field.
- DestructiveConfirmModal gains an `expectedTokenOverride` prop. When
set, it bypasses the `${action}-${kind}-${runId}` formula and uses
the explicit string. This decouples the UI label (`kind`) from the
wire-format token (server-provided), so future endpoints can use a
kind-specific label without their kind name leaking into the token.
- BulkEditorPanel passes `:expected-token-override="bulkProjected?.confirm_token"`
— no client-side crypto, no kind-prefix mismatch.
- MinDimensionCard refactored to the same explicit pattern (was
slicing the 8-char suffix off the backend's token and passing it
through `runId`; now passes the full backend token via
`expected-token-override` directly). Cleaner; one source of truth.
**Banked memory**
`feedback_no_secure_context_apis.md` documents the full table of
Secure-Context-gated APIs, which ones FC currently uses, and how each
is handled. Indexed in MEMORY.md. Sites for the audit also listed in
the memory for future drift-checking.
No other Secure-Context-gated APIs found in `frontend/src/`. The same
shape won't recur unless someone adds a new dependency on one — at
which point the banked memory should fire.
bvandeusenandClaude Opus 5