36 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	9b82a95b7e	feat: Settings can re-download the Discord images the None naming broke (3999) ... The operator chose a clean re-download over relinking in place. The ~1,600 broken files can't be relinked reliably: their message ids are gone, and their sidecars collided. Settings → Maintenance → "Repair Discord downloads" previews, then applies: - Deletes every image whose path is `…/discord/None/<8 digits>_None_…`. Both the folder and the name are required, because that pair is only what the bug produced. It reuses cleanup_service.delete_images for the record and file deletes. - Sweeps the leftover collided sidecars from those folders and removes the emptied folders. - Only then clears gallery-dl's archive rows `discord%`, excluding `discordasset_%`. Upstream keys message attachments as `discord{message_id}_{num}`. Since the broken files lost their message ids, per-source forgetting is impossible. Every pre-fix Discord download is broken, and files fetched after the fix still exist on disk, so gallery-dl's `skip` won't re-fetch them. - Arms a fresh backfill on every Discord source. The apply defaults to preview at both the route and the task, runs on maintenance_long, and is never on a beat. The card uses the confirm-dialog pattern of AttachmentReclaimCard. Supporting refactors, with no behaviour change: - gallery_dl.archive_path() is the single definition of the archive location. - source_service.arm_backfill() is the mutation start_backfill already did, now shared with the sync repair. Tests (tests/test_discord_repair.py): - The archive clear leaves other platforms and Discord assets alone, and counting mutates nothing. - Case-twin artist folders are both found. - The folder sweep works. - An integration run shows only the broken image goes. A correctly named Discord file and a `None` folder under Patreon survive, and only Discord sources are re-armed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9	2026-09-13 20:52:32 -04:00
bvandeusenandClaude Opus 5	51e78a329b	feat: offer the creator you already track as the one you subscribe to (388 E4) ... **The verification the step asked for came back "not the schema".** `Source.artist_id` is a plain FK so many sources per artist already works; `POST /api/sources` already takes an `artist_id`; the add-source dialog already has an artist autocomplete that attaches to an EXISTING artist; and `SourceService.reassign` already moves a source between artists WITH post and image re-attribution. A sweep for one-source-per-artist assumptions found only `func.count()` calls — the opposite of assuming one. So no parallel association table was built for a relationship the schema already expresses (rule 28). What was missing is FC OFFERING the link, and that is all this adds. **Accepting adds a SOURCE. It never merges two artists.** That asymmetry sets the whole posture: adding a source is trivially undone, while a wrong merge silently mixes two creators' work and corrupts tagging, series and provenance downstream with nothing left to tell them apart by. A test asserts the artist count is unchanged by accepting. The weights encode the judgement rather than a code path doing it — name 0.65, declared 0.35, cut at 0.60 — so that: * an EXACT name match alone proposes (same slug on both sides is strong, and demanding corroboration would propose almost nothing); * a CONTAINMENT match alone does not ("art" sits inside "artgirl"), and short slugs are excluded from containment entirely because a 3-character slug is inside a great many longer ones; * the declaration ALONE never proposes, because a creator may link another creator's Patreon and a link is not a claim of identity. A guard test pins all three against WEIGHTS directly and says not to fix a failure by moving the numbers. Two corrections carried forward from earlier steps rather than rediscovered: * The declaration is NOT read from `ExternalLink`. `SUPPORTED_HOSTS` is file hosts only and `host_for()` returns None for patreon.com, so no row is ever written for one — the same trap that caught E5 for Discord invites. It reads the raw body, because these links live in an `href` and `html_to_plain` discards attributes. * `vanity` is not a column: C1 modelled the roster before any platform was characterised, which is exactly what `details` exists for. `vanity_or_none()` reads it from there and falls back to the URL's last segment, so a row written before the field was understood still resolves. Two fixes during the writing. `accept()` first created a bare `Source()`, skipping the platform/URL validation, duplicate check and #693 backfill-arming that a hand-added source gets — a second, quieter way to create a source is how two paths drift until one is subtly broken; it now goes through `SourceService.create`. And the candidate query used a bare `exists().where()`, which has no FROM to correlate against; now `select(...).exists()`. Chained onto the roster sweep rather than given its own beat entry: a suggestion can only be as good as the roster behind it, so any other cadence would just propose from staler data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-11 07:56:16 -04:00
bvandeusenandClaude Opus 5	751e7ddb9f	feat: the membership sweep, and the state that makes its failures readable (387 C3) ... A daily sweep that walks each platform's roster into `platform_membership`. Daily because memberships change on a BILLING cycle, not a download cadence. **`membership_sync` is the part that earns its keep.** Without it three very different situations are one indistinguishable state — the account subscribes to nothing, the sweep never ran, the sweep failed — and all three leave zero rows in `platform_membership`. "You are tracking 12 sources you no longer subscribe to" is correct in the first case and an invitation to cancel things the operator is actively paying for in the other two. So C4 gates its CONCLUSIONS on `last_success_at`, not merely its display, and `roster_is_fresh` is computed server-side so no caller can forget to. Two timestamps rather than one: `last_attempt_at` moves every run, `last_success_at` only on a clean walk. The gap between them is the signal — a sweep hammering a broken credential every day must not look healthy because it ran recently, and there is a test for exactly that. Rejected shortcuts, both tempting: `MAX(platform_membership.last_seen_at)` cannot tell "synced fine, found nothing" from "never synced"; `task_run` is worse, since its retention prunes ok rows after 24h and a sweep that last succeeded three days ago would leave no trace at all. **The fetch completes before anything is written.** That ordering is the safety property: a walk that dies mid-pagination writes nothing, so a failure can never leave a roster half this week's and half last week's. `touch_membership` never deletes, so a failure cannot empty the roster either — but "intact" should mean intact, not merely non-empty. Rule 89's four, each where it actually lives: recovery is "run it again" (upsert, no deletes); retention is C1's age-out-never-delete, because disappearing IS the signal; the wall-clock deadline is per-platform and distinct from the per-REQUEST timeout the client already has (rule 156 — a paginated roster answering every page slowly-but-within-timeout would never trip that one and would sit on a worker indefinitely); duration comes from the existing TaskRun signal plumbing. **A bug caught in review, not production:** the broad `except Exception` would have swallowed Celery's SoftTimeLimitExceeded — which is an ORDINARY Exception subclass, not a BaseException — letting the sweep run past the soft limit into the hard one, where it is SIGKILLed mid-transaction. A sweep that cannot be stopped is worse than one that fails. Now re-raised explicitly, with a test that also asserts SoftTimeLimitExceeded is still an Exception, so the re-raise cannot quietly become dead code. Rule 164 is why this ships with UI rather than backend-only: a roster that never synced must be VISIBLE as such. The card says "never synced" in words and states no count at all — rendering it as 0 is the precise conflation the whole step exists to prevent — while a real zero behind a real sync is reported as zero, because that one IS an answer. Pinned in both directions. Three independent gates decide whether a platform is swept — registered here, client exposes `iter_memberships`, credential exists — each silent, so adding SubscribeStar (D1) is one line and nothing else. A missing credential is not an error: recording a failure would light up the UI for a feature never enabled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 23:40:48 -04:00
bvandeusenandClaude Opus 5	d0b0458d27	feat: the announcement link, on the card and in a review queue (388 E5) ... Rule 27 — E5's other half. The matcher can propose; this is where the operator decides, and where an accepted link actually shows up. **The review queue** (Settings → Ingestion & filters). Each proposal shows the per-signal breakdown, not just the total: "why did it suggest this" is the question the operator actually has, and a lone percentage cannot answer it. So a row reads "72% · timing 95% · says so 60%", and the copy states outright that a pair always needs two reasons — which is the property that stops a busy posting day from producing false pairs. The empty state says so explicitly. Nothing proposed is the EXPECTED state most of the time, and an empty queue that looks like a failure invites turning the threshold down until it produces noise. **On the card**, both directions, and accepted links only: the teaser gets "The full set is in Discord", the drop gets "Announced on Patreon". A pending proposal is a question for the review queue, never a claim to render beside the artwork — that distinction is the whole confirm-only design, so it is asserted in the backend (only `linked` rows reach the payload) and again here. One detail worth the comment it carries: the link's target is `{ query: { post_id } }` with no name or path. In vue-router that means "the current route with these query params", so it works identically from Latest and from Browse — and, more usefully, the card never reaches for `useRoute()`, which it has no other reason to know about and which is not available when it is mounted in a test without a router. Backend CI on 235393c was green: all 13 E5 tests and migration 0094. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:53:17 -04:00
bvandeusenandClaude Opus 5	7071c87cd6	feat: a grouped post says so, and the operator can tune the grouping (388 E2) ... Rule 27 — E2's other half. The backend can author posts; this is what makes that visible and adjustable. **The honesty marker.** A chip on every synthetic post's card: "grouped by FabledCurator", titled with what it was built from ("Grouped from 4 Discord messages"). This chip is the only thing standing between "FC assembled this" and the card reading as something the artist authored, so it keys off nothing but the flag, and it states the member count rather than just disclosing that grouping happened — a claim you can check beats a claim you're asked to trust. A synthetic post has no title on purpose (inventing one is the one place this feature could put words in a creator's mouth), and the untitled fallback would otherwise have printed the internal key: "Post fc-drop:99887766". It now names the post for what it is. There's a test for that specifically. **The tuning card.** Ingestion & filters gets a Discord-drop-grouping tile: the switch, the distance cut and the drop window, each with the sentence that tells the operator which one to reach for. The window's copy says outright that it is the setting doing most of the work — without it, everything an artist ever drew of one character collapses into a single post. Both directions are pinned in postCard.spec.js, including the one that actually matters: an ordinary post is never marked. Also covered — a post dict composed before these fields existed degrades to unmarked rather than throwing on `synthesis.message_count`. Also fixes the ruff UP017 that failed the lint job on 73eeb7a (timezone.utc → datetime.UTC, the convention everywhere else in this repo). The integration suite on that SHA was green: all 12 grouping tests passed and migration 0092 applied cleanly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:17:56 -04:00
bvandeusenandClaude Opus 4.8	ec66ea5f83	refactor(ui): settings-card primitives + fix threshold clamp / card misgroup (#161) ... Tier-3 frontend DRY for the ML settings cards, plus the F-D2 clamp bug and the F-D3 card misgrouping. New primitives (components/common + composables): - <SettingToggleRow> — the accent-icon + .fc-section-h label + right-aligned switch row (HeadsCard x3, CropProposersCard). iconColor prop absorbs the on/off dim. - <SettingNumberField> — compact numeric field that CLAMPS to [min,max] on commit. This fixes F-D2: HeadsCard/CropProposersCard previously sent Number(raw) straight to the API, so an out-of-range threshold bounced off the 400 validator (only TranslationCard clamped). density prop for the grid cards. - useSettingSave(patchFn) — the busy + patch + toast + revert-on-failure flow each card hand-rolled (HeadsCard x6 handlers, CropProposersCard, MLBackfillCard, VideoEmbeddingCard). Returns ok/false for the optimistic-switch revert. Adopted in HeadsCard, CropProposersCard, MLBackfillCard (handler only — its plain labelled switch is a different affordance), VideoEmbeddingCard. F-D3: MLThresholdSliders.vue actually rendered a "Video embedding" (frame- sampling) card but sat under "Tagging → Suggestion thresholds". Renamed it VideoEmbeddingCard.vue and moved it to the "GPU agent & embeddings" section. Left deliberately (over-DRY guard): TranslationCard uses an inline error ALERT (not a toast), already clamps its confidence with a NaN fallback, and lives on the ImportStore — a genuinely different save pattern, so forcing it onto useSettingSave would change its UX. Behaviour-preserving refactor; CI has no Vue type-check so this needs a live UI pass (toggles persist + revert on failure, thresholds clamp on blur, video card now under Embeddings). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi	2026-07-13 22:28:47 -04:00
bvandeusenandClaude Opus 4.8	83c1745fd0	feat(ui): translation-forward post text + Settings card (#143 step 5) ... PostCard + modal ProvenancePanel show the English title/description by default when a translation exists, with a per-card "show original (<lang>)" toggle — translated bodies render as plain text, originals keep their sanitized HTML. New TranslationCard in Settings → Ingestion & filters: enable switch, Interpreter base URL (generic placeholder, no default host), target language, a reachability indicator + untranslated-posts count + "Translate now". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-07 12:38:41 -04:00
bvandeusenandClaude Opus 4.8	ab362bc79c	feat(ml): Settings → Tagging 'Crop proposers' card (#134 step 3) ... Exposes the detector config (per-proposer enable + weights + confidence, caps, dedupe IoU) in Settings → Tagging, backed by MLSettings via /api/ml/settings. ml_admin adds the detector fields to _EDITABLE + GET payload + validation (conf 0..1, caps >=1, IoU 0..1). New CropProposersCard.vue (mirrors HeadsCard) with working defaults pre-filled, per-field live-save (no restart — the agent picks changes up on its next lease), weights-format help, switch-revert on error. Closes milestone #134: all three proposers are on out-of-the-box and tunable in the UI. Test: detector defaults GET + patch round-trip + range validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-05 19:51:55 -04:00
bvandeusenandClaude Fable 5	5b34c9221c	feat(ia): wave 1 — Import tab dissolves, Maintenance regroups by system, one extension home ... Settings IA per the approved A3 design (the old layout was the two-app merge fossilized): - Import tab retired: ImportTriggerPanel + ImportTaskList deleted (manual /import scans stay API-level; imports arrive via downloads/extension, heal via the Layer-2 auto-refetch sweep, and show in Activity). ImportFiltersForm moves to Maintenance → 'Ingestion & filters' and loads its own settings; the import store shrinks to settings-only (no remaining consumers of the scan/task-list machinery). Overview's pending banner now points at Activity. - Maintenance regrouped: Ingestion & filters / GPU agent & embeddings (GpuAgent, Failed processing, CPU embedding backfill) / Tagging (sliders, Heads, Aliases) / Library health (MissingFiles, Thumbnails, DB, Archive re-extract demoted last) / Storage. - One extension home: BrowserExtensionCard moves from Settings → Overview to Subscriptions → Settings, above the API key bar it authenticates. - Single-color import filter WIRED: skip_single_color/threshold existed since FC-2 but nothing read them (the audit module's docstring said as much) — now enforced on both import paths via the audit's canonical predicate (tolerance 30, matching the Cleanup card default; animated images exempt like the transparency check). Default stays off; test added. - Dead weight: PlaceholderView (zero refs) and the permanently-disabled 'Export failed logs (CSV — v2)' menu stub deleted; stale docs fixed (celery queue docstring, threshold comment citing retired tasks, ml package docstring, HeadsCard 'replaces Camie' blurb). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-02 17:37:21 -04:00
bvandeusenandClaude Fable 5	eaea4308fc	chore: retire the tag-eval harness — it proved the heads system, job done (operator-approved) ... The head-vs-centroid eval (#1130) existed to prove the 'frozen embedding + trained head' spine; the operator accepted the tagging system and dropped the harness. Removed per rule 22: TagEvalCard + store, /api/tag_eval blueprint, tag_eval_run ml task, recover-stalled-tag-eval-runs sweep + beat entry, TagEvalRun model + table (migration 0073), and its tests. The eval's data loaders + metric helpers were NOT eval-specific — the nightly heads trainer runs on them — so they moved verbatim to services/ml/training_data.py (heads.py import updated; behavior unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-02 12:41:24 -04:00
bvandeusenandClaude Fable 5	a7abcc41ca	feat(triage): failed-processing triage — probe errored files, flag defects, recover (#125 C1-C3) ... An errored GPU job's stored reason is a suspicion; the file probe is the verdict. A 15-min beat sweep (triage_gpu_errors) runs verify_integrity's own probe (sha256 + decode) on each errored image ONCE and writes both verdicts: ImageRecord.integrity_status and the new GpuJob.triage_status ('defect' | 'file_ok', migration 0072). Every classification logs at WARNING so it surfaces in Logs/System Activity. - 'defect' rows are excluded from /retry_errors (re-running a known-bad file burns agent time re-minting the tombstone); response now reports defects_kept and the GpuAgentCard toast says so. - GET /api/gpu/errors: triage view — reason buckets (classify_reason), probe verdicts, per-job detail. POST /errors/triage runs the sweep now. - POST /api/gpu/errors/<id>/recover: reuses the Layer-2 refetch pattern — delete the defective copy + record (full cascade takes the tombstones too) and re-poll its subscription Source so a fresh copy re-imports and re-enters the pipeline; 'no_source' when nothing pollable resolves. - New 'Failed processing' card (GpuTriageCard) in Maintenance: verdict counts, reason summary, probe-now, defect list with thumbnails + per-image Recover. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM	2026-07-02 12:36:02 -04:00
bvandeusenandClaude Opus 4.8	485387ff0b	refactor(ml): retire the Camie tagger + allowlist bulk-apply (#1189) ... Heads + CCIP are the tag source and head auto-apply is the earned propagation. The Camie tagger ran only to feed the allowlist bulk-apply (its ImagePrediction rows had no other consumer), and the allowlist was a SECOND, un-earned auto-apply path firing in parallel with heads on every accept — exactly the un-earned spray the v2 pivot replaced. Retire both. Behavior change: accepting a suggestion now applies the tag to THAT image only (source='ml_accepted', a head-training positive) — it no longer allowlists + fans the tag across the library via Camie. Propagation is heads' earned auto-apply. (Loses instant cold-start propagation for booru-vocab tags; that was un-earned and bypassed the precision gate.) - tag_and_embed is now EMBED-ONLY (no Camie load/infer, no ImagePrediction writes); backfill enqueues it for images with no embedding. - Removed: services/ml/tagger.py, apply_allowlist_tags + helpers + daily beat + every enqueue caller (accept/alias/merge/per-image), api/allowlist.py + blueprint, ImagePrediction + TagAllowlist models/tables (migration 0067), AllowlistTable.vue + allowlist store, the accept coverage-projection payload. - AllowlistService gutted to accept/dismiss/undismiss/reject (the rejection store the rail still needs); accept returns nothing, API returns {accepted, tag_id}. - tag merge no longer repoints/triggers the allowlist; _keep_as_alias now keys on ML-applied image_tag sources (incl. head_auto) instead of the allowlist. - UI: MLBackfillCard relabelled to embedding-only; accept toast simplified; MaintenancePanel drops the allowlist tile. Left for a follow-up hygiene pass (now-inert, harmless): the dead settings columns (tagger_store_floor, tagger_model_version, suggestion_threshold_*, video_min_tag_frames), image_record.tagger_model_version, MLThresholdSliders trim, and the Camie model download in download_models.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa	2026-06-30 13:04:31 -04:00
bvandeusenandClaude Opus 4.8	3d77a38a25	refactor(ml): remove the dead per-tag centroid subsystem (#1189) ... The v2 pivot replaced per-tag SigLIP centroids with learned heads + CCIP. Centroids were still recomputed (on every tag merge + a daily beat) but NOTHING read them — suggestions come from heads+CCIP and apply_allowlist_tags applies via Camie predictions, not centroids. Pure dead wiring; remove it. Removed: CentroidService, recompute_centroid/recompute_centroids tasks, the daily beat, POST /api/ml/recompute-centroids, the recompute-on-merge trigger, the tag_reference_embedding table + model, the centroid_similarity_threshold + min_reference_images settings (migration 0066), the CentroidRecomputeCard + its store action + MaintenancePanel tile, and the centroid slider in MLThresholdSliders. _keep_as_alias drops its vestigial has-centroid branch (the allowlist branch already covers "could re-emit"); tag merge no longer clears a table that no longer exists. NOT touched (still live, parallel to heads): the Camie tagger, ImagePrediction, and the allowlist bulk-apply — accepting a suggestion still allowlists + applies it across the library. The tag-eval "centroid" baseline metric is unrelated (in-memory) and stays. (image_record.centroid_scores JSON column also remains — separate legacy field, its own micro-cleanup.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa	2026-06-30 11:48:09 -04:00
bvandeusenandClaude Opus 4.8	d91eef7a4b	feat(gpu): GPU agent admin card — token, queue, backfill (#114) ... The FC-side control surface the operator asked for: Settings → Tagging → "GPU agent". Generate/reveal/copy/rotate the agent bearer token (with the FC URL to point the agent at), see the live job-queue depth (pending/in-flight/done/ errored, polled), and a "Queue character embedding (CCIP)" button that triggers the library backfill. Plain-HTTP-safe copy (copyText resolves on success, throws on fail). Closes the "how do I get the token in the UI" gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa	2026-06-29 11:53:46 -04:00
bvandeusenandClaude Opus 4.8	06d5e83da4	feat(heads): admin card to train + inspect concept heads (#114 B) ... The UI for the heads subsystem: Settings → Tagging → "Concept heads". Shows head count, auto-apply-ready count, and last-trained; a Train/Retrain button (one run at a time, polls while running, surfaces a failed run's error); an empty state guiding the operator to tag first; and a per-concept table (name, category, +tags, AP, P, R, auto-apply ⚡) sorted strongest-first so weak/under- tagged concepts are obvious. Rehydrates status from GET /api/heads on mount so it survives navigation. Pulls head_min_positives from ML settings for copy. Slice C (swap the rail's suggestions to heads, remove Camie + centroid) is next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa	2026-06-28 10:44:35 -04:00
bvandeusenandClaude Opus 4.8	6cd7281af5	feat(settings): tag-eval admin card — trigger + persisted report (survives nav) ... Frontend for #1130. A maintenance tile in Settings → Tagging: - Editable concept list + "Run eval" → POST /api/tag-eval (one running at a time). - Rehydrates on mount via the persisted run (getRun by latest id) and polls while running — so the report SURVIVES navigation (operator-flagged); the task runs backend-side regardless and the card reconnects to its row. - Renders the saved report: per-concept head-vs-centroid metrics table (AP/F1/ precision/recall) with Δ AP, the learning curve (AP @ N positives), and thumbnail galleries (head-would-suggest / head-doubts-positive) for eyeballing. Backend: _examples now stores thumbnail_urls (not just ids) so the report is a self-contained artifact that renders without per-id lookups on reload. No new top-level surface — slots into the existing maintenance area. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 22:56:41 -04:00
bvandeusenandClaude Opus 4.8	311fe0ee9c	feat(settings): tidy Maintenance tab into compact tiles + center the views (pass 2) ... Goal (operator 2026-06-18): the overview of a Settings tab fits one unscrolled viewport; expanding a tile to read into it is the only reason to scroll. - Every Maintenance card converted to the collapsible MaintenanceTile (collapsed by default = icon + short title + one-line blurb). Task cards (ML backfill, centroids, thumbnails, archive re-extract, missing-file repair, DB maintenance) sit in a responsive grid; running tasks auto-expand. Tagging config (suggestion thresholds, allowlist, aliases) grouped in one Tagging section as collapsible tiles; Backup is its own collapsible tile. - Three labeled sections mirror the Cleanup tab: Backfills and reprocessing / Tagging / Storage. - Center the whole Settings surface: SettingsView is now a centered, width-capped (1140px) column so the tab strip and every panel sit in a tidy centered measure (was full-width). CleanupView drops its own left-aligned max-width to fill it. All card logic unchanged - only the chrome. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-18 00:10:58 -04:00
bvandeusenandClaude Opus 4.8	3b435dc0ba	feat(settings): tidy Cleanup tab into sectioned compact tiles (pass 1) ... The Cleanup + Maintenance sections had ~17 full-width stacked cards with long descriptions — a hunt to scan. Operator wants compact, sectioned, scannable tiles (2026-06-18: keep both tabs, group inside, compact tiles in a grid). New common/MaintenanceTile.vue: a compact expandable tile (icon + short title + one-line blurb collapsed; click the header to expand the full controls/preview/ result inline; keyboard-accessible button + focus ring; tints the icon, keeps a running task expanded). Cleanup tab (this pass) restructured into 3 sections — Import-filter audits (Min dimensions, Transparency, Single-color) / Duplicates & posts (Bare posts, Duplicate posts, Deduplicate videos, Gated-post previews) / Tags (Unused, Legacy, Reset content tagging, Standardize casing) — each a responsive grid of tiles. PostMaintenanceCard split into 2 tiles, TagMaintenanceCard into 4. Moved VideoDedupCard + GatedPurgeCard from the Maintenance tab here (both are destructive content cleanup). All card logic unchanged — only the chrome. Maintenance tab tiling is pass 2 (TODO noted in MaintenancePanel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 23:48:18 -04:00
bvandeusenandClaude Opus 4.8	540151290b	feat(cleanup): purge misgrabbed gated-post blurred previews (#874 follow-up) ... A one-shot Maintenance action to remove the blurred locked-preview images the ingester downloaded from tier-gated Patreon posts before #874. current_user_can_view was never persisted, so the cleanup re-walks each enabled Patreon source (read-only) to re-derive which posts are gated now and the blurred filehashes Patreon serves for them, then matches by CONTENT HASH against stored source_filehash. Because the hash is content-addressed, a real file downloaded when access existed has a different hash and can never match — regained-then-lost-access content is provably spared (operator's hard requirement). NULL source_filehash => unverifiable, kept + reported. On apply: delete matched ImageRecords + files (provenance cascades), clear seen/dead-letter ledger rows for those hashes so the real media re-ingests if access returns, and delete gated posts left bare. Shares one match predicate between preview and apply (rule 93). - cleanup_service: collect_gated_previews + purge_gated_previews - tasks.admin: purge_gated_previews_task (async re-walk bridge, timeboxed) - api.admin: POST /maintenance/purge-gated-previews - GatedPurgeCard.vue in Settings > Maintenance (preview -> confirm -> apply) - tests: collect predicate, hash-match delete/spare/unverifiable, ledger clear, bare-post removal, no-op Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 15:20:35 -04:00
bvandeusenandClaude Opus 4.8	41652db20f	feat(maintenance): retroactive video-dedup action — preview + apply (#871) ... Phase 2 of #871: clean up the duplicate videos already in the library (the #859 "same video from multiple sources" clutter). Import-time dedup (Phase 1) only prevents NEW dups; this is the operator-triggered cleanup of existing ones. cleanup_service.dedup_videos(dry_run): - backfill_video_durations: re-probe NULL-duration videos (pre-#871 rows) so the existing library participates; idempotent (only NULL rows), writes a negative sentinel for un-probeable files so they're neither re-probed forever nor matched. - find_video_dup_groups: cluster same-artist videos by duration (±tol) + aspect, anchored per cluster to bound the span (no chain drift); keeper = highest pixel area then bytes. Reuses the importer's _VIDEO_DUP_* tolerances. - apply: re-point each loser's post links to the keeper (so no post loses the video) THEN delete the redundant records + files via delete_images (cascade). dry_run shares the same discovery predicate and returns the projection only (rule 93). Tags on a loser are NOT merged (noted; videos rarely hand-curated). - dedup_videos_task (maintenance queue; summary → task_run.metadata). - POST /maintenance/dedup-videos {dry_run} + GET /maintenance/task-result/<id> so the card shows the dry-run projection before the destructive apply. - VideoDedupCard: Preview → shows groups/redundant/reclaimable, then Apply behind a confirm dialog. Mounted in the Maintenance panel. Tests: dedup collapses + re-links the loser's post to the keeper + removes the file; dry-run deletes nothing; distinct durations aren't grouped; task registered. (Migration 0052 for duration_seconds already shipped with Phase 1.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 08:31:50 -04:00
bvandeusenandClaude Opus 4.8	949c9abcc6	fix(external): path-safe unlink + per-link staging + orphan repair (#859) ... External downloads import IN PLACE, so the post-attach dedup-skip unlink could delete a file that IS an ImageRecord's backing file — orphaning the record and 404-ing on playback. Two sources of that: - Two links on the same post (same film from mega + gdrive) emitted the same filename into one external/<post_id>/ dir; the second overwrote the first. Stage per-LINK now (external/<post_id>/<link_id>/) so each file keeps its path. - The duplicate_hash/duplicate_phash branch unlinked `f` unconditionally. Make it path-safe: only unlink when `f` is NOT the existing record's canonical file. Plus an operator-triggered orphan-repair maintenance task (prune_missing_file_records_task) to clean up records already orphaned by the bug: scans ImageRecords, deletes those whose file is gone (cascade), with an NFS-stall guard that aborts without deleting if a large sample is mostly missing. Wired through POST /api/admin/maintenance/prune-missing-files and a MissingFileRepairCard in the Maintenance panel. Tests: refetch-same-link keeps the canonical file; orphan repair deletes only real orphans and aborts on the mostly-missing guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-15 01:48:38 -04:00
bvandeusenandClaude Opus 4.8	3610ba495f	feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3) ... Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images). Removes the old JSON column and everything that fed it: - ImageRecord.tagger_predictions column removed; migration 0046 DROPs it. tagger_model_version kept as the "tagged / current?" signal the backfill sweep reads (needs-tagging check switched to tagger_model_version IS NULL). - tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. - importer re-import reset drops the JSON line (image_prediction rows are already deleted on re-import). - Retired the one-time #768 backfill task + the #764 prune task, their admin endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard). - Tests seed/assert via image_prediction; stale column refs removed. Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run `VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's the source of truth now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 18:52:33 -04:00
bvandeusenandClaude Opus 4.8	65211a3f2f	fix(migration): make 0045 DDL-only; backfill image_prediction via batched task (#768) ... The inline INSERT…SELECT backfill in migration 0045 wrapped the table creation and a ~100 GB pass over image_record.tagger_predictions in one transaction: nothing committed until the end, it was unmonitorable, and an earlier MATERIALIZED-CTE form spilled the full 100 GB to temp on NFS. A deploy got stuck on it for ~2h with image_prediction never appearing. Split the concerns: - 0045 now creates ONLY the table + indexes (instant DDL → web boots). - New backend.app.tasks.admin.backfill_image_predictions_task copies the >= store-floor predictions from the JSON into image_prediction, batched by id window and committed per chunk: live progress, resumable (re-enqueues from the last committed id), idempotent (ON CONFLICT DO NOTHING). json_each stays in the DB executor streaming each window — no Python-side 100 GB load, no materialization. - POST /api/admin/maintenance/backfill-predictions + a Maintenance-tab card to trigger the one-time run after upgrading. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 09:18:25 -04:00
bvandeusenandClaude Opus 4.8	d55e52ae9b	feat(admin): prune_low_confidence_predictions backfill task + UI (#764) ... The one-time backfill that actually shrinks the DB: drops stored tagger_predictions entries below ml_settings.tagger_store_floor from every image_record row, and clamps any allowlist min_confidence below the floor up to it. Keep predicate (confidence >= floor) mirrors Tagger.infer's store gate so backfilled rows match new imports. Keyset by id ASC, idempotent, self-resumes on the soft time limit; runs on the maintenance_long lane. pg_dump copies live data only, so this alone fixes the #739 backup timeout — the reclaim (VACUUM FULL / pg_repack on image_record) is a separate, optional disk-return step, brief because post-prune the live data is tiny. - admin.prune_low_confidence_predictions_task + POST /api/admin/maintenance/prune-predictions - PrunePredictionsCard in the Maintenance panel (shows the current floor) - tests: registration + prune-keeps->=floor/drops-<floor + allowlist clamp Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-10 13:57:39 -04:00
bvandeusenandClaude Opus 4.8	a497104661	feat(maintenance): re-extract archive attachments + link to post — #713 part 2 ... Existing PostAttachments that are actually archives (filed opaquely before the magic-byte gate) need extracting retroactively. cleanup_service. reextract_archive_attachments scans PostAttachments, magic-detects the archives, and for each reconstructs the post's sidecar from the DB + re-runs attach_in_place in a temp dir — so the members extract and re-link to the SAME post via find_or_create_post (source_id + external_post_id). Idempotent (members dedupe by sha256). Enqueues thumbnail+ML for new members. Wired as a maintenance-queue Celery task (tasks/admin) + POST /api/admin/maintenance/reextract-archives (202) + a "Re-extract archive attachments" card in Settings → Maintenance. Test: a zip stored under a mangled extension-less name extracts + links its member to the post via ImageProvenance, and a second run is a no-op (idempotent). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-06 14:43:26 -04:00
bvandeusenandClaude Opus 4.8	ef3ee5aceb	feat(maintenance): DB maintenance UI card + fix ruff I001 ... - Settings → Maintenance gains a "Database maintenance" card: a "Run VACUUM ANALYZE now" button (enqueues the maintenance task) plus a per-table bloat readout (live/dead/dead%/last vacuum) from /api/admin/maintenance/db-stats. - dbMaintenance store (loadStats / runVacuum) + test. - Fix ruff I001: combine the two _sync_engine imports onto one line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:52:36 -04:00
bvandeusenandClaude Opus 4.7	8649a13118	refactor(I5): remove one-and-done GS/IR migration tooling ... The GS/IR migration cutover is complete, so the runbook tooling is dead weight. Removed: - services/migrators/ (gs_ingest, ir_ingest, tag_apply, ml_queue, verify, cleanup), tasks/migration.py, api/migrate.py (+ blueprint registration) - MigrationRun model; alembic 0027 drops the migration_run table - frontend LegacyMigrationCard + migration store (+ MaintenancePanel ref) - celery include + task route + celery_signals queue mapping for migration.* - the 1 GB MAX_CONTENT_LENGTH / MAX_FORM_MEMORY override (added solely for the ir_ingest upload) - migration-surface tests (test_api_migrate, test_migration_verify, test_ir_ingest, test_gs_ingest, test_tag_apply) Kept: the alembic schema-migration tests (test_migration_00XX — unrelated) and cleanup_service.py (the permanent artist-cascade/unlink home). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-29 14:38:59 -04:00
bvandeusenandClaude Opus 4.7	35fe420701	feat(maintenance): live queue-depth bar under the backfill buttons ... Each maintenance button (ML backfill, centroid recompute → ml queue; thumbnail backfill → thumbnail queue) now shows a status bar with the live pending count for its Celery queue, so the operator can see work is already queued/running before re-triggering and piling on. MaintenancePanel polls /api/system/activity/queues every 4s; QueueStatusBar reads the depth and turns warning-colored when the queue is busy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-28 20:03:32 -04:00
bvandeusenandClaude Opus 4.7	06913eba8e	feat(thumb-backfill): MaintenancePanel — wire ThumbnailBackfillCard into grid, broaden intro to cover non-ML backfills ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-26 10:37:09 -04:00
bvandeusen	2505b197ae	feat(fc-cleanup): Pinia store + 3 cards + CleanupView + SettingsView tab + TagMaintenanceCard moved from Maintenance + ruff lint fixes — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-26 08:16:46 -04:00
bvandeusen	3531f373ee	feat(settings): move BrowserExtensionCard from Maintenance to Overview tab — Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 20:33:44 -04:00
bvandeusenandClaude Opus 4.7	de1a4b64b7	fc3k(ui): TagMaintenanceCard — preview-then-commit prune-unused under Settings → Maintenance ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 00:52:30 -04:00
bvandeusenandClaude Opus 4.7	83bd3b4b2d	fc3h(ui): slot BackupCard into Maintenance panel above migration card ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-24 23:06:22 -04:00
bvandeusenandClaude Opus 4.7	d85c108cec	fc3g(ui): BrowserExtensionCard on Settings → Maintenance — one-click Firefox install + key copy/rotate ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-23 23:40:56 -04:00
bvandeusenandClaude Opus 4.7	a9b354780b	fc5: LegacyMigrationCard — file uploads + step buttons + active-run + history; slotted into MaintenancePanel ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-22 09:04:24 -04:00
bvandeusenandClaude Opus 4.7	cd7721fb03	feat(fc2b): Settings Maintenance tab ... MaintenancePanel hosts: backfill + centroid recompute trigger cards, the five suggestion-threshold sliders (autosave on slider release), the allowlist table (inline editable min_confidence, delete), and the alias table (mapping display, delete). Wired as a third Settings tab, ML settings loaded lazily when the tab opens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-15 07:57:42 -04:00