100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusenandClaude Opus 4.8	3b2f7a41c3	feat(patreon): ingester rate-limit resilience — #703 step 1 ... A single 429 mid-walk used to fail the run → RATE_LIMITED → platform-wide cooldown → every Patreon source dark ("testing dead"). The native path also ignored the operator's existing politeness setting. Fixed both: - Pacing (avoid 429s): honor download_rate_limit_seconds (gallery-dl's `rate_limit`, read off self.gdl) as a pre-download sleep on real media downloads only (skips don't pace); pace /api/posts page fetches with the per-source sleep_request override, defaulting to max(0.5, rate_limit/4) — the same API-pacing default gallery-dl used for `sleep-request`. - 429 backoff (ride out transient limits): PatreonClient._fetch retries a 429 with backoff (honor Retry-After, else exponential 2·2^(n-1), capped 30s, ≤3 tries); only a PERSISTENT 429 propagates as terminal RATE_LIMITED. Light 2-retry on a media-GET 429 too. Threaded via PatreonIngester(rate_limit=, request_sleep=) → PatreonClient/PatreonDownloader; download_service sources them. Injected test client/downloader are unaffected (carry their own pacing). Tests mock time.sleep (no real sleeping): retry-then-success, persistent 429 raises after N, Retry-After honored, request_sleep paces, media pacing per real download, skips don't pace, media 429 retried. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 23:13:46 -04:00
bvandeusenandClaude Opus 4.8	218bfebb92	feat(downloads): native Patreon verify + uniform backend dispatch (plan #697) ... The credential Verify button still ran gallery-dl --simulate for Patreon after the cutover — testing the wrong path (and prone to the vanity "Failed to extract campaign ID" the native resolver fixes). Wire it to the native ingester, behind a DRY dispatch so callers never branch on platform. - services/download_backends.py (new): the ONE place that knows which platforms are native vs gallery-dl. `uses_native_ingester(platform)` is the shared predicate; `verify_source_credential(...)` is the uniform probe (same (ok|None, message) contract for both backends). As a platform migrates, it moves into NATIVE_INGESTER_PLATFORMS here and BOTH download routing and verify switch together. - PatreonClient.verify_auth(campaign_id): one authenticated /api/posts fetch → True (valid) / False (401/403/HTML-login) / None (drift or network — inconclusive, not a credential verdict). - patreon_ingester.verify_patreon_credential(): resolve campaign id, then verify_auth — the verify counterpart to the download path. - patreon_resolver.resolve_campaign_id_for_source(): extracted the override / id:-URL / vanity resolution into ONE helper now shared by the download ingester and verify (download_service no longer carries its own copy + regex; −`import re`). - download_service: routes on uses_native_ingester() instead of inline `== "patreon"` (3 sites); uses the shared resolver. - api/credentials: calls verify_source_credential — no platform branch. Tests: verify_auth mapping, resolve_campaign_id_for_source (override/id:/ vanity/none), the dispatch predicate, verify_patreon_credential glue, credentials endpoint proves Patreon uses the native path (gallery-dl verify asserted not-called); repointed the gallery-dl verify test to subscribestar. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 22:49:43 -04:00
bvandeusenandClaude Opus 4.8	ec43e823e1	feat(patreon): recovery UI + gallery-dl cutover — build step 5 (plan #697) ... Final step of the native Patreon ingester: a first-class Recovery action, and removal of the now-dead gallery-dl Patreon path. Recovery (rules #23/#24/#27 — full product, with UI): - source_service.start_recovery arms the #693 backfill state machine PLUS `_backfill_bypass_seen`, flipping download mode to recovery (bypass the seen-ledger to re-fetch dropped-and-deleted near-dups and re-evaluate under the current pHash threshold). Stop via the shared stop_backfill. - SourceRecord exposes backfill_bypass_seen; POST /sources/{id}/backfill gains action="recover". - Frontend: Recovery button (Patreon-only, mdi-backup-restore) on SourceRow + SourceCard; the running badge labels "Recovering (N)" vs "Backfilling (N)"; the Stop tooltip says "Stop recovery". sources.js recoverSource + SubscriptionsTab onRecover. Cutover (rule #22 — no legacy): - gallery_dl: removed PLATFORM_DEFAULTS["patreon"], the patreon files/cursor branch in _build_config_for_source, and the patreon/Mux yt-dlp Referer/Origin block (was patreon-specific and wrongly tagged the other platforms' yt-dlp fetches; native ingester owns it now). - download_service: removed the dead campaign-id-retry helpers (_looks_like_campaign_id_failure / _CAMPAIGN_ID_FAILURE_PATTERN) and _effective_url. Vanity→campaign resolution + resume_cursor + the cursor/PARTIAL lifecycle stay — they serve the native ingester. Tests: removed the two obsolete patreon-gallery-dl config tests (yt-dlp Referer, resume-cursor); repointed the generic skip-value config tests to subscribestar; added start_recovery + recover-endpoint coverage (backfill_bypass_seen). gallery-dl stays for the other 5 platforms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 22:18:03 -04:00
bvandeusenandClaude Opus 4.8	682beafbc5	feat(patreon): drift detection + error categorization — build step 4 (plan #697) ... Typed, loud failure mapping for the native Patreon ingester so a changed API shape or expired auth never silently zero-downloads as "success". - New ErrorType.API_DRIFT (free varchar error_type col → no migration): distinct from auth so the operator knows the fix is updating the ingester, not rotating cookies. - patreon_client: PatreonAPIError carries status_code; new PatreonAuthError for 401/403 + HTML-login/non-JSON bodies (reclassified from drift — expired-session is auth, actionable as "rotate cookies"). - patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR, PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR, transport→NETWORK_ERROR. (429 thus drives the platform cooldown.) - FailingSourcesCard: api_drift chip (red) + hint. Contract test (new test_patreon_contract.py): the recorded /api/posts fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the request params must still carry every field the parser depends on (file_name, image_urls/download_url, images/attachments_media/media includes, content/post_file/image post fields) — a trim of either trips a red build. Plus client HTTP-status classification tests and ingester error-type mapping tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 21:56:58 -04:00
bvandeusenandClaude Opus 4.8	96c30eba13	feat(patreon): phase-2 ingester integration — build step 3 (plan #697) ... Branch download_service phase 2 by platform: Patreon now routes to the native PatreonIngester (zero per-file HEADs, native cursor/resume, loud drift detection) instead of gallery-dl; the other 5 platforms are unchanged. The ingester returns a DownloadResult-shaped object so phase 1 (DB setup) and phase 3 (import → pHash → thumbs → ML) are untouched. Three modes wired from config_overrides state: - tick: skip seen (tier-1 ledger + tier-2 disk), early-out after N contiguous already-have-it items. - backfill: full-history time-boxed chunk, cursor checkpoint via gallery-dl-style "Cursor: <token>" lines in stdout (reuses the #693 lifecycle + parse_last_cursor verbatim). - recovery: backfill that BYPASSES the tier-1 seen-ledger so dropped-and-deleted near-dups get re-fetched and re-evaluated under the current pHash threshold. Rides the #693 state machine via a _backfill_bypass_seen flag, cleared on completion / stop. The seen-ledger uses short-lived sync sessions (injected sessionmaker), never held across the walk (avoids the connection-reaping trap). Campaign id resolves from override, an id: URL, or a vanity lookup; unresolvable = loud NOT_FOUND, never a silent empty success. Tests: new test_patreon_ingester.py (modes, ledger skip/idempotency, budget→PARTIAL, recovery bypass, tier-2 disk, drift). The patreon-oriented download_service tests now drive the ingester branch via a stub; the gallery-dl campaign-retry test is replaced by resolution/caching coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 21:38:42 -04:00
bvandeusenandClaude Opus 4.8	2ec7d86a3b	feat(patreon): native media downloader — ingester build step 2b (plan #697) ... PatreonDownloader.download_post: writes resolved MediaItems to gallery-dl's exact on-disk layout (<slug>/patreon/<date>_<id>_<title40>/<NN>_<file>) + a sidecar the importer's find_sidecar/parse_sidecar consume unchanged. Two-tier skip (injected seen predicate, then disk). Streamed GET (.part→rename) + file_validator quarantine; Mux/m3u8 video shells out to yt-dlp with Patreon Referer/Origin. Pure (no DB) — ledger + orchestration land in step 3. Unit tests stub the session + yt-dlp seams (no network/subprocess). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 19:39:12 -04:00
bvandeusenandClaude Opus 4.8	6222928746	feat(patreon): seen-ledger table + model — ingester build step 2a (plan #697) ... patreon_seen_media(source_id, filehash, post_id, seen_at), UNIQUE(source_id, filehash) — our own queryable replacement for gallery-dl's archive.sqlite3. Routine walks skip seen media; recovery mode bypasses the ledger. filehash is a 32-hex CDN MD5 or a video:<post>:<media> sentinel (String(128)). alembic 0037 (← 0036). Integration test covers dedup + savepoint recovery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 19:26:57 -04:00
bvandeusenandClaude Opus 4.8	1bdaa04aa2	test(patreon): fix self-contradictory attachment/postfile dedup assertion ... The fixture gives the attachment and post_file the same filehash, so they correctly collapse to one item; the test asserted both survival and collapse. Rewrite to verify the cross-kind dedup (postfile kind covered by video test). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 19:17:56 -04:00
bvandeusenandClaude Opus 4.8	1c2dc7659a	feat(patreon): native JSON-API client — ingester build step 1 (plan #697) ... PatreonClient: cookie-auth requests session, /api/posts cursor pagination, JSON:API included flattening, per-post media extraction (images/image_large/ attachments/postfile/content) with filehash dedup, loud drift detection. Zero per-file HEADs — every media URL+file_name comes from the API. Not yet wired into download_service (later step). Pure-parsing unit tests + fixture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 19:16:20 -04:00
bvandeusenandClaude Opus 4.8	618dafde85	feat(subscriptions): smarter-backfill UI — Start/Stop + state badge ... Plan #693 (frontend). Backend landed in 96fffaf. - sources store: setBackfill(runs) → startBackfill/stopBackfill ({action}). - SubscriptionsTab: the deep-scan window.prompt for N runs becomes a Start/Stop toggle keyed on source.backfill_state. - SourceRow + SourceCard: the 'backfill (N×)' chip becomes a state badge — Backfilling (chunk N) / Backfilled / Stalled — and the scan button toggles between Backfill (mdi-magnify-scan) and Stop (mdi-stop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 15:06:30 -04:00
bvandeusenandClaude Opus 4.8	96fffaff64	feat(download): smarter backfill — time-boxed chunks, run-until-done (backend) ... Plan #693. Large-catalog backfill (Anduo) no longer sprints to the timeout wall and dies as an error each run. Builds on the cursor checkpoint (#689). - Time-boxed chunks: BACKFILL_TIMEOUT_SECONDS(1170)→BACKFILL_CHUNK_SECONDS(600), far under the 1350 soft limit. Hitting it = normal chunk boundary (the TimeoutExpired path already captures partial output + the cursor), not a near-wall death. - Run-until-done state machine driven by config_overrides[_backfill_state] (running/complete/stalled). A running backfill auto-continues in chunks across ticks until gallery-dl exits cleanly (rc=0 = reached the bottom → 'complete'); a safety-cap (BACKFILL_MAX_CHUNKS=200) + the #689 stall-guard pause a pathological walk as 'stalled'. Replaces the N-runs counter (backfill_runs_remaining repurposed as the cap countdown). - Progress, not error: a chunk that timed out but advanced (cursor moved and/or files written) is reclassified TIMEOUT→PARTIAL (status 'ok'). - Retry storm tamed: gallery-dl retries 3→2, downloader timeout 120→60s, so one stuck CDN file fails in ~1-2 min not ~10 (Anduo #40838). - API: POST /sources/{id}/backfill now takes {action: start|stop}; service start_backfill/stop_backfill; new enabled sources auto-arm run-until-done; source dict exposes backfill_state + backfill_chunks. Frontend (Start/Stop control + state badge) lands in the next push. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 15:02:46 -04:00
bvandeusenandClaude Opus 4.8	add1c1ad14	fix(modal): mobile — no tag autofocus + sticky image over scrolling panel ... Operator-flagged 2026-06-05 (mobile): - TagAutocomplete no longer autofocuses on the ≤900px stacked layout — focusing popped the soft keyboard, shrinking the viewport and shoving the pinned image + nav/close controls out of view. matchMedia gate (safe on plain HTTP); desktop autofocus unchanged. - ImageViewer stacked layout: the body now scrolls and the media pane is sticky (height:55vh, top:0), so the image + prev/next/close + integrity badge stay pinned while the metadata panel scrolls beneath. Replaces the old fixed image + 40vh internally-scrolled panel that could push the controls off. Prev/next re-centered over the image band; opaque obsidian bg so the scrolling panel doesn't bleed through. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-05 08:35:22 -04:00
bvandeusenandClaude Opus 4.8	593f65c9cc	feat(download): cursor-paged Patreon backfill for large catalogs ... Large Patreon creators (Anduo: weekly 50-120-image Reports back months = thousands of files) couldn't backfill: each run re-walked newest→oldest from the top, and gallery-dl's polite ~0.75s/request HEAD walk alone exceeded the 1170s subprocess budget, so the run died during enumeration with 0 files written and NO forward progress — re-stranding every time (event #40411). Checkpoint gallery-dl's pagination cursor so each backfill window advances the frontier: - gallery_dl.py: SourceConfig.resume_cursor; _build_config_for_source sets extractor.patreon.cursor=<resume> (PLATFORM_DEFAULTS leave log-only True for a fresh run); parse_last_cursor() pulls the last emitted 'Cursor: <token>' from stdout+stderr — survives a timed-out run since the TimeoutExpired path returns partial output. - download_service.py: phase2 stays in BACKFILL mode while a cursor is pending (even after the run budget drains) and threads resume_cursor; _apply_backfill_lifecycle() checkpoints the advancing cursor each non-completing run, completes on a clean rc=0 finish (walk reached bottom), and a stuck-guard clears the cursor after 2 non-advancing runs so a wedged walk can't re-strand forever. patreon-only (sole platform with a resumable cursor); other platforms keep the simple counter semantics. Cursor state lives in config_overrides JSON (patreon_campaign_id precedent) — no migration. Time-budget ladder (1170/1350/1500) unchanged. Plan #689. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 23:44:31 -04:00
bvandeusenandClaude Opus 4.8	86efbf7f2c	fix(modal): kebab menus open via explicit v-model, not activator click ... Operator-confirmed on a fresh build: both the tag-chip and suggestion kebabs still never opened. The prior 8326e54 'fix' only wrapped them in a <span @click.stop> — inert for SuggestionItem (no parent capture) — and never addressed why the `#activator`/`v-bind="props"` click failed to toggle the menu inside the teleported ImageViewer modal. The dialogs in that same modal open via v-model and work, so drive the menus the same way: - The activator (v-btn / v-icon) toggles a reactive flag with @click.stop (which also shields the chip's close button / any parent). - The v-menu binds that flag (v-model / :model-value) and uses activator="parent" with :open-on-click="false" purely for positioning, so opening no longer depends on Vuetify's activator-click path. - TagPanel tracks a single openTagId (one chip menu open at a time). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 23:01:55 -04:00
bvandeusenandClaude Opus 4.8	3a0cca5aca	fix(tags): allow creating a character with no fandom ... Not all characters belong to a fandom (original characters, unsorted). The create flow forced every new character through FandomPicker, whose only outcomes were 'Use this fandom' (disabled until one is picked) or Cancel (which aborts the whole creation) — there was no way to confirm a character with no fandom. - FandomPicker: add a 'No fandom' action that emits confirm(null). - TagAutocomplete.onFandomChosen: pass fandom_id: null when null is emitted. Backend already supported this end to end (Tag.fandom_id nullable, the CHECK only forbids fandom_id on non-character kinds, tag_service find_or_create defaults fandom_id=None, API reads body.get). A fandom can still be assigned later from the chip kebab's 'Set fandom…'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 22:50:19 -04:00
bvandeusenandClaude Opus 4.8	a5b3702863	feat(settings): surface the near-duplicate (pHash) control + reorder import tab ... The phash_threshold knob (controls whether edits/variants of an image are dropped as near-duplicates on import) was buried at the bottom of the import filters form and labelled opaquely, so it read as 'missing'. Hoist it to the TOP of the form as a 'Near-duplicate sensitivity' section: a labelled slider (Exact / Strict / Default / Loose stops, 0-16) for the gist + the precise number field, both bound to phash_threshold, with copy that says plainly to lower it if variants are being dropped. Also swap the import-tab order to filters → trigger → recent-tasks (filters on top per operator); the task list stays directly under the trigger for hit/miss feedback adjacency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 17:42:32 -04:00
bvandeusenandClaude Opus 4.8	509a7958cf	feat(posts): images open the modal; only text expands in place ... Post cards no longer expand the whole card on click (the old two-click path to the images). The card is compact-only now: - Hero / rail thumbs / the +N tile are buttons that open the post-scoped image modal (modal.open(id, { postImageIds })) so you view big + arrow through ALL the post's images. The feed caps thumbnails at 6, so for posts with more we lazily getPostFull to get the complete id list; +N opens at the first hidden image. - The description is the ONLY in-place expansion: a Show more / Show less toggle shown only when the text is actually truncated (server description_truncated flag OR a measured CSS-clamp overflow, ResizeObserver-guarded). Expanding loads description_full when server-truncated and renders it unclamped. - Attachments: download chips now render inline in the compact card (the feed already carries download_url), since the expanded view is gone. Removes PostImageGrid.vue (the mosaic, now unused). Tests cover show-more visibility + image-click opening the scoped modal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 17:29:19 -04:00
bvandeusenandClaude Opus 4.8	5a6a95682d	fix(cleanup): library scans survive navigation, reconnect on return ... The transparency / single-color audit cards held the run + poll timer in local component state, so navigating away destroyed both and onMounted never reconnected — the Celery scan kept running and writing LibraryAuditRun, but the UI forgot it. Now each card, on mount, fetches its rule's latest run (GET /api/cleanup/audit?rule=<rule>&limit=1) and rehydrates: shows progress + resumes polling if still running, or shows the completed result (ready/applied/ error) so the operator can act on it after returning. Adds the ?rule= filter to the audit-history endpoint + cleanup store latestAuditForRule(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 16:55:20 -04:00
bvandeusenandClaude Opus 4.8	91b0145bc8	feat(tags): 'Reset content tagging' admin action ... Wipe every general + character tag so the operator can re-tag from scratch via the Camie auto-suggest, while PRESERVING fandoms, series (+ series_page order), and each image's stored tagger_predictions (so suggestions repopulate immediately). One set-based DELETE FROM tag WHERE kind IN ('general','character') — the five tag-referencing tables all cascade, so applications + aliases + allowlist + rejections + centroids clear automatically; series tags aren't deleted so series survive; Tag.fandom_id is SET NULL so fandoms are untouched. Reuses the established dry-run-preview -> confirm pattern: cleanup_service. reset_content_tagging() + POST /api/admin/tags/reset-content + TagMaintenanceCard section with a backup-first warning and a red confirm showing exact counts (tags by kind + image applications). Irreversible except via DB backup restore; the wipe only fires when the operator confirms. Tests: service dry-run counts + live delete preserves fandom/series/series_page while content tags + their image_tag cascade away; API dry-run wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 16:47:36 -04:00
bvandeusenandClaude Opus 4.8	26e47a86cb	fix(gallery): render similar-mode results (flat list when no date groups) ... The 'See all similar' takeover fetched /api/gallery/similar fine (200, ~100 results) but the grid showed nothing: GalleryGrid renders ONLY by iterating store.dateGroups, and similar-mode returns date_groups=[] (results are ranked by cosine distance, not chronological). Zero groups → zero tiles despite store.images being full. Add a flat fallback: when there are no date groups but images exist, render them as one ungrouped list in ranked order (no date headers). The modal Related strip was unaffected (it renders its images directly). Test locks both the flat and grouped paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 16:24:52 -04:00
bvandeusenandClaude Opus 4.8	928e3037f0	fix(ui): purpose-built mobile layout for subscriptions hub ... Vuetify's auto card-stack was too verbose (one subscription filled the whole phone screen) and the expanded sources still needed lateral scroll. Replace it below 600px (useDisplay) with a custom compact-card list: each subscription is a 2-line card (name + health + expand chevron, then platform chips + sources count + last activity) so several fit per screen. Expanding shows the action row + each source as a STACKED SourceCard (new) — platform/url/enabled/last/ next/errors/actions laid out vertically, no horizontal scroll. The mobile cards drive the same selected/expanded key arrays as the desktop data-table, so selection and bulk actions are unchanged. Desktop keeps the v-data-table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 13:40:25 -04:00
bvandeusenandClaude Opus 4.8	b08b12eb8f	fix(ui): subscriptions table → card layout on mobile ... The subscriptions v-data-table (select + expand + 6 cols + a nested 8-col sources sub-table) horizontally-scrolled on phones. Set mobile-breakpoint=600 so Vuetify stacks each subscription row into a label:value card below 600px; the custom item slots (platform chips, health dot, action buttons) render as card rows. The expanded sources detail reclaims its desktop indent on mobile and keeps its own horizontal scroll for the wide source columns. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 12:53:40 -04:00
bvandeusenandClaude Opus 4.8	4fd6d4cc29	fix(ui): mobile pass 2 — Posts & Subscriptions filter bars ... PostsFilterBar didn't wrap and its artist/platform fields had inline min-widths (240/180px) a media query can't override → horizontal overflow on phones. Moved widths to classes, added flex-wrap, and <600px each field takes a full-width row. SubscriptionsTab's status/search inline max-widths likewise moved to classes; <600px they go full-width and the v-spacer is dropped so the search isn't shoved around. Verified as already-fine (sweep false positives, no change): PostCard (default body is a stacked column; only goes row at container >=800px), SeriesReaderView (already has a <=768px block: nav drawer 150px, quick-nav stacks). The subscriptions v-data-table scrolls horizontally within its own wrapper, so it doesn't widen the page — a true mobile card layout is a larger follow-up if wanted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 09:26:56 -04:00
bvandeusenandClaude Opus 4.8	304e8aa878	fix(ui): mobile responsiveness — nav hamburger + primary-path fixes ... The top nav packed brand + health + pipeline chip + ~7 inline links + an action slot into one flex row, colliding/overflowing on phones (operator: 'almost unusable'). Below 768px the links now fold into a hamburger v-menu; below 480px the brand text hides (glyph still brands). Plus the primary browsing path: - BulkEditorPanel: fixed 320px -> min(320px, 90vw) so it can't swallow the screen. - GalleryFilterBar: <600px gives search its own full-width row (its 200px min-width was jamming the wrapping bar); sort grows. - GalleryFacetPanel: <480px wraps groups + lets the side-by-side date inputs grow full-width. - ArtistsView grid: minmax(min(440px,100%),1fr) so a card never overflows (single column on phones). - GalleryView: hide the year/month timeline strip <600px. ImageViewer already stacks its side panel below the image <900px (left as-is). Secondary surfaces (Posts/Subscriptions filter bars, SubscriptionsTab table, SeriesReader, PostCard) still need a mobile pass — follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 09:06:33 -04:00
bvandeusenandClaude Opus 4.8	0497394710	feat(ui): double showcase cadence + filter bar matches TopNav frost ... - Showcase reveal cadence 80ms -> 160ms (slower, more deliberate one-at-a-time cascade per operator). Bump showcase.spec timer advances to cover 60x160ms. - Gallery filter bar now uses the EXACT gradiated-obsidian frost + blur as TopNav (was a flat rgba(...,0.55) block), so the two read as one continuous piece of chrome with images visibly scrolling under both; the nav's transparent bottom edge against the bar's opaque top leaves a faint seam that separates them at the very top of scroll. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 09:02:21 -04:00
bvandeusenandClaude Opus 4.8	21a73cd1dc	feat(gallery): visual 'more like this' UI (Phase 3 frontend) ... Modal 'Related' strip (RelatedStrip.vue) — top-12 similar thumbs, fetched on its own DEFERRED, single-flighted path (200ms after the modal is up) so it never blocks or slows the modal; collapses silently on empty/slow/error and is hidden when the image has no embedding (has_embedding flag). 'See all similar' closes the modal and navigates the gallery to ?similar_to=<id>. Gallery store: similar_to filter field + loadSimilar() (ranked, hasMore=false, no timeline); applyFilterFromQuery routes similar-mode to /similar with the scope filters composed; cloneFilter/filterToQuery carry similar_to. Filter bar: clearable 'Similar to #id' chip, sort hidden in similar-mode; timeline sidebar hidden too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 08:52:42 -04:00
bvandeusenandClaude Opus 4.8	79cd1234e2	feat(gallery): visual 'more like this' search (Phase 3 backend) ... GalleryService.similar() ranks images by pgvector cosine distance to a source image's precomputed SigLIP embedding — no query-time ML inference. Composes with the Phase-1/2 scope filters (AND) but replaces the date sort (always nearest-first, bounded top-N, no cursor). Returns None for a missing source (→404), [] for a source with no embedding (video / pending ML); excludes self and NULL-embedding rows. New GET /api/gallery/similar?similar_to=<id>&limit=N. Image-detail payload gains has_embedding so the UI can hide the surface. Alembic 0036 adds an HNSW vector_cosine_ops index on siglip_embedding (1152<2000 dims) so the search is sub-50ms ANN instead of a full scan; one-time ~30-60s build over existing embeddings on deploy. Shared _gallery_images/_image_json helpers de-dup the scroll/similar builders. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 08:47:43 -04:00
bvandeusenandClaude Opus 4.8	3f6ea601f8	perf(ci): collapse the 3 integration shards into one job ... With fsync-off the whole integration suite runs in ~45s (was ~13min across shards), so the 3-way split only triplicated the ~2min fixed overhead (container + install + migrate) and consumed 3 of 6 runner slots for no wall-clock gain. Merge intapi/intimp/intcore into one `integration` job: spin up once, install once, migrate once, run `pytest -m integration` over the whole suite. Frees 2 runner slots (6 jobs -> 4) and drops ~140 lines of near-duplicate YAML. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 08:17:50 -04:00
bvandeusenandClaude Opus 4.8	6a25db4b8b	perf(ci): relax Postgres durability in integration shards (fsync off) ... Option 1 (pooling the teardown connection) left teardowns at ~1.5-2s/test, so the cost is the per-test TRUNCATE's commit forcing an fsync, not the connect handshake. Each shard now ALTER SYSTEM SETs fsync/synchronous_commit/ full_page_writes off + pg_reload_conf() right after deps install, before alembic — sighup/user-context GUCs apply with no restart. The DB is ephemeral (rebuilt per run) so fsync-off is safe; the step is non-fatal so a perms surprise can't red a shard. Speeds up every test's commit (setup inserts + the teardown TRUNCATE), stacking on the pooled engine from the prior commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:45:21 -04:00
bvandeusenandClaude Opus 4.8	c802b26406	perf(ci): reuse a session-scoped engine for the per-test DB reset ... The autouse integration teardown created a fresh SQLAlchemy engine + Postgres connection for EVERY test, then disposed it — --durations showed the 15 slowest ops in both long shards were all ~1.5-2s teardowns (the connect+SCRAM handshake, not test logic). Hoist the truncate engine to a session-scoped, pool_pre_ping'd fixture so the pooled connection is reused across teardowns; the TRUNCATE+restore still runs per test, so isolation is unchanged. Lazy create_engine means the no-DB unit job instantiates but never connects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:35:28 -04:00
bvandeusenandClaude Opus 4.8	3a4270e6be	fix(showcase): reveal each tile only once its image is fully decoded ... The buffered cascade revealed tiles on an 80ms timer regardless of image load, so the flip-in animation played on a gray placeholder and the thumbnail popped in afterward. Worse, MasonryGrid ALSO applied a per-index animation-delay (index×70ms) that compounded on top of the insert cadence, so the cascade visibly dragged and desynced as it grew. Now the producer preloads each queued thumbnail (decode pipelined ahead) and the consumer awaits that decode before pushing the item — every tile animates in fully loaded, strictly one at a time. Drop the compounding CSS stagger; the store's one-item-at-a-time push is the sole pacer, so each tile animates the instant it mounts. New utils/preloadImage.js (load+decode+timeout gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:19:44 -04:00
bvandeusenandClaude Opus 4.8	ae569c0f9a	fix(gallery): ruff C408 (dict literal) + panel auto-open on deep-link ... Rewrite facets() common/plat_scope as dict literals (C408). Open the refine panel via a watch on hasRefineFilters rather than reading filter state at bar-setup time — the parent applies the URL query in its onMounted, after the bar child has set up, so the initial read was always the default (empty) state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:14:15 -04:00
bvandeusenandClaude Opus 4.8	1adc47f59c	feat(gallery): faceted refine panel UI (Phase 2 frontend) ... Add a 'Refine ▾' toggle to the gallery filter bar that expands a full-width GalleryFacetPanel below it, inside the same sticky hazey chrome. The panel offers platform chips (with live counts + a 'No platform' unsourced bucket), two count-badged curation-flag toggles (Untagged / No artist), and a from/to date range bounded by the facet min/max. Store gains the platform/untagged/no_artist/date_from/date_to filter params (URL-mirrored, AND-composed) and a panel-gated, single-flighted loadFacets() that fetches /api/gallery/facets scoped to the active filter. Shared cloneFilter/filterToQuery helpers keep the bar and panel writing one URL format. The panel auto-opens on deep-link when refine filters are present and refetches counts (debounced) on every filter change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:06:51 -04:00
bvandeusenandClaude Opus 4.8	9fe534139a	feat(gallery): faceted filter params + /facets counts endpoint (Phase 2 backend) ... Extend the composable gallery filter with platform / untagged / no_artist / date_from / date_to, AND-composed with the existing tag/artist/media/sort params and threaded through scroll, timeline, and jump_cursor. Add GalleryService.facets() + GET /api/gallery/facets returning live counts scoped to the current filter with per-group minus-self semantics: platform counts (COUNT(DISTINCT image) incl. a null unsourced bucket), curation-flag counts (untagged / no_artist), and effective_date min/max bounds. The UNSOURCED_PLATFORM sentinel makes filesystem-imported content reachable via the platform facet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 07:01:07 -04:00
bvandeusenandClaude Opus 4.8	16e0268da7	fix(modal): next arrow clears metadata panel + arrows work in empty tag input ... Image viewer (#609): - The next (▶) arrow was offset from the viewport edge (right:16px) so it floated over the 320px metadata side panel. Offset it off a shared --fc-side-w var so it sits at the image's right edge instead; full-width again below 900px when the panel stacks under the image. - Arrow nav was fully disabled whenever a text field was focused. Now it yields to the caret ONLY when the field has text; an empty tag-entry field still navigates ←/→. Extracted to utils/textEntry.js (arrowNavAllowed). ESC behaviour unchanged (already closes the modal, overlay-aware). Test: arrowNavAllowed — empty/non-text → navigate, text present → don't. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 06:37:16 -04:00
bvandeusenandClaude Opus 4.8	1f4ce8513b	style: ruff I001 — keep _sync_engine as-import on its own line (combine-as-imports=false) ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:59:26 -04:00
bvandeusenandClaude Opus 4.8	5a116ca9d0	style: ruff I001 — aliased _sync_session_factory sorts before get_sync_engine ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:55:18 -04:00
bvandeusenandClaude Opus 4.8	ef3ee5aceb	feat(maintenance): DB maintenance UI card + fix ruff I001 ... - Settings → Maintenance gains a "Database maintenance" card: a "Run VACUUM ANALYZE now" button (enqueues the maintenance task) plus a per-table bloat readout (live/dead/dead%/last vacuum) from /api/admin/maintenance/db-stats. - dbMaintenance store (loadStats / runVacuum) + test. - Fix ruff I001: combine the two _sync_engine imports onto one line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:52:36 -04:00
bvandeusenandClaude Opus 4.8	914033db29	feat(maintenance): scheduled + manual DB VACUUM ANALYZE + bloat readout ... The TABLESAMPLE showcase reads physical blocks (bloat-sensitive), and the periodic prune/backfill/recovery tasks churn dead tuples faster than autovacuum always keeps up — so explicit maintenance earns its keep here. - tasks.maintenance.vacuum_analyze: VACUUM (ANALYZE) over high-churn tables (VACUUM_TABLES) on an AUTOCOMMIT connection (VACUUM can't run in a txn). Scheduled weekly via Beat; also operator-triggerable. - _sync_engine.get_sync_engine(): expose the process engine for the autocommit connection. - GET /api/admin/maintenance/db-stats: per-table n_live/n_dead/dead_pct + last (auto)vacuum/analyze from pg_stat_user_tables — visibility, not a black box. - POST /api/admin/maintenance/vacuum: enqueue the task on demand. Tests: vacuum task runs + reports tables; db-stats shape; trigger queues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:49:39 -04:00
bvandeusenandClaude Opus 4.8	d495605c12	style(gallery): hazey filter bar attached to the TopNav ... - Filter bar gets the same obsidian translucent + backdrop-blur as the TopNav so the two read as one piece of chrome. - margin-top:-8px cancels the v-container's pt-2 so the bar sits flush at 64px even at scroll 0 — fixes the gap/separation when scrolled to top. - Inputs/toggles get a more-opaque backing so they stay legible on the haze. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:40:32 -04:00
bvandeusenandClaude Opus 4.8	76d8ad42a8	fix(showcase): buffered producer/consumer for a steady cascade ... The cascade "burped" — chunks appeared unevenly — because the old pipeline coupled display to fetch timing: it trickled each batch right after its fetch and assumed the next round-trip would land inside the ~240ms trickle window. When a fetch ran long (TABLESAMPLE hits random, sometimes-cold blocks; RTT jitter) the animation starved, then a clump burst in. Decouple the two: - Producer (_fill) races ahead fetching batches into a buffer up to a target depth, refilling when it dips below BUFFER_MIN. - Consumer (_drain) reveals one item every CADENCE_MS regardless of when fetches land; it only waits if the buffer genuinely starves. A small PRIME buffer precedes the drain so it doesn't starve at the front; the buffer (BUFFER_MIN×CADENCE runway) absorbs per-fetch jitter so images appear at an even pace. Public store API (loadInitial/shuffle/fetchPage/ images/loading/hasMore/isEmpty) unchanged — ShowcaseView/MasonryGrid need no change. Test (fake timers): fire-order + dedup, one-item-per-cadence rate limit, empty-library flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 00:35:25 -04:00
bvandeusenandClaude Opus 4.8	6d630d13d6	feat(gallery): pinned filter bar (Phase 1) ... Gallery now has in-view filtering, styled like the app's sticky v-tabs chrome (pinned at top:64px under TopNav). - GalleryFilterBar: combined tag+artist autocomplete (searches /api/tags + /api/artists), closable filter chips (multi-tag AND), media toggle (All/Images/Videos), Newest/Oldest sort, Clear. Writes all state to the URL via router.push. - gallery store: filter is now { tag_ids, artist_id, media_type, sort, post_id }; applyFilterFromQuery makes the URL the single source of truth (deep-linkable, back-button works); chip labels resolved by id or pre-noted on pick. Replaces the standalone tag chip + setTag/PostFilter. - GalleryView: renders the bar (hidden in post-detail), syncs route.query → store on mount + every query change. Also untracks the transient .claude/scheduled_tasks.lock committed in 3f30327 and gitignores it. Tests: store parses query → composable scroll params, post_id exclusivity, newest-sort omitted, label pre-seed, single initial fetch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:57:11 -04:00
bvandeusenandClaude Opus 4.8	3f30327fa5	feat(gallery): composable scroll filter (multi-tag AND, media, sort) ... Phase 1 backend for the gallery filter bar. Extends scroll/timeline/jump from a single mutually-exclusive filter to a composable one: - tag_ids: image must carry ALL of them (one correlated EXISTS per tag — AND, no row multiplication), replacing the single-tag JOIN. - artist_id composes with tags; media_type ('image'|'video') narrows by mime; post_id stays the exclusive post-detail path. - sort ('newest'|'oldest') flips the effective_date/id cursor comparison and ordering; the cursor value is unchanged (direction comes from the request). jump_cursor honors sort too. - Shared _apply_scope helper applied across scroll/timeline/jump so the timeline sidebar reflects the filtered set. API _parse_filters parses tag_id (comma list), artist_id, media, sort. Tests: multi-tag AND, media filter, sort reversal (service + API); post_id-excludes-others; single tag_id back-compat. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:50:19 -04:00
bvandeusenandClaude Opus 4.8	4f9464d215	feat(gallery,tags): clear active filters ... Two gaps where a filter couldn't be removed: - Gallery: a tag_id filter (from clicking a tag) had no indicator or clear control — only post_id did (PostInfoHeader). Add an "Tag: <name> ✕" chip that clears the filter by dropping tag_id from the URL. New lightweight GET /api/tags/<id> resolves the name; the store fetches it on filter set. - Tags view: the kind chip-group used mandatory="false" — a STRING ("false" is truthy in JS), which made the group mandatory so the active kind chip couldn't be deselected. Fixed to :mandatory="false" so the filter clears. Tests: GET /tags/<id> shape + 404; gallery store resolves filterTagName. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:26:04 -04:00
bvandeusenandClaude Opus 4.8	e678d1dfdf	feat(tags): fandom-edit UI in tags directory + image modal ... Adds the missing UI to change a character tag's fandom, in both places: - FandomSetDialog (shared): pick an existing fandom, create a new one, or clear it; on a name collision in the target fandom it surfaces a merge confirmation and resolves via setFandom(merge:true). Reuses the tags store's fandom cache. - TagCard kebab gains "Set fandom…" for character tags (→ TagsView opens the dialog, reloads on success). - TagPanel chip kebab gains "Set fandom…" for character tags (→ reloads the modal's tag list on success). - tags store: setFandom(tagId, fandomId, {merge}) action + test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:21:25 -04:00
bvandeusenandClaude Opus 4.8	d9ab6e15c6	feat(tags): edit a character tag's fandom (backend) ... No way existed to change which fandom a character tag belongs to after creation — PATCH /tags/<id> only renamed. - TagService.set_fandom(tag_id, fandom_id, merge=False): set / change / clear (fandom_id=None) a character's fandom, with the same validation as find_or_create. On a name collision in the target fandom it raises TagMergeConflict (→ 409, same shape as rename); merge=True resolves it by merging this tag INTO the existing character. - Extract _do_merge(source, target) from merge() so set_fandom can perform the deliberate CROSS-fandom merge the public merge() validation forbids. - PATCH /tags/<id> now accepts optional fandom_id (+ merge flag) alongside name, and returns fandom_id. Tests: set/change/clear, non-character + bad-ref rejection, collision raises, merge resolves; API set/clear + collision→merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 23:13:24 -04:00
bvandeusenandClaude Opus 4.8	e05e0b9f37	perf(gallery): materialize indexed effective_date sort key ... The gallery cursored on COALESCE(post.post_date, image_record.created_at) across the Post outer join — an expression spanning two tables that no index can serve, so every /scroll sorted a large slice of the library (and the old frontend fired ten serially). Materialize it: - image_record.effective_date column + ix_image_record_effective_date (effective_date DESC, id DESC); alembic 0035 backfills COALESCE(primary post's post_date, created_at) for existing rows. - gallery_service._effective_date_col() now returns the column, so scroll / timeline / jump / neighbors all order off the index instead of re-deriving the COALESCE. _neighbors reads record.effective_date directly (drops an extra Post lookup). - importer._apply_sidecar maintains it: when a primary post with a date is linked, effective_date = post.post_date; plain inserts keep the created_at-equivalent server default. Tests: sidecar import asserts effective_date == post.post_date; gallery ordering/timeline/jump test seeds set effective_date alongside created_at. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 22:58:46 -04:00
bvandeusenandClaude Opus 4.8	56cc253009	feat(gallery): reveal tiles on image load + single initial fetch ... The 5×10 metadata batching only staggered the cheap layer (JSON); thumbnails load as independent <img> requests and clustered, so tiles "popped in together" after a wait. Two changes: - GalleryItem reveals each tile when ITS OWN thumbnail fires @load (with an onMounted complete-check for cached thumbs), playing a showcase-style flip-up entrance. Tiles now cascade in natural load order instead of all at once. Honors prefers-reduced-motion. - gallery store does ONE initial fetch (limit=50) instead of 10 serial /scroll round-trips. Fewer RTTs, faster first paint; the reveal-on-load is what makes appearance progressive now. Infinite scroll pulls 25/trigger. Tests: GalleryItem gains is-loaded only after @load; loadInitial issues exactly one scroll request at the initial limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 22:50:03 -04:00
bvandeusenandClaude Opus 4.8	576e16d14d	fix(download): release DB connections across the gallery-dl subprocess ... Backfill events were STILL stranding empty after the timeout-ladder fix. Worker logs showed the salvage path working ("Download timeout for anduo/patreon after 1170.0s (18 files written)") but then: Retry in 3s: DBAPIError(ConnectionDoesNotExistError: connection was closed in the middle of operation) ...succeeded in 0.149s <- in-flight guard no-op Root cause: DownloadService held the async + sync DB connections checked out across the entire (≤19.5-min backfill) gallery-dl subprocess. The server reaps the idle connection, so phase 3's first query hits a dead socket. That DBAPIError trips download_source's autoretry_for, the retry re-enters _phase1_setup, sees the event still 'running', returns in_flight and no-ops — leaving the event to be stranded empty by the recovery sweep. pool_pre_ping was already on both engines but can't help a *held* connection (it only validates on pool checkout). Fix: - DownloadService.download_source closes the async + sync sessions after phase 1, before the subprocess, so phase 3 re-acquires a live connection (matches the class's "Phase 2 — no DB connection" docstring). - The per-task async engine switches to NullPool so phase 3 always opens a fresh connection rather than a pooled one the server may have reaped. Tests: assert connections are released before gdl.download runs and the event still finalizes; assert the task engine uses NullPool. Also fixes a stale 1800s->1170s comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 21:49:52 -04:00
bvandeusenandClaude Opus 4.8	9cb24c9e1b	style(test): fix ruff I001 import order in download task test ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 17:04:29 -04:00
bvandeusenandClaude Opus 4.8	6590dcdb39	fix(download): salvage soft-time-limit kills + fix timeout ladder ... Backfill downloads stranded with empty logs + a generic "stranded by recovery sweep" error. Root cause: the backfill gallery-dl subprocess timeout (1170s) exceeded download_source's Celery soft_time_limit (900s), so SoftTimeLimitExceeded preempted subprocess.TimeoutExpired. The TimeoutExpired path (which captures partial stdout/stderr and finalizes the event) never ran, the event was left 'running', and phase 3 never decremented backfill_runs_remaining — so the source re-ran and re-stranded every tick (Anduo #39912). Two layers: 1. Raise download_source limits (soft 900→1350, hard 1200→1500) so both subprocess budgets (870 tick / 1170 backfill) sit below the soft limit with phase-3 persist headroom. Promote to module constants and guard the invariant with a test. 2. Catch SoftTimeLimitExceeded in download_source and finalize the in-flight event with a real reason, mirror phase-3 source-health, and decrement backfill so a chronically-slow source self-heals to tick mode. The existing celery_signals handler only covered TaskRun, not DownloadEvent — that was the gap. Updates stale 900/1200 references in gallery_dl.py + maintenance.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-03 16:56:13 -04:00
bvandeusen	3162cff96b	fix(artist): ruff UP017 + test_directory_card_shape pin ... Two CI bounces on b65e956: 1. ruff UP017 — Python 3.14's preferred form is `datetime.UTC`, not `timezone.utc`. Switch the test's two TZ literals. 2. test_directory_card_shape pinned the card key set to the pre-feature shape; `unseen_count` was added to the API payload but the pin wasn't updated. Same shape as the recurring 'plan-grep-pinned-tests' trap — should have grepped tests/ for card.keys() before pushing.	2026-06-03 15:45:59 -04:00
bvandeusen	b65e956ad2	feat(artist): "new since last visit" badge + banner ... Per-artist "+N" accent pill on the artists directory and a "N new since last visit" banner inside ArtistView. Counts new IMAGES (not posts) so multi-image posts increment correctly. - alembic 0034: artist_visit (artist_id PK, last_viewed_at NOT NULL). Seeds every existing artist with last_viewed_at=NOW() so the badge starts at 0 across the board — no noisy "5000 unseen images" on first deploy. - ArtistService.find_or_create autoseeds a visit row alongside new artists, so freshly imported content doesn't read as unseen. - ArtistService.overview reads pre-visit last_viewed_at, counts images created since, then atomically UPSERTs last_viewed_at=NOW() via postgres ON CONFLICT DO UPDATE (no SELECT-then-INSERT race per reference_scalar_one_or_none_duplicates). Returns the pre-update count as `unseen_count_at_visit` so the banner has data. - ArtistDirectoryService.list_artists adds an `unseen_count` aggregate to each card via LEFT JOIN artist_visit + conditional COUNT. NULL last_viewed_at (artist created before this code shipped) defensively counts as "never visited" → all images unseen. - Frontend: ArtistCard renders an accent pill in the preview-strip corner when unseen_count > 0 (capped at 99+); ArtistView shows a closable v-alert banner on initial load when unseen_count_at_visit > 0, re-arms on slug change. Single-row-per-artist (no user_id) — rule #47 multi-user ACL is aspirational; widens to (user_id, artist_id) PK when User lands, per rule #22. Scribe plan #597.	2026-06-03 15:27:11 -04:00
bvandeusen	d3245f0c22	feat(ext): verify cookies in-browser before uploading (1.0.7) ... Pre-upload verify request: after capturing the live browser cookies, hit a known authenticated endpoint with credentials:'include' from the extension's background context. If the platform reports we're not logged in, abort the upload so we don't overwrite FC-side credentials with stale data. - platforms.js: add `verify` config per cookie-auth platform - hentaifoundry: HEAD /?enterAgree=1 (mirrors gallery-dl's HF _init_site_filters; same 401 path the operator hit 2026-06-03) - patreon: GET /api/current_user (clean 401 when logged out) - subscribestar, deviantart: no stable auth endpoint, skip verify - cookies.js: verifyCookiesForPlatform() returns {ok, status, reason}. ok=true/false/null tri-state — null = verify not configured, caller treats as "proceed". - background.js EXPORT_COOKIES + EXPORT_ALL_COOKIES: verify gates the upload; failures bubble up with the platform's name + reason. - popup.js: success message now appends "(verified ✓)" when applicable. - manifest + package.json: 1.0.6 → 1.0.7.	2026-06-03 14:04:45 -04:00
bvandeusen	e450145304	fix(ml): preserve digit-only tag names in normalize (year tags) ... Rule 8 'no letters -> drop' was over-eager: bare digit tags like '2005' returned None even though they're legitimate (booru year-tag shape). Widen the keep-condition to any alphanumeric. Emoticons (':/', '^_^', '+_+') still drop since they contain neither letters nor digits.	2026-06-03 13:09:35 -04:00
bvandeusen	a6e8d4b52e	feat(ml): normalize Camie suggestion names to human-readable ... Camie's booru-style vocab strings (`uchiha_sasuke_(naruto)`, `#unicus_(idolmaster)`, `1000-nen_ikiteru_(vocaloid)`, `:/`) were surfacing raw in SuggestionsPanel — and worse, the SAME raw string was written to tag.name on Accept, polluting the DB with `underscored_lowercase` names that don't match the operator's "Title Case" tag convention. Add backend/app/services/ml/tag_name.py with a single normalize() applying nine rules (strip leading junk #/./+/;/~/_/ws, drop trailing _(disambiguator) blocks iteratively, strip wrapping quotes, underscores to spaces, space after colon, title-case each word's first char, preserve hyphens/apostrophes/digits, drop entries with no letters). Wire into SuggestionService.for_image: - raw Camie key kept for alias_map lookup (alias rows are hand-curated against raw keys; don't disturb) - display_name = normalize(raw); None means drop the candidate - existing-tag lookup widened to case-insensitive match against BOTH raw and normalized forms so legacy underscore-named Tag rows accepted before this change still surface as "existing" not "+ new"	2026-06-03 13:00:08 -04:00
bvandeusen	f1860866de	chore(modal): drop the ?image=N soft-compat from G5.4 ... Operator confirmed they have no existing ?image=N bookmarks to preserve, so the soft-compat read on initial mount + router.replace strip is dead weight. The modal is now purely a Pinia overlay — no URL involvement on open OR initial mount. Drops 33 lines plus the now-unused vue-router imports in both ArtistGalleryTab (entire onMounted gone) and GalleryView (just the ?image=N block; the post_id/tag_id filter handling stays).	2026-06-02 19:24:09 -04:00
bvandeusen	b181d779fe	fix(test): default suggestion_threshold_general now 0.70 (alembic 0033)	2026-06-02 18:49:24 -04:00
bvandeusen	0fbb19dc24	fix(modal): TagPanel kebab — apply the wrapping-span fix that the prior commit missed ... The previous commit (8326e54) updated the CSS but the structural edit to the v-menu wrapper didn't take. Re-applying so the chip kebab actually opens.	2026-06-02 18:48:53 -04:00
bvandeusen	8326e5447a	fix(modal): kebab menus weren't opening (chip + suggestion rows) ... Operator-flagged 2026-06-02. Both kebab activators were broken: - TagPanel chip kebab had `@click.stop` directly on the v-icon activator. In Vue 3, an explicit @click on the same element as `v-bind="props"` overrides the spread onClick — so Vuetify's activator handler never fired. Menu never opened. - SuggestionItem kebab didn't have @click.stop, but for consistency and to make both kebabs follow the same shape, wrap it too. The fix: each kebab is now wrapped in a `<span @click.stop>`. The v-icon / v-btn receives Vuetify's onClick cleanly and opens the menu; the bubbling click then reaches the span where stopPropagation absorbs it before it can affect a parent (the v-chip's close button in TagPanel's case).	2026-06-02 18:48:32 -04:00
bvandeusen	1fd594baaf	chore(ml): suggestion_threshold default 0.50 → 0.70 ... Operator-flagged 2026-06-02 — the 0.50 default (set on 2026-06-01) surfaces too many low-confidence picks in the modal's Suggestions rail. 0.70 keeps the rail signal-rich while still showing more than the original 0.95 (which hid almost everything). Alembic 0033 updates the singleton row conditionally — only rows still at the old 0.50 default flip to 0.70. Operators who tuned to some other value via Settings → ML keep their pick. Settings UI already exposes both sliders (MLThresholdSliders.vue), so further tuning continues to work without a deploy.	2026-06-02 18:38:12 -04:00
bvandeusen	ecac6c4bda	fix(audit-g5): centroid version DB-as-truth + modal as overlay ... Closes the last two findings from the 2026-06-02 audit (G5.1 + G5.4). G5.1 — Centroid version no longer drifts: CentroidService now reads MLSettings.embedder_model_version (the DB row tag_and_embed already writes from) for both the centroid model- version stamp and the drift-detection comparison. Previously the centroid sites imported MODEL_VERSION from env, so the version stamped on centroids could disagree with the version stamped on the embeddings they were built from. By construction those now match, so list_drifted won't silently miss the env-vs-DB drift case. embedder.py keeps MODEL_VERSION as an env-driven constant for the actual model loader — that's a different concern (which weights are loaded) from the version-stamp that gets persisted alongside data. G5.4 — Modal is a Pinia-only overlay: The previous URL↔modal sync in GalleryView and ArtistGalleryTab leaked the modal across route changes (RouterLink to /artist/<slug> left the modal mounted on top of the new route) and re-opened it on history back/forward with stale ?image=N entries. Now: openImage() just calls modal.open(id) — no URL push. GalleryView's dead closeImage helper is deleted. A route.name watcher in App.vue closes the modal whenever the route changes, which auto-fixes RouterLink-in-modal and back/forward. Backward-compat: ?image=N is still honored on initial mount as a one-shot deep-link opener, then router.replace strips the query so the URL doesn't re-trigger and no extra history entry is added. Existing bookmarks / shared URLs keep working; new opens stay Pinia-only.	2026-06-02 18:28:57 -04:00
bvandeusen	9f7261b9c0	fix(audit-g5c): set CURATOR_BOOTSTRAP_NEW_KEY=1 in conftest ... CredentialCrypto's safety check fires on create_app() instantiation because the test environment has no pre-seeded Fernet key file. Set the bootstrap env var before any test imports so the auto-create path is allowed during tests.	2026-06-02 17:55:25 -04:00
bvandeusen	f05aaa707b	fix(audit-g5d): surface ErrorType taxonomy on FailingSourcesCard ... Alembic 0032 adds Source.error_type (varchar(32), indexed). _update_source_health stamps it alongside last_error on status='error' and clears it on 'ok'. SourceRecord/to_dict exposes it. FailingSourcesCard renders a colored chip next to the consecutive- failures count, with a tooltip explaining the suggested operator action. Color reflects intent: - warning (yellow) — operator action needed (auth_error) - info (blue) — backend-paced (rate_limited / timeout / network_error / partial / tier_limited) - error (red) — likely terminal without intervention (not_found / access_denied / validation_failed / unsupported_url / http_error / unknown_error) Audit 2026-06-02: the backend computed 13 ErrorType categories but only the free-text last_error reached the operator. Bulk-triage by class ("all auth_error → rotate cookies", "12 rate_limited → just wait") required opening Logs per row.	2026-06-02 17:54:44 -04:00
bvandeusen	4df98171ab	fix(audit-g5c): refuse silent Fernet key regeneration on partial restore ... Audit 2026-06-02: `_load_or_create_key` silently minted a new Fernet key whenever the key file was missing — no log, no warning. The failure mode the audit flagged: a partial disaster restore where the DB was restored but `/images/secrets/` was lost would produce a working-looking system in which every authenticated download fails AUTH_ERROR until the operator re-uploads every credential by hand. Two opt-ins now needed for auto-creation: 1. Explicit `bootstrap_ok=True` kwarg (tests, scripts), OR 2. `CURATOR_BOOTSTRAP_NEW_KEY=1` env var (operator first-time setup) Otherwise the constructor raises `MissingCredentialKey` so the app fails fast at startup and the operator can restore the key file from backup before encrypted_blob rows go undecryptable. Also: docstring path was wrong (said "images/data root" but actual location is `/images/secrets/credential_key.b64`) — corrected. Tests updated to pass `bootstrap_ok=True` explicitly, and two new tests cover the safety behavior (missing-key-raises, env-var-bootstraps).	2026-06-02 17:04:27 -04:00
bvandeusen	8d75ade1d5	fix(audit-g5b): race-poisoning in three find-or-create sites ... Audit 2026-06-02 flagged three SELECT-then-INSERT sites that lost the race under concurrent writers / recovery-sweep replays, poisoning the outer transaction with an unrecoverable IntegrityError and crashing the calling task. Same shape as the banked 2026-05-26 ImageProvenance incident (see reference_scalar_one_or_none_duplicates memory). Mirrors the importer._get_or_create pattern in all three: savepoint via begin_nested + IntegrityError rollback to that savepoint + re- SELECT to grab the row the other worker committed first. - importer._capture_attachment: PostAttachment.sha256 UNIQUE. attachments.store is sha-addressed so both workers race to write the same on-disk path (shutil.copy2 + rename is idempotent), so no extra cleanup needed. - TagService.find_or_create: partial uniqueness index on (name, kind, COALESCE(fandom_id, -1)). The previous docstring claimed "INSERT ... ON CONFLICT" but the implementation was SELECT-then-INSERT with no recovery. - ArtistService.find_or_create: already had IntegrityError handling but did session.rollback() (unwinds the WHOLE transaction); now uses begin_nested + sp.rollback() so the surrounding request's progress isn't lost.	2026-06-02 17:02:12 -04:00
bvandeusen	75c63e1511	fix(audit-g5a): ruff isort — platforms after patreon_resolver	2026-06-02 16:55:56 -04:00
bvandeusen	98673d4dca	fix(audit-g5a): small architectural cleanups bundle ... Five small G5 findings from the 2026-06-02 audit. Each is local and follows an established FC pattern. - download_service: replace hardcoded ('discord','pixiv') tuple with auth_type_for(platform) == 'token'. A 7th token-platform now picks up the right credential path without touching this site. - /api/tags/<source_id>/merge enqueues recompute_centroid.delay after merge so the target's centroid reflects its new image set immediately. Daily list_drifted catches it within 24h, but eager recompute closes the suggestion-quality dip in the meantime. - backfill_thumbnails added to beat_schedule (daily). The task docstring claimed periodic Beat but the entry was never registered, so the library got no self-healing thumbnail repair; only the manual admin-UI button fired it. - modal.createAndAdd pushes a kind='fandom' tag into tagsStore.fandomCache so FandomPicker sees the new fandom on next open. Was: cache-gated load (length===0) skipped refetch, new fandom invisible until full page reload. - cleanup cluster: - Drop .webp from cleanup_service.unlink — thumbnailer only writes .jpg/.png; the third tuple member was dead code. - Drop effective_date from /api/gallery/scroll response — no FE consumer reads it. Service still computes the attribute for timeline ordering; this just trims the JSON. - Rename store.recentMinute → store.recentRuns across the systemActivity store + three consumers (SystemActivitySummary, QueuesTable, SystemActivityTab). The data is the last 200 runs (not actually "last minute"), so the name lied. NOT in this bundle: the duplicate tag-merge endpoint (/api/tags vs /api/admin/tags) is harder — has 1 FE caller and 3 tests on the admin variant; consolidation is its own change.	2026-06-02 16:46:46 -04:00
bvandeusen	4bff1d8558	fix(audit-g4): status-enum miss batch ... Five extension-miss findings from the 2026-06-02 audit, where a status value was added on one side but a downstream consumer didn't pick it up. - download_service._phase3_persist: explicit branches for ImportResult.status in ('failed','refreshed'). For 'failed' (archive probe crash from _import_archive), unlink the source file so the filesystem scanner doesn't re-import and re-crash on the same archive forever. 'refreshed' is currently unreachable from the download path (no deep=True) but matches the importer's documented contract; treat as 'attached'. - gallery-dl backfill auto-complete now gates on dl_result.success + no error_type, not just return_code==0 + files_downloaded==0. VALIDATION_FAILED exits the subprocess with returncode=0 and files_downloaded=0 when every file was quarantined, matching the prior predicate exactly and zeroing the operator's armed backfill budget on the FIRST quarantine run instead of decrementing. - attach_in_place archive dispatch now threads artist + source_row through _import_archive (and _import_media for archive members) and _supersede. The path-walk fallback (_resolve_artist) is still used by filesystem-import; the download path now binds ImageProvenance to the explicit subscription Source instead of rediscovering by (artist_id, platform). - Three FE handlers now recognize status:'deferred' from /api/sources/<id>/check: SubscriptionsTab.onCheck (was toasting "event #undefined"), SubscriptionsTab.checkAll (was counting deferred as queued), DownloadEventRow.onRetry (was saying "re-queued" when nothing was). Pattern matches DownloadsTab.onRetryAll which already had it. - celery_signals._queue_for now maps backup/admin/library_audit prefixes to 'maintenance' (matching task_routes). TaskRun.queue was returning 'default' for those rows, so per-queue dashboard filters and per-queue threshold overrides (added in G3) silently missed them.	2026-06-02 16:04:59 -04:00
bvandeusen	e30f50e6fe	fix(audit-g3): lifecycle batch — recovery sweeps, retention, timeouts ... Plugs the FC long-running-entity discipline gaps the 2026-06-02 audit flagged: every entity that can get stuck now has recovery + retention + timeout, and the long-runners no longer collide with the FC-3i sweep. Recovery sweeps (every 5 min): - recover_stalled_backup_runs — flips BackupRun stuck in running/restoring past 7h (covers the 6.5h images-backup hard limit) to error. prune_backups docstring corrected — the FC-3i TaskRun sweep never touched BackupRun rows. - recover_stalled_library_audit_runs — flips LibraryAuditRun stuck past 135 min (10-min buffer above scan_library_for_rule's 2h5m hard limit) to error. Previously a SIGKILL'd row blocked all future audits until manual DB surgery. - recover_stalled_import_batches — finalizes ImportBatch rows stuck running >2h whose child tasks are all terminal (orphan case where the orchestrator crashed before the closing UPDATE). Uses the same EXISTS predicate /api/system/stats already had. Retention (daily): - prune_library_audit_runs — 30-day window. Audit rows carry matched_ids JSONB blobs that can hold tens of thousands of ids. - prune_import_batches — 30-day window. Cascades to ImportTask via the model relationship. time_limits on five long-runners that previously had none (the audit's headline finding — every one of these collided with the recover_stalled_task_runs 5-min default and could be marked 'error' mid-flight): - scan_directory: 60m soft / 70m hard - verify_integrity: 60m / 70m - backfill_phash: 30m / 35m - apply_allowlist_tags: 30m / 35m - recompute_centroids: 30m / 35m QUEUE_STUCK_THRESHOLD_MINUTES now covers maintenance (75) and scan (75) — above the longest task on each — with per-task overrides for the outliers (backup_images_task 420, restore_images_task 420, scan_library_for_rule 130). start_audit_run guard is now age-aware: a 'running' row older than the audit hard limit doesn't block a new run (the sweep will catch it within 5 min). Previously a SIGKILL'd row blocked forever. /api/import/status now uses the same EXISTS predicate /api/system/stats does, so the two endpoints no longer disagree on the active-batch question. DownloadEvent.started_at resets on pending→running so a freshly- promoted event from a busy queue isn't measured against its original enqueue time (was racing recover_stalled_download_events on heavy-queue days).	2026-06-02 14:30:46 -04:00
bvandeusen	e66987f092	fix(audit-g2): async race / state-leak across eight stores ... Extracts gallery.js's hand-rolled inflightId pattern into a new useInflightToken composable; adopts in every store that previously had no guard against late-response overwrites or wrong-image URL interpolation. Two operator-impacting bugs the audit (workflow wf_bbe3fdb1-e62) flagged: - modal.removeTag rolled back the chip rail unconditionally even when only the secondary dismiss POST had failed — UI lied until refresh. And all tag-mutation URLs interpolated currentImageId AFTER an await, so a fast prev/next could route DELETE/POST to the wrong image. Both fixed: split try/catch (dismiss failure surfaces a warning, doesn't roll back the delete); imageId captured at call-time and used in URLs throughout. - suggestions.accept dereferenced currentImageId after the awaited POST /api/tags, so the subsequent /suggestions/accept could apply A's chosen tag to image B AND push it to B's allowlist. Fixed by capturing imageId at click-time + inflight guard on load(). Same shape across artist / downloads / artistDirectory / tagDirectory / posts stores: rapid filter/nav changes used to interleave responses (last-writer-wins). Now the late response is discarded and the most-recent request wins. Filter-change-during- search no longer drops the second fetch because the loading flag was still true from the first. gallery.js's inflightId removed in favor of the shared composable so the pattern stays consistent.	2026-06-02 14:07:58 -04:00
bvandeusen	80ef9bce48	fix(test): credentials.upload reflects record into cache (not invalidate) ... Stale assertion pinned the buggy pre-G1.6 behavior (the test name "upload invalidates the cache" literally describes the bug). The audit's correction makes upload mirror the returned record into the store so the card updates immediately — update the assertion to match the corrected behavior.	2026-06-02 13:11:39 -04:00
bvandeusen	3898ce7be4	fix(audit-g1): six one-liner drift fixes from 2026-06-02 audit ... - BACKFILL_TIMEOUT_SECONDS 1800→1170: keep the subprocess timeout 30s below Celery's hard time_limit=1200 so SIGKILL doesn't beat TimeoutExpired (matched the tick 870s/900s rationale). Backfill runs that hit the cap let the next tick continue via the archive. - recover_interrupted_tasks orphan UPDATE now stamps finished_at; without it cleanup_old_tasks' WHERE finished_at<cutoff never reaped orphan-swept rows. recover_stalled_task_runs also now sets duration_ms (matches celery_signals.finalize's millisecond math). - ExtensionService.quick_add_source arms NEW_SOURCE_BACKFILL_RUNS=3 on Source creation, mirroring SourceService.create. Without it, Firefox quick-add on a creator with >20 unsynced posts walked the full feed until subprocess timeout. Renamed the constant from _NEW_SOURCE_BACKFILL_RUNS so it can be imported cross-module. - gallery_dl.verify() accepts TIER_LIMITED as auth-success alongside NO_NEW_CONTENT — the download path (line 712) already does, and TIER_LIMITED proves auth reached the post and was told it was tier-gated. Verify endpoint previously showed red on this and prompted operators to rotate working cookies. - prune_unused_tags now runs a single DELETE with the NOT-IN predicate find_unused_tags uses, instead of SELECT-ids → DELETE-WHERE-IN. Removes the psycopg 65535-param cliff that would have surfaced on a tag explosion (>65k unused tags). - credentials.upload() reflects the returned record into the store cache (`.set(platform, rec)`) instead of evicting it; previously the card briefly rendered "no credential" between upload and loadAll().	2026-06-02 11:24:05 -04:00
bvandeusen	91be9df671	fix(download): dispatch archive/non-media in attach_in_place; reshuffle showcase on mount ... attach_in_place mirrored only the media flow, so gallery-dl-downloaded zips/PDFs/audio bounced back as `skipped+invalid_image`, which download_service counted as an ingest error and flipped runs to status="error" despite N successful image attaches. Lustria patreon event #38998 (21 images + 1 OST zip) went red for exactly this reason. Now attach_in_place dispatches the same way as import_one: archives → _import_archive (extracts media members, captures archive as PostAttachment), non-media → _capture_attachment. Download_service accepts the new `attached` result and treats non-duplicate skips as soft skips, not ingest errors. Also: ShowcaseView always loadInitial() on mount, not just when the store is empty — Pinia persists across navigations and operator wants a fresh shuffle every time the showcase loads.	2026-06-02 08:16:13 -04:00
bvandeusen	412edec028	fix(showcase): don't exhaust on all-dupe batches; retry up to 8x ... /api/showcase returns a random sample; after enough scrolling, an unlucky 3-item batch can be entirely in the `seen` Set, which was flipping `exhausted=true` and surfacing "End." mid-scroll. The showcase is endless by intent — only a genuinely empty API response (library has 0 images) should mark it exhausted. Tiny-library fallback: cap retries at 8 to avoid spinning forever.	2026-06-01 23:37:21 -04:00
bvandeusen	937421485d	fix(modal): refresh tag chips after accepting a suggestion ... Operator-flagged 2026-06-01: clicking Accept on a suggestion added the tag to the database but the modal's chip rail kept showing the old set. Suggestion would disappear from the suggestions list (suggestionsStore drops it from byCategory) and the toast would confirm "Tagged: …", but visually the modal looked unchanged until you closed and reopened it. Root cause: useSuggestionsStore.accept() POSTs to the backend and updates its own state, but never tells useModalStore that the backing image's tag set has changed. The addExistingTag and createAndAdd flows in TagPanel already call modal.reloadTags() after applying — the suggestions path needed the same refresh. Two-line fix in SuggestionsPanel: after store.accept() and store.aliasAccept(), call modal.reloadTags() so TagPanel's `modal.current?.tags` rebinds.	2026-06-01 23:02:57 -04:00
bvandeusen	43b778aa04	fix(test): include 'scanned' in all backfill result assertions ... Same 'change shared shape, miss pinned tests' trap as cfa4fb4 — the prior commit updated only test_backfill_mixed_aggregate; three sibling tests in the same file also pinned the old {enqueued,ok,regenerated} shape without 'scanned' and CI bounced.	2026-06-01 22:09:02 -04:00
bvandeusen	9cbdb70e13	fix(thumbnails): surface backfill results + tighten validity check ... Two coupled problems, operator-flagged 2026-06-01: "missing thumbnails but triggering backfill found nothing." 1. **Backfill UI was a black box.** `POST /api/thumbnails/backfill` returned just `{celery_task_id}` and the admin card said "Enqueued." with no counts. There was no way to tell whether the scan found 0 candidates, 5000 candidates, or whether the worker even picked up the task. "Found nothing" was indistinguishable from a broken queue. Fix: refactor the scan into a sync helper (`_run_backfill_scan`) shared by the Celery task and the API endpoint. The API now runs the scan in an executor and returns `{scanned, enqueued, ok, regenerated}`. The actual thumbnail generation work still goes to the thumbnail Celery queue per row via `generate_thumbnail.delay()` — the scan itself is fast (SELECT id+thumbnail_path + a file.stat() per row). 2. **`_thumb_is_valid` accepted header-only corrupt files.** The magic-byte check passed for any 8-byte file starting with a JPEG or PNG header, including empty/truncated/zero-pad files that browsers render as broken. Backfill counted these as `ok` and never regenerated. Fix: also require file size ≥ MIN_THUMB_BYTES (256). Real thumbnails are minimum ~2KB even on solid-color sources; header- only corrupt files top out around 12 bytes. 256 is well above the corrupt floor and well below any legitimate thumbnail. Plus the admin card now shows the per-run counts instead of "Enqueued.": Scanned 5,432 · enqueued 3 (2 regenerated) · 5,429 ok	2026-06-01 21:57:29 -04:00
bvandeusen	bd06794647	fix(downloads): enqueue thumbnail + ML tasks per attached image ... Operator-flagged 2026-06-01: downloaded images stayed at thumbnail_path=NULL until a periodic backfill sweep picked them up, surfacing as broken-thumbnail tiles in the gallery for hours after the download landed. Importer.attach_in_place deliberately skips inline thumbnail generation (importer.py:591-592) so the import queue stays moving — the CALLING task is responsible for the enqueue. tasks/import_file.py already did this (line 228-239). tasks/download.py / download_service did not — every gallery-dl-attached image landed un-thumbnailed. Fix in download_service._phase3_persist: after each `attach_in_place` returning status in (imported, superseded), fan out `generate_thumbnail.delay()` + `tag_and_embed.delay()` for each image_id. Lazy import avoids circular-import risk between download_service and the celery task modules that depend on it. Mirrors the existing pattern verbatim — single source of truth for "what fires after a successful attach" remains a comment in two places (filesystem-import task, download orchestrator) rather than a shared helper, because the contexts differ enough (sync session vs async orchestrator) that abstracting would obscure more than it'd share. Test covers the happy-path with two attached files: both get the thumbnail enqueue AND the ML enqueue, with image_ids drawn from ImportResult (so future supersede-on-attach paths stay covered).	2026-06-01 21:39:17 -04:00
bvandeusen	f575cfb93b	ux(failing-sources): visible row separators + clearer hover ... The zebra striping in 717b601 was too subtle — inside the tonal error card, surface-tint contrast washed out and rows still looked uniform. Operator-flagged the same issue twice. Replace the surface-tint approach with hard visual structure: - Bottom border on each row (1px white/0.08) — clearly delineated rows regardless of card background - Hover darkens to black/0.25 — much more obvious than the previous error-tint hover, gives the eye a clear horizontal track - Slightly more vertical padding (8px vs 6px) for tap-target comfort - Drop the gap; borders are the separator now	2026-06-01 20:57:41 -04:00
bvandeusen	717b601c81	ux(failing-sources): zebra striping + hover highlight on rows ... Operator-flagged 2026-06-01: with the failing-sources panel expanded, all rows have the same flat low-opacity background, no visual track from the artist name on the left to the Logs/Retry buttons on the right. Hard to tell which row a button belongs to when scanning down a list of 17. Three pure-CSS changes (no DOM, no logic): - 3px gap between rows (was 2px) — slightly more breathing room - Even rows get a darker surface tint (zebra striping) - Hover paints the whole row in a low-opacity error tint, so moving the cursor toward Logs/Retry lights up the whole horizontal band and the eye traces back to the artist name automatically	2026-06-01 20:14:48 -04:00
bvandeusen	cfa4fb4084	fix(test): drop stale 'starts at 0' assertion after auto-arm-on-create ... `test_set_backfill_runs_arms_source` was pinning the pre-auto-arm initial value (0) when checking that the override works. Now that create() pre-arms enabled sources to 3, the assertion is stale — the test was already verifying the override path, the pre-assertion just decorated it. Drop the pre-assertion; keep the override check. Other tests use raw Source(...) constructors (default to 0 via the column server_default), not SourceService.create(), so they're unaffected.	2026-06-01 20:12:55 -04:00
bvandeusen	2aa2002f22	feat(subscriptions): newly added enabled sources start in backfill mode ... A freshly created subscription has no gallery-dl archive yet, so the first poll in tick mode would walk forever — exit:20 doesn't trip until 20 contiguous archive-hits, and there are none. The wall-clock cap kicks in mid-walk, and the partial-success classifier (plan #544) gracefully labels it status=ok, but the operator still wonders why their new subscription isn't grabbing the back-catalog. Pre-arm `backfill_runs_remaining = 3` on create() when `enabled=True`. Same default as the manual "Deep scan" button, same auto-decrement and auto-reset rules — once the queue drains (clean exit + zero new files) or the budget runs out, tick mode resumes naturally. Disabled sources (sidecar synthetics with `url='sidecar:<platform>:<slug>'` that arrive disabled = False, or operator-added sources that start disabled deliberately) skip the pre-arm — they're never polled, no budget to waste.	2026-06-01 20:02:19 -04:00
bvandeusen	66ff671f09	fix(downloads): forward Patreon Referer/Origin to yt-dlp for Mux videos ... Operator-flagged 2026-06-01 (DaferQ patreon, event #38919). Video posts hosted on Mux carry a JWT that encodes a playback restriction policy (`playback_restriction_id` in the token). The token signature alone is not sufficient — Mux's CDN ALSO checks Referer/Origin on every fetch. gallery-dl's HEAD probe to `stream.mux.com/<id>.m3u8?token=...` returns 200 (token is valid, HEAD sends no Referer that Mux's policy rejects), but when yt-dlp follows up with the actual manifest GET it sends its own default Referer and Mux 403s. yt-dlp retries 4 times, gives up, the single video fails — every other image in the same post still downloads. Static `downloader.ytdl.raw-options.http_headers` block now pins Referer and Origin to `https://www.patreon.com` so yt-dlp's manifest fetch clears the policy check. Headers-only restrictions are now handled. Mux IP-range restrictions (if a creator opts into them) remain unfixable from our worker — those would need a Patreon-region IP. Per-video PARTIAL classifier (shipped in plan #544 last commit) already handles the residual case: a run that grabs all images and fails 1 video classifies as status=ok rather than flipping the whole event red.	2026-06-01 19:00:50 -04:00
bvandeusen	19aece1fc4	feat(download): tick/backfill modes + partial-success classifier (plan #544) ... Routine subscription polls walked the entire post history every tick even when nothing had changed, because gallery-dl's default `skip: True` continues iterating archived posts. A creator with ~550 archived posts (Knuxy patreon) saturates the 870s wall-clock cap before completing, even with zero downloads needed. Plus, a tier-limited run that downloaded hundreds of files but ran out the clock should be a warning, not an error. Two coupled changes, both operator-flagged 2026-06-01: * **Tick mode (default, cron polls).** New `TICK_SKIP_VALUE = "exit:20"` asks gallery-dl to exit after 20 contiguous archived items. Fresh subscriptions + new-content cases still walk normally; established subscription with zero new content exits in ~30s of HEAD requests instead of pegging the timeout. 20 (not 5) gives headroom against paywall warnings interleaving with archived items. * **Backfill mode (explicit, operator-triggered).** Sticky for N runs via new `Source.backfill_runs_remaining` (alembic 0031). While > 0, downloads use `skip: True` + 1800s timeout. Auto-decrements per run with early-reset to 0 when a clean run finds zero files (queue drained). N defaults to 3 — multiple runs give the system enough budget to finish a deep walk across timeout boundaries. New `POST /api/sources/{id}/backfill` arms the source; "Deep scan" button on each SourceRow (chip shows remaining count) wires it. Plus partial-success classifier: non-zero gallery-dl exit + ≥1 file downloaded + no source-level error fires `ErrorType.PARTIAL`, which download_service maps to `status=\"ok\"`. The run did real work; the next tick continues via gallery-dl's archive. No more red events for "timed out mid-walk after downloading 300 files." Retires `SourceConfig.skip_existing` — skip value is now derived from the source state and passed as a separate `skip_value` parameter through download() / _build_config_for_source(). `GD_DEFAULTS` drops the now-dead key (was inert data after this refactor). Tests cover: * tick + backfill skip-value emission in _build_config_for_source * PARTIAL classifier branch + TIER_LIMITED-wins-over-PARTIAL ordering * SourceService.set_backfill_runs validation + persistence * /api/sources/{id}/backfill 200/400/404 paths * download_service auto-decrement / auto-reset / tick-mode-no-touch * PARTIAL → status=ok in the orchestrator (no consecutive_failures bump)	2026-06-01 18:23:28 -04:00
bvandeusen	c9089b1d03	fix(gallery+tests): alias Post inside artist EXISTS; tests stop asserting synthetic Source ... gallery_service._provenance_clause artist branch was correlating its bare Post reference to the outer query's primary_post_id outer-join, so the artist filter silently matched zero rows for images with no primary post. Alias Post inside the EXISTS subquery so SQLAlchemy adds it to the inner FROM rather than treating it as a correlated outer table. Five sidecar/import tests still asserted that a synthetic Source row appears after a filesystem import. Alembic 0030 retired that behavior; the Post sits null-source and the artist linkage lives on Post.artist_id. Updated test_sidecar_creates_provenance, test_reimport_same_post_idempotent, test_sidecar_artist_used_when_no_folder_artist, test_supersede_applies_new_file_sidecar, and test_apply_sidecar_recovers_from_integrity_error to assert post.source_id IS NULL + post.artist_id linkage instead.	2026-06-01 14:40:28 -04:00
bvandeusen	644d538bab	fix(migration): use canonical fk_<table>_<col>_<ref> names per Base naming_convention	2026-06-01 14:23:18 -04:00
bvandeusen	ff35da4743	fix(lint): drop unused Source import after Post.artist_id refactor	2026-06-01 14:19:37 -04:00
bvandeusen	2f66de2928	feat(model): nullable Post.source_id + denormalized Post.artist_id; retire sidecar synthetics ... Operator-asked 2026-06-01 after the Dymkens orphan investigation (Scribe plan #540). The pre-2030 sidecar-synthetic Source pattern (`sidecar:<platform>:<slug>` enabled=false rows) existed solely to satisfy `Post.source_id NOT NULL`, and leaked into the Subscriptions UI as phantom subscriptions. Now the data model says what's true: filesystem-imported content with no live subscription has NULL source_id, full stop. ## Schema (alembic 0030) - `post.artist_id` — NEW NOT NULL FK to artist (CASCADE). Backfilled from source.artist_id in the migration. Indexed for the artist-filter queries. - `post.source_id` — NOT NULL → nullable; FK ondelete CASCADE → SET NULL. Deleting a Source detaches its Posts instead of destroying archived content (subscription ends, archive stays). - `image_provenance.source_id` — same nullable + SET NULL. - Partial unique index `uq_post_artist_external_id_null_source` on (artist_id, external_post_id) WHERE source_id IS NULL — guards filesystem-import dedup since the existing source-bound unique ignores NULLs (Postgres treats NULL != NULL). - Sidecar synthetic Sources deleted: NULL out FKs in post, image_provenance first, then DELETE FROM source WHERE url LIKE 'sidecar:%'. The Dymkens cleanup. ## Model + service changes - `Post.source_id` → `Mapped[int | None]`; new `Post.artist_id` denormalized. - `ImageProvenance.source_id` → `Mapped[int | None]`. - Importer: `_source_for_sidecar` (synthetic-creating) → `_lookup_source_for_sidecar` (returns None when no subscription). `_find_or_create_post` takes required `artist_id`; matches on (source_id, external_post_id) for source-bound posts or (artist_id, external_post_id) for NULL-source posts. - Service queries switched off the Source detour to use Post.artist_id directly: post_feed_service.scroll/around/get_post (LEFT JOIN to Source so NULL-source posts surface); artist_service date_row/ activity/post_count; provenance_service.for_image/for_post (LEFT JOIN); gallery_service._provenance_exists_where_artist via Post.artist_id instead of ImageProvenance.source_id → Source. - `_to_dict` and provenance dict-builders emit `"source": null` for NULL-source rows. ## Frontend - `ProvenancePanel.vue` + `PostCard.vue`: render `e.source?.platform ?? 'filesystem import'` so NULL-source posts get a clear "filesystem import" affordance instead of a NaN crash. ## Tests - `test_importer_upsert_helpers`: removed the four synthetic-anchor tests; added `_find_or_create_post_idempotent_with_null_source` (dedup via the partial unique index) and `_lookup_source_for_sidecar_returns_*` (existing-subscription + none cases). The existing `_find_or_create_post_idempotent` now also passes `artist_id` and asserts it. - 8 other test files updated: every direct `Post(...)` construction gains `artist_id=<artist>.id`. The `_seed_post` helper in `test_post_feed_service` looks up artist_id from the source row so callsites stay one-arg. ## Verification on deploy After alembic 0030 runs: - `SELECT COUNT(*) FROM source WHERE url LIKE 'sidecar:%'` → 0. - `SELECT COUNT(*) FROM post WHERE source_id IS NULL` → count of filesystem-imported posts (Dymkens + any other historical). - Every `post.artist_id` non-null; consistent with source.artist_id for source-bound rows. - Subscriptions tab: no Dymkens phantom row. - Artist detail → Posts/Gallery: Dymkens's content still reachable via Post.artist_id. - Provenance panel renders "filesystem import" chip for NULL-source posts; PostCard same. ## Out of scope - UI to manage/delete orphan NULL-source Posts. Data model is right; UI follows if operator wants it.	2026-06-01 14:17:52 -04:00
bvandeusen	94e7d20792	feat(downloads): Logs button on failing-sources rollup rows ... Operator-asked 2026-06-01: each row in the "X sources are failing" rollup needs a button to surface the last run's stdout/stderr/error without leaving the Downloads tab to find the matching event row. Wired: - `downloadsStore.loadLastForSource(sourceId)` two-steps via `GET /api/downloads?source_id=N&limit=1` (most recent event) then the existing `loadOne(id)` for the full detail payload. Returns null if no events exist (toast warns). - `FailingSourcesCard` row: new text button `Logs` with `mdi-text-box-search-outline`, per-row spinner via `logLoadingIds`, emits `view-logs` with the source record. - `DownloadsTab.onViewFailingLogs` is the handler — same `DownloadDetailModal` instance the event-row clicks use.	2026-06-01 10:04:22 -04:00
bvandeusen	fb605af959	fix(posts): anchored view scroll inherits artist_id/platform filters ... Operator-flagged 2026-06-01: clicking a Provenance post title from the view modal opens the post in the posts feed scoped to that artist (`/posts?post_id=N&artist_id=A`), but the older/newer infinite-scroll loaded UNFILTERED global posts instead of staying in the artist's stream. Root cause: the posts store's loadAround/loadOlder/loadNewer sent only `{around, cursor, direction}` — never the `artist_id` / `platform` filters. Backend `/api/posts` accepts them on every path (post_feed_service.scroll filters via `Source.artist_id`); the frontend just wasn't passing them. Fix: - `posts.js` `loadAround(postId, filters)` now takes the filter snapshot and stores it. `_aroundParams(extra)` mixes the active filters into around/cursor calls. - `PostsView.vue` `loadAroundAndAnchor` passes `artist_id` + `platform` from the route query.	2026-06-01 08:24:43 -04:00
bvandeusen	4c56cf121f	fix(modal): Esc closes from tag input; Provenance cards scroll at ~2.5 ... Two operator-flagged UX gaps from 2026-06-01: 1. **Esc trapped inside the tag-entry field** The autofocused TagAutocomplete input made Esc-to-close unreachable because ImageViewer's keydown handler bailed early on `isTextEntry(ev.target)` for every key including Escape. Now Escape closes the modal from text inputs too — except when a Vuetify overlay is open (FandomPicker, autocomplete dropdown, suggestion 3-dot menu), in which case that overlay's own Esc-handling fires instead of closing the whole modal mid-interaction. Detected via `.v-overlay--active`. Arrow keys still gate on isTextEntry so the tag input handles typing without navigating images. 2. **Provenance cards expanded the side panel unbounded** When an image had many ImageProvenance entries the cards stack pushed the Tags section below the fold. Wrapped the cards in a `.fc-prov__cards` container with max-height 270px (≈2.5 cards) and thin overflow scrollbar. Title stays anchored at top; Tags panel sits at a consistent position below regardless of provenance count.	2026-06-01 08:17:54 -04:00
bvandeusen	9564d073b9	fix(ci): POSIX-safe SHORT_SHA in build.yml (runner uses dash, not bash) ... `${GITHUB_SHA:0:7}` substring expansion is bash-only; the runner executes the step via dash/BusyBox sh and errored with `Bad substitution` (act_runner workflow shell, observed on the 65386f0 main-push run). Switched to `SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)` which works in both shells. Both build-web and build-ml tag-determination steps updated.	2026-06-01 07:53:33 -04:00
bvandeusen	f87a06a6bd	feat(modal): post title is the primary click (artist-scoped) + suggestion rows look like buttons ... Two operator-asked modal UX changes (Scribe plan #514): 1. **Post title click → artist-scoped posts feed** - The bold post title in ProvenancePanel is now the primary click target. Click navigates to /posts?post_id=N&artist_id=A; the PostsView already filters on `artist_id`, so the user lands in that creator's stream, not the global one. - The redundant "View post" link is removed. "Show description" stays as the only action link below the meta line. - Title is styled as a button-link: accent color, hover underline, focus ring for keyboard nav (family rule 24 — UI quality bar). 2. **Suggestion rows look like buttons** - SuggestionItem becomes a chip-card: visible border, hover/focus background, unified container for name + score + actions (operator's "nothing visual to unify the buttons to their object" complaint). - Accept is an explicit tonal pill button labeled "Accept" (operator's pick over whole-row-clickable). 3-dot menu retains alias/dismiss, now `variant="outlined"` so it reads as a button. - Score is a fixed-width monospace pill on the right. - "+ new" badge upgraded to a pill chip with accent border so it's visibly an annotation, not part of the tag name.	2026-06-01 02:21:59 -04:00
bvandeusen	5d284aae9f	fix(test): unpin general-threshold test from old 0.95 default (alembic 0029)	2026-06-01 02:18:54 -04:00
bvandeusen	af7b5c95e9	feat(modal): autofocus tag input, expand general suggestions, retire copyright/artist categories ... Four coupled operator-asked changes to the view modal (Scribe plan #509): 1. **Autofocus tag entry on modal open** — TagAutocomplete grabs focus in onMounted/nextTick so the caret is in the input the moment the modal renders. No click needed to start typing. 2. **General suggestions expanded by default** — SuggestionsPanel's general-category group now mounts with `:default-open="true"`. Operator can collapse if too noisy, but the v1 frame shows them. 3. **Lower general threshold default 0.95 → 0.50** — MLSettings. suggestion_threshold_general default matches character. Alembic 0029 also bumps the existing singleton row's value if it's still at the old 0.95. Operator can re-tune from Settings → ML. 4. **Retire `copyright` + `artist` as ML suggestion categories** — neither feeds a Tag.kind (`artist` retired in FC-2d-vii-c, never really existed as a copyright tag-kind). They were surfaced in the suggestions pipeline + threshold settings UI but had no follow- through. Drop from SURFACED_CATEGORIES, suggestions._threshold_for, ml_admin GET/PATCH allowlist, MLSettings columns (alembic 0029 drops the two columns), frontend CATEGORY_ORDER + CATEGORY_LABELS, SuggestionsPanel.peopleCats, AliasPickerDialog kind-check, and MLThresholdSliders rows. Out of scope (intentional): `tag_kind` Postgres enum still includes `artist` for historic Tag row queryability (per the model comment); no operator pain reported, no enum-shrink needed. Tests: - test_surfaced_categories asserts {character, general}, excludes artist + copyright. - test_threshold_for_artist_is_unsurfaced extended to cover copyright. - test_get_and_patch_settings asserts new 0.50 default and the absent artist + copyright keys in the GET payload.	2026-06-01 02:08:10 -04:00
bvandeusen	8de7ccd07d	build(ci): per-commit :c-<short_sha> tag on main-push per family rule #46 ... Adds an immutable per-commit docker tag to every main-push build: `git.fabledsword.com/bvandeusen/fabledcurator{,-ml}:c-<short_sha>`, alongside the existing floating `:main` + `:latest`. Implements the new family release-posture rule "Tags are milestones, not gates — commit-SHA images are the rollback unit" so rollback to any commit on main is `docker pull …:c-<sha>` with no release ceremony required. Behavior change summary: - main-push: was {:main, :latest} → now {:main, :latest, :c-<short_sha>} - tag-push (opt-in vYY.MM.DD only, no .N): unchanged - safety-net dev: unchanged No code changes; the rule is about how the tag list is constructed. Tag-push workflows stay as-is — vYY.MM.DD milestone cuts can still fire them when the operator wants a labeled checkpoint.	2026-06-01 01:28:55 -04:00
bvandeusen	d65f0b2091	feat(extension): probe shows current state before click; v1.0.6 ... Operator-asked 2026-05-31 (during sidecar synthetic anchor cleanup): "the add source/subscription button idea to the firefox extension so it can tell me if a source/artist is added or not and offer an option to add it if it isn't." Plan tracked in Scribe task #507. ## Backend - `ExtensionService.probe(url)` — read-only resolution. Reuses `_derive` for platform+slug, then 2 SELECTs. Returns one of: - `source_match` (exact (artist, platform, url) Source exists) - `artist_match` (artist exists, this URL isn't a Source yet; collapses the sidecar-synthetic-only case from v26.06.01.0) - `new` (neither exists) - `unknown_platform` (URL didn't match any artist-page regex) - `GET /api/extension/probe?url=...` route with `X-Extension-Key` auth posture matching `/quick-add-source`. Read-only, side-effect free. - 6 backend tests in tests/test_api_extension.py covering each state + auth + invalid URL. ## Extension - `api.js`: `probeSource(url)` mirroring `quickAddSource` shape. - `background.js`: `PROBE_SOURCE` + `OPEN_ARTIST_PAGE` handlers. The latter strips the `/api` suffix from configured `apiUrl` (placeholder format per options.html) and opens `${base}/artist/{slug}` in a new tab via `browser.tabs.create`. - `content-script.js`: probe-first render — on page-load and SPA navigation, asks the backend for the URL's state and renders the chip in the matching color/copy on FIRST paint instead of flashing generic "Add" and updating after. Click handler branches: `source_match` → OPEN_ARTIST_PAGE; `artist_match`/`new` → existing ADD_AS_SOURCE flow (then re-probes so the chip flips green immediately, no wait for next nav). - `content-script.css`: three state-color modifiers (--new, --artist-match, --source-match) on the FC parchment-on-slate palette. Sage for already-added, amber for artist-exists, accent orange for new. ## Versioning - `extension/manifest.json` + `extension/package.json` → 1.0.6. build.yml's sign-extension job will fire on push to main since no `ext-1.0.6` Forgejo/Gitea release exists yet — exercises the regenerated AMO keys end-to-end. ## Behavior on the sidecar-synthetic case Filesystem-imported "Dymkens"-style artist with only a sidecar synthetic Source: probe returns `artist_match` (not `new`), so the chip reads "+ Add Patreon source to Dymkens" rather than offering to recreate the artist. Clicking adds the real Source; existing `_source_for_sidecar` preference logic (v26.06.01.0) routes future gallery-dl Posts to the real one.	2026-06-01 00:41:17 -04:00
bvandeusen	66f19d67f5	fix(download): tier-gated = warning, race subprocess timeout, install yt-dlp ... Three coupled operator-reported pains from the 2026-05-31 download event audit: 1. `[patreon][warning] Not allowed to view post N` was bubbling up as an error event, bumping consecutive_failures and parking the source in "needs attention." The classifier's tier-gated branch was gated on `return_code in (1, 4)`. Gallery-dl returns a different exit code for mixed-failure runs (e.g. paywall warnings + a missing yt-dlp dep flipping the exit bits), so the branch never fired and the path fell through to UNKNOWN_ERROR. Widen the gate: when no source-level error fired AND tier-gated warnings are present, classify as TIER_LIMITED regardless of return code. 2. Knuxy event #38275 (2026-05-31) ran 30 min and finalized with "stranded by recovery sweep (no terminal status after time_limit)" + empty stdout/stderr. Root cause: subprocess.run timeout (900s) and Celery soft_time_limit (900s) raced; when Celery won, SIGKILL wiped the in-memory captured output and the DownloadEvent ended up empty-logged 18 minutes later when the sweep finalized it. Drop gallery-dl's default subprocess timeout to 870s — a 30s margin shy of Celery's soft limit — so subprocess.TimeoutExpired always wins the race and captures the partial stdout/stderr via the existing handler. 3. `[downloader.ytdl][error] Cannot import yt-dlp or youtube-dl` was firing on every video attachment, causing per-item download failures that masked legitimate tier-gated classification. Add yt-dlp>=2025.1 to requirements.txt. Once it's in the image, video posts download normally and the per-item failure noise disappears. Tests added: - pure tier-gated stderr with exit code 128 → TIER_LIMITED + success - mixed tier-gated + yt-dlp + per-item failures → still TIER_LIMITED	2026-05-31 23:30:39 -04:00
bvandeusen	6fc8ae3106	fix(subscriptions): hide sidecar synthetic Sources + prefer real on lookup ... Two coupled bugs surfaced 2026-05-31 by the Subscriptions UI showing "phantom" subscriptions like `sidecar:patreon:dpmaker`: 1. `SourceService.list()` returned every Source, no filter on URL. alembic 0022 (2026-05-26) consolidated old per-post-URL Sources into one canonical row per (artist, platform); when no real campaign URL was salvageable it rewrote the canonical to `sidecar:<plat>:<slug>` enabled=false as a disabled anchor. The UI then listed those anchors as if they were polls — disabled, but visible. Fix: `list()` excludes `url LIKE 'sidecar:%'` by default; `include_synthetic=True` opts back in for admin tooling. 2. `importer._source_for_sidecar` picked the lowest-id Source for (artist, platform). When alembic 0022 had rewritten a per-post row into a synthetic anchor (lower id) AND the operator later added the real subscription (higher id), every gallery-dl download silently attached its Post to the SYNTHETIC instead of the real Source. Fix: prefer a non-`sidecar:%` URL when one exists; fall back to the synthetic; only create a new synthetic when nothing exists for (artist, platform). alembic 0028 is the data half: for every (artist, platform) with both a synthetic AND a real Source, pre-merge Post+ImageProvenance collisions on the canonical, bulk-repoint Posts/ImageProvenance/ DownloadEvent.source_id onto the real Source, and delete the synthetic. Lone synthetics (no real twin) are left intact — they anchor real imported content the operator may still want; the list-filter hides them so they no longer surface as phantoms.	2026-05-31 23:08:38 -04:00