d6c15f4ea0478bc009db41a07a9ab94f29b5a006
632 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
d6c15f4ea0 |
test(patreon): fake gdl needs real _rate_limit for native pacing — #703 step 1 fix
_run_patreon_ingester reads self.gdl._rate_limit for the native pacing config (max(0.5, rate_limit/4)); the MagicMock fake gdl broke the arithmetic. Give it real _rate_limit/_validate_files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3b2f7a41c3 |
feat(patreon): ingester rate-limit resilience — #703 step 1
A single 429 mid-walk used to fail the run → RATE_LIMITED → platform-wide
cooldown → every Patreon source dark ("testing dead"). The native path also
ignored the operator's existing politeness setting. Fixed both:
- Pacing (avoid 429s): honor download_rate_limit_seconds (gallery-dl's
`rate_limit`, read off self.gdl) as a pre-download sleep on real media
downloads only (skips don't pace); pace /api/posts page fetches with the
per-source sleep_request override, defaulting to max(0.5, rate_limit/4) —
the same API-pacing default gallery-dl used for `sleep-request`.
- 429 backoff (ride out transient limits): PatreonClient._fetch retries a
429 with backoff (honor Retry-After, else exponential 2·2^(n-1), capped
30s, ≤3 tries); only a PERSISTENT 429 propagates as terminal
RATE_LIMITED. Light 2-retry on a media-GET 429 too.
Threaded via PatreonIngester(rate_limit=, request_sleep=) →
PatreonClient/PatreonDownloader; download_service sources them. Injected
test client/downloader are unaffected (carry their own pacing).
Tests mock time.sleep (no real sleeping): retry-then-success, persistent
429 raises after N, Retry-After honored, request_sleep paces, media pacing
per real download, skips don't pace, media 429 retried.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
218bfebb92 |
feat(downloads): native Patreon verify + uniform backend dispatch (plan #697)
The credential Verify button still ran gallery-dl --simulate for Patreon after the cutover — testing the wrong path (and prone to the vanity "Failed to extract campaign ID" the native resolver fixes). Wire it to the native ingester, behind a DRY dispatch so callers never branch on platform. - services/download_backends.py (new): the ONE place that knows which platforms are native vs gallery-dl. `uses_native_ingester(platform)` is the shared predicate; `verify_source_credential(...)` is the uniform probe (same (ok|None, message) contract for both backends). As a platform migrates, it moves into NATIVE_INGESTER_PLATFORMS here and BOTH download routing and verify switch together. - PatreonClient.verify_auth(campaign_id): one authenticated /api/posts fetch → True (valid) / False (401/403/HTML-login) / None (drift or network — inconclusive, not a credential verdict). - patreon_ingester.verify_patreon_credential(): resolve campaign id, then verify_auth — the verify counterpart to the download path. - patreon_resolver.resolve_campaign_id_for_source(): extracted the override / id:-URL / vanity resolution into ONE helper now shared by the download ingester and verify (download_service no longer carries its own copy + regex; −`import re`). - download_service: routes on uses_native_ingester() instead of inline `== "patreon"` (3 sites); uses the shared resolver. - api/credentials: calls verify_source_credential — no platform branch. Tests: verify_auth mapping, resolve_campaign_id_for_source (override/id:/ vanity/none), the dispatch predicate, verify_patreon_credential glue, credentials endpoint proves Patreon uses the native path (gallery-dl verify asserted not-called); repointed the gallery-dl verify test to subscribestar. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
ec43e823e1 |
feat(patreon): recovery UI + gallery-dl cutover — build step 5 (plan #697)
Final step of the native Patreon ingester: a first-class Recovery action, and removal of the now-dead gallery-dl Patreon path. Recovery (rules #23/#24/#27 — full product, with UI): - source_service.start_recovery arms the #693 backfill state machine PLUS `_backfill_bypass_seen`, flipping download mode to recovery (bypass the seen-ledger to re-fetch dropped-and-deleted near-dups and re-evaluate under the current pHash threshold). Stop via the shared stop_backfill. - SourceRecord exposes backfill_bypass_seen; POST /sources/{id}/backfill gains action="recover". - Frontend: Recovery button (Patreon-only, mdi-backup-restore) on SourceRow + SourceCard; the running badge labels "Recovering (N)" vs "Backfilling (N)"; the Stop tooltip says "Stop recovery". sources.js recoverSource + SubscriptionsTab onRecover. Cutover (rule #22 — no legacy): - gallery_dl: removed PLATFORM_DEFAULTS["patreon"], the patreon files/cursor branch in _build_config_for_source, and the patreon/Mux yt-dlp Referer/Origin block (was patreon-specific and wrongly tagged the other platforms' yt-dlp fetches; native ingester owns it now). - download_service: removed the dead campaign-id-retry helpers (_looks_like_campaign_id_failure / _CAMPAIGN_ID_FAILURE_PATTERN) and _effective_url. Vanity→campaign resolution + resume_cursor + the cursor/PARTIAL lifecycle stay — they serve the native ingester. Tests: removed the two obsolete patreon-gallery-dl config tests (yt-dlp Referer, resume-cursor); repointed the generic skip-value config tests to subscribestar; added start_recovery + recover-endpoint coverage (backfill_bypass_seen). gallery-dl stays for the other 5 platforms. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
682beafbc5 |
feat(patreon): drift detection + error categorization — build step 4 (plan #697)
Typed, loud failure mapping for the native Patreon ingester so a changed
API shape or expired auth never silently zero-downloads as "success".
- New ErrorType.API_DRIFT (free varchar error_type col → no migration):
distinct from auth so the operator knows the fix is updating the
ingester, not rotating cookies.
- patreon_client: PatreonAPIError carries status_code; new PatreonAuthError
for 401/403 + HTML-login/non-JSON bodies (reclassified from drift —
expired-session is auth, actionable as "rotate cookies").
- patreon_ingester._failure_result maps: PatreonAuthError→AUTH_ERROR,
PatreonDriftError→API_DRIFT ("Patreon API changed — ingester needs
update"), HTTP 429→RATE_LIMITED, 404→NOT_FOUND, other HTTP→HTTP_ERROR,
transport→NETWORK_ERROR. (429 thus drives the platform cooldown.)
- FailingSourcesCard: api_drift chip (red) + hint.
Contract test (new test_patreon_contract.py): the recorded /api/posts
fixture must parse end-to-end (no drift, 5 media across 4 posts) AND the
request params must still carry every field the parser depends on
(file_name, image_urls/download_url, images/attachments_media/media
includes, content/post_file/image post fields) — a trim of either trips a
red build. Plus client HTTP-status classification tests and ingester
error-type mapping tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
96c30eba13 |
feat(patreon): phase-2 ingester integration — build step 3 (plan #697)
Branch download_service phase 2 by platform: Patreon now routes to the
native PatreonIngester (zero per-file HEADs, native cursor/resume, loud
drift detection) instead of gallery-dl; the other 5 platforms are
unchanged. The ingester returns a DownloadResult-shaped object so phase 1
(DB setup) and phase 3 (import → pHash → thumbs → ML) are untouched.
Three modes wired from config_overrides state:
- tick: skip seen (tier-1 ledger + tier-2 disk), early-out after N
contiguous already-have-it items.
- backfill: full-history time-boxed chunk, cursor checkpoint via
gallery-dl-style "Cursor: <token>" lines in stdout (reuses the #693
lifecycle + parse_last_cursor verbatim).
- recovery: backfill that BYPASSES the tier-1 seen-ledger so
dropped-and-deleted near-dups get re-fetched and re-evaluated under
the current pHash threshold. Rides the #693 state machine via a
_backfill_bypass_seen flag, cleared on completion / stop.
The seen-ledger uses short-lived sync sessions (injected sessionmaker),
never held across the walk (avoids the connection-reaping trap). Campaign
id resolves from override, an id: URL, or a vanity lookup; unresolvable =
loud NOT_FOUND, never a silent empty success.
Tests: new test_patreon_ingester.py (modes, ledger skip/idempotency,
budget→PARTIAL, recovery bypass, tier-2 disk, drift). The patreon-oriented
download_service tests now drive the ingester branch via a stub; the
gallery-dl campaign-retry test is replaced by resolution/caching coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
2ec7d86a3b |
feat(patreon): native media downloader — ingester build step 2b (plan #697)
PatreonDownloader.download_post: writes resolved MediaItems to gallery-dl's exact on-disk layout (<slug>/patreon/<date>_<id>_<title40>/<NN>_<file>) + a sidecar the importer's find_sidecar/parse_sidecar consume unchanged. Two-tier skip (injected seen predicate, then disk). Streamed GET (.part→rename) + file_validator quarantine; Mux/m3u8 video shells out to yt-dlp with Patreon Referer/Origin. Pure (no DB) — ledger + orchestration land in step 3. Unit tests stub the session + yt-dlp seams (no network/subprocess). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6222928746 |
feat(patreon): seen-ledger table + model — ingester build step 2a (plan #697)
patreon_seen_media(source_id, filehash, post_id, seen_at), UNIQUE(source_id, filehash) — our own queryable replacement for gallery-dl's archive.sqlite3. Routine walks skip seen media; recovery mode bypasses the ledger. filehash is a 32-hex CDN MD5 or a video:<post>:<media> sentinel (String(128)). alembic 0037 (← 0036). Integration test covers dedup + savepoint recovery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1bdaa04aa2 |
test(patreon): fix self-contradictory attachment/postfile dedup assertion
The fixture gives the attachment and post_file the same filehash, so they correctly collapse to one item; the test asserted both survival and collapse. Rewrite to verify the cross-kind dedup (postfile kind covered by video test). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1c2dc7659a |
feat(patreon): native JSON-API client — ingester build step 1 (plan #697)
PatreonClient: cookie-auth requests session, /api/posts cursor pagination, JSON:API included flattening, per-post media extraction (images/image_large/ attachments/postfile/content) with filehash dedup, loud drift detection. Zero per-file HEADs — every media URL+file_name comes from the API. Not yet wired into download_service (later step). Pure-parsing unit tests + fixture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
618dafde85 |
feat(subscriptions): smarter-backfill UI — Start/Stop + state badge
Plan #693 (frontend). Backend landed in
|
||
|
bvandeusen
|
96fffaff64 |
feat(download): smarter backfill — time-boxed chunks, run-until-done (backend)
Plan #693. Large-catalog backfill (Anduo) no longer sprints to the timeout wall and dies as an error each run. Builds on the cursor checkpoint (#689). - Time-boxed chunks: BACKFILL_TIMEOUT_SECONDS(1170)→BACKFILL_CHUNK_SECONDS(600), far under the 1350 soft limit. Hitting it = normal chunk boundary (the TimeoutExpired path already captures partial output + the cursor), not a near-wall death. - Run-until-done state machine driven by config_overrides[_backfill_state] (running/complete/stalled). A running backfill auto-continues in chunks across ticks until gallery-dl exits cleanly (rc=0 = reached the bottom → 'complete'); a safety-cap (BACKFILL_MAX_CHUNKS=200) + the #689 stall-guard pause a pathological walk as 'stalled'. Replaces the N-runs counter (backfill_runs_remaining repurposed as the cap countdown). - Progress, not error: a chunk that timed out but advanced (cursor moved and/or files written) is reclassified TIMEOUT→PARTIAL (status 'ok'). - Retry storm tamed: gallery-dl retries 3→2, downloader timeout 120→60s, so one stuck CDN file fails in ~1-2 min not ~10 (Anduo #40838). - API: POST /sources/{id}/backfill now takes {action: start|stop}; service start_backfill/stop_backfill; new enabled sources auto-arm run-until-done; source dict exposes backfill_state + backfill_chunks. Frontend (Start/Stop control + state badge) lands in the next push. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
add1c1ad14 |
fix(modal): mobile — no tag autofocus + sticky image over scrolling panel
Operator-flagged 2026-06-05 (mobile): - TagAutocomplete no longer autofocuses on the ≤900px stacked layout — focusing popped the soft keyboard, shrinking the viewport and shoving the pinned image + nav/close controls out of view. matchMedia gate (safe on plain HTTP); desktop autofocus unchanged. - ImageViewer stacked layout: the body now scrolls and the media pane is sticky (height:55vh, top:0), so the image + prev/next/close + integrity badge stay pinned while the metadata panel scrolls beneath. Replaces the old fixed image + 40vh internally-scrolled panel that could push the controls off. Prev/next re-centered over the image band; opaque obsidian bg so the scrolling panel doesn't bleed through. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
593f65c9cc |
feat(download): cursor-paged Patreon backfill for large catalogs
Large Patreon creators (Anduo: weekly 50-120-image Reports back months = thousands of files) couldn't backfill: each run re-walked newest→oldest from the top, and gallery-dl's polite ~0.75s/request HEAD walk alone exceeded the 1170s subprocess budget, so the run died during enumeration with 0 files written and NO forward progress — re-stranding every time (event #40411). Checkpoint gallery-dl's pagination cursor so each backfill window advances the frontier: - gallery_dl.py: SourceConfig.resume_cursor; _build_config_for_source sets extractor.patreon.cursor=<resume> (PLATFORM_DEFAULTS leave log-only True for a fresh run); parse_last_cursor() pulls the last emitted 'Cursor: <token>' from stdout+stderr — survives a timed-out run since the TimeoutExpired path returns partial output. - download_service.py: phase2 stays in BACKFILL mode while a cursor is pending (even after the run budget drains) and threads resume_cursor; _apply_backfill_lifecycle() checkpoints the advancing cursor each non-completing run, completes on a clean rc=0 finish (walk reached bottom), and a stuck-guard clears the cursor after 2 non-advancing runs so a wedged walk can't re-strand forever. patreon-only (sole platform with a resumable cursor); other platforms keep the simple counter semantics. Cursor state lives in config_overrides JSON (patreon_campaign_id precedent) — no migration. Time-budget ladder (1170/1350/1500) unchanged. Plan #689. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
86efbf7f2c |
fix(modal): kebab menus open via explicit v-model, not activator click
Operator-confirmed on a fresh build: both the tag-chip and suggestion
kebabs still never opened. The prior
|
||
|
bvandeusen
|
3a0cca5aca |
fix(tags): allow creating a character with no fandom
Not all characters belong to a fandom (original characters, unsorted). The create flow forced every new character through FandomPicker, whose only outcomes were 'Use this fandom' (disabled until one is picked) or Cancel (which aborts the whole creation) — there was no way to confirm a character with no fandom. - FandomPicker: add a 'No fandom' action that emits confirm(null). - TagAutocomplete.onFandomChosen: pass fandom_id: null when null is emitted. Backend already supported this end to end (Tag.fandom_id nullable, the CHECK only forbids fandom_id on non-character kinds, tag_service find_or_create defaults fandom_id=None, API reads body.get). A fandom can still be assigned later from the chip kebab's 'Set fandom…'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
a5b3702863 |
feat(settings): surface the near-duplicate (pHash) control + reorder import tab
The phash_threshold knob (controls whether edits/variants of an image are dropped as near-duplicates on import) was buried at the bottom of the import filters form and labelled opaquely, so it read as 'missing'. Hoist it to the TOP of the form as a 'Near-duplicate sensitivity' section: a labelled slider (Exact / Strict / Default / Loose stops, 0-16) for the gist + the precise number field, both bound to phash_threshold, with copy that says plainly to lower it if variants are being dropped. Also swap the import-tab order to filters → trigger → recent-tasks (filters on top per operator); the task list stays directly under the trigger for hit/miss feedback adjacency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
509a7958cf |
feat(posts): images open the modal; only text expands in place
Post cards no longer expand the whole card on click (the old two-click path
to the images). The card is compact-only now:
- Hero / rail thumbs / the +N tile are buttons that open the post-scoped image
modal (modal.open(id, { postImageIds })) so you view big + arrow through ALL
the post's images. The feed caps thumbnails at 6, so for posts with more we
lazily getPostFull to get the complete id list; +N opens at the first hidden
image.
- The description is the ONLY in-place expansion: a Show more / Show less toggle
shown only when the text is actually truncated (server description_truncated
flag OR a measured CSS-clamp overflow, ResizeObserver-guarded). Expanding
loads description_full when server-truncated and renders it unclamped.
- Attachments: download chips now render inline in the compact card (the feed
already carries download_url), since the expanded view is gone.
Removes PostImageGrid.vue (the mosaic, now unused). Tests cover show-more
visibility + image-click opening the scoped modal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
5a6a95682d |
fix(cleanup): library scans survive navigation, reconnect on return
The transparency / single-color audit cards held the run + poll timer in local component state, so navigating away destroyed both and onMounted never reconnected — the Celery scan kept running and writing LibraryAuditRun, but the UI forgot it. Now each card, on mount, fetches its rule's latest run (GET /api/cleanup/audit?rule=<rule>&limit=1) and rehydrates: shows progress + resumes polling if still running, or shows the completed result (ready/applied/ error) so the operator can act on it after returning. Adds the ?rule= filter to the audit-history endpoint + cleanup store latestAuditForRule(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
91b0145bc8 |
feat(tags): 'Reset content tagging' admin action
Wipe every general + character tag so the operator can re-tag from scratch via
the Camie auto-suggest, while PRESERVING fandoms, series (+ series_page order),
and each image's stored tagger_predictions (so suggestions repopulate
immediately). One set-based DELETE FROM tag WHERE kind IN ('general','character')
— the five tag-referencing tables all cascade, so applications + aliases +
allowlist + rejections + centroids clear automatically; series tags aren't
deleted so series survive; Tag.fandom_id is SET NULL so fandoms are untouched.
Reuses the established dry-run-preview -> confirm pattern: cleanup_service.
reset_content_tagging() + POST /api/admin/tags/reset-content +
TagMaintenanceCard section with a backup-first warning and a red confirm
showing exact counts (tags by kind + image applications). Irreversible except
via DB backup restore; the wipe only fires when the operator confirms.
Tests: service dry-run counts + live delete preserves fandom/series/series_page
while content tags + their image_tag cascade away; API dry-run wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
26e47a86cb |
fix(gallery): render similar-mode results (flat list when no date groups)
The 'See all similar' takeover fetched /api/gallery/similar fine (200, ~100 results) but the grid showed nothing: GalleryGrid renders ONLY by iterating store.dateGroups, and similar-mode returns date_groups=[] (results are ranked by cosine distance, not chronological). Zero groups → zero tiles despite store.images being full. Add a flat fallback: when there are no date groups but images exist, render them as one ungrouped list in ranked order (no date headers). The modal Related strip was unaffected (it renders its images directly). Test locks both the flat and grouped paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
928e3037f0 |
fix(ui): purpose-built mobile layout for subscriptions hub
Vuetify's auto card-stack was too verbose (one subscription filled the whole phone screen) and the expanded sources still needed lateral scroll. Replace it below 600px (useDisplay) with a custom compact-card list: each subscription is a 2-line card (name + health + expand chevron, then platform chips + sources count + last activity) so several fit per screen. Expanding shows the action row + each source as a STACKED SourceCard (new) — platform/url/enabled/last/ next/errors/actions laid out vertically, no horizontal scroll. The mobile cards drive the same selected/expanded key arrays as the desktop data-table, so selection and bulk actions are unchanged. Desktop keeps the v-data-table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
b08b12eb8f |
fix(ui): subscriptions table → card layout on mobile
The subscriptions v-data-table (select + expand + 6 cols + a nested 8-col sources sub-table) horizontally-scrolled on phones. Set mobile-breakpoint=600 so Vuetify stacks each subscription row into a label:value card below 600px; the custom item slots (platform chips, health dot, action buttons) render as card rows. The expanded sources detail reclaims its desktop indent on mobile and keeps its own horizontal scroll for the wide source columns. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4fd6d4cc29 |
fix(ui): mobile pass 2 — Posts & Subscriptions filter bars
PostsFilterBar didn't wrap and its artist/platform fields had inline min-widths (240/180px) a media query can't override → horizontal overflow on phones. Moved widths to classes, added flex-wrap, and <600px each field takes a full-width row. SubscriptionsTab's status/search inline max-widths likewise moved to classes; <600px they go full-width and the v-spacer is dropped so the search isn't shoved around. Verified as already-fine (sweep false positives, no change): PostCard (default body is a stacked column; only goes row at container >=800px), SeriesReaderView (already has a <=768px block: nav drawer 150px, quick-nav stacks). The subscriptions v-data-table scrolls horizontally within its own wrapper, so it doesn't widen the page — a true mobile card layout is a larger follow-up if wanted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
304e8aa878 |
fix(ui): mobile responsiveness — nav hamburger + primary-path fixes
The top nav packed brand + health + pipeline chip + ~7 inline links + an action slot into one flex row, colliding/overflowing on phones (operator: 'almost unusable'). Below 768px the links now fold into a hamburger v-menu; below 480px the brand text hides (glyph still brands). Plus the primary browsing path: - BulkEditorPanel: fixed 320px -> min(320px, 90vw) so it can't swallow the screen. - GalleryFilterBar: <600px gives search its own full-width row (its 200px min-width was jamming the wrapping bar); sort grows. - GalleryFacetPanel: <480px wraps groups + lets the side-by-side date inputs grow full-width. - ArtistsView grid: minmax(min(440px,100%),1fr) so a card never overflows (single column on phones). - GalleryView: hide the year/month timeline strip <600px. ImageViewer already stacks its side panel below the image <900px (left as-is). Secondary surfaces (Posts/Subscriptions filter bars, SubscriptionsTab table, SeriesReader, PostCard) still need a mobile pass — follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
0497394710 |
feat(ui): double showcase cadence + filter bar matches TopNav frost
- Showcase reveal cadence 80ms -> 160ms (slower, more deliberate one-at-a-time cascade per operator). Bump showcase.spec timer advances to cover 60x160ms. - Gallery filter bar now uses the EXACT gradiated-obsidian frost + blur as TopNav (was a flat rgba(...,0.55) block), so the two read as one continuous piece of chrome with images visibly scrolling under both; the nav's transparent bottom edge against the bar's opaque top leaves a faint seam that separates them at the very top of scroll. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
21a73cd1dc |
feat(gallery): visual 'more like this' UI (Phase 3 frontend)
Modal 'Related' strip (RelatedStrip.vue) — top-12 similar thumbs, fetched on its own DEFERRED, single-flighted path (200ms after the modal is up) so it never blocks or slows the modal; collapses silently on empty/slow/error and is hidden when the image has no embedding (has_embedding flag). 'See all similar' closes the modal and navigates the gallery to ?similar_to=<id>. Gallery store: similar_to filter field + loadSimilar() (ranked, hasMore=false, no timeline); applyFilterFromQuery routes similar-mode to /similar with the scope filters composed; cloneFilter/filterToQuery carry similar_to. Filter bar: clearable 'Similar to #id' chip, sort hidden in similar-mode; timeline sidebar hidden too. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
79cd1234e2 |
feat(gallery): visual 'more like this' search (Phase 3 backend)
GalleryService.similar() ranks images by pgvector cosine distance to a source image's precomputed SigLIP embedding — no query-time ML inference. Composes with the Phase-1/2 scope filters (AND) but replaces the date sort (always nearest-first, bounded top-N, no cursor). Returns None for a missing source (→404), [] for a source with no embedding (video / pending ML); excludes self and NULL-embedding rows. New GET /api/gallery/similar?similar_to=<id>&limit=N. Image-detail payload gains has_embedding so the UI can hide the surface. Alembic 0036 adds an HNSW vector_cosine_ops index on siglip_embedding (1152<2000 dims) so the search is sub-50ms ANN instead of a full scan; one-time ~30-60s build over existing embeddings on deploy. Shared _gallery_images/_image_json helpers de-dup the scroll/similar builders. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3f6ea601f8 |
perf(ci): collapse the 3 integration shards into one job
With fsync-off the whole integration suite runs in ~45s (was ~13min across shards), so the 3-way split only triplicated the ~2min fixed overhead (container + install + migrate) and consumed 3 of 6 runner slots for no wall-clock gain. Merge intapi/intimp/intcore into one `integration` job: spin up once, install once, migrate once, run `pytest -m integration` over the whole suite. Frees 2 runner slots (6 jobs -> 4) and drops ~140 lines of near-duplicate YAML. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6a25db4b8b |
perf(ci): relax Postgres durability in integration shards (fsync off)
Option 1 (pooling the teardown connection) left teardowns at ~1.5-2s/test, so the cost is the per-test TRUNCATE's commit forcing an fsync, not the connect handshake. Each shard now ALTER SYSTEM SETs fsync/synchronous_commit/ full_page_writes off + pg_reload_conf() right after deps install, before alembic — sighup/user-context GUCs apply with no restart. The DB is ephemeral (rebuilt per run) so fsync-off is safe; the step is non-fatal so a perms surprise can't red a shard. Speeds up every test's commit (setup inserts + the teardown TRUNCATE), stacking on the pooled engine from the prior commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c802b26406 |
perf(ci): reuse a session-scoped engine for the per-test DB reset
The autouse integration teardown created a fresh SQLAlchemy engine + Postgres connection for EVERY test, then disposed it — --durations showed the 15 slowest ops in both long shards were all ~1.5-2s teardowns (the connect+SCRAM handshake, not test logic). Hoist the truncate engine to a session-scoped, pool_pre_ping'd fixture so the pooled connection is reused across teardowns; the TRUNCATE+restore still runs per test, so isolation is unchanged. Lazy create_engine means the no-DB unit job instantiates but never connects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3a4270e6be |
fix(showcase): reveal each tile only once its image is fully decoded
The buffered cascade revealed tiles on an 80ms timer regardless of image load, so the flip-in animation played on a gray placeholder and the thumbnail popped in afterward. Worse, MasonryGrid ALSO applied a per-index animation-delay (index×70ms) that compounded on top of the insert cadence, so the cascade visibly dragged and desynced as it grew. Now the producer preloads each queued thumbnail (decode pipelined ahead) and the consumer awaits that decode before pushing the item — every tile animates in fully loaded, strictly one at a time. Drop the compounding CSS stagger; the store's one-item-at-a-time push is the sole pacer, so each tile animates the instant it mounts. New utils/preloadImage.js (load+decode+timeout gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
ae569c0f9a |
fix(gallery): ruff C408 (dict literal) + panel auto-open on deep-link
Rewrite facets() common/plat_scope as dict literals (C408). Open the refine panel via a watch on hasRefineFilters rather than reading filter state at bar-setup time — the parent applies the URL query in its onMounted, after the bar child has set up, so the initial read was always the default (empty) state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1adc47f59c |
feat(gallery): faceted refine panel UI (Phase 2 frontend)
Add a 'Refine ▾' toggle to the gallery filter bar that expands a full-width GalleryFacetPanel below it, inside the same sticky hazey chrome. The panel offers platform chips (with live counts + a 'No platform' unsourced bucket), two count-badged curation-flag toggles (Untagged / No artist), and a from/to date range bounded by the facet min/max. Store gains the platform/untagged/no_artist/date_from/date_to filter params (URL-mirrored, AND-composed) and a panel-gated, single-flighted loadFacets() that fetches /api/gallery/facets scoped to the active filter. Shared cloneFilter/filterToQuery helpers keep the bar and panel writing one URL format. The panel auto-opens on deep-link when refine filters are present and refetches counts (debounced) on every filter change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
9fe534139a |
feat(gallery): faceted filter params + /facets counts endpoint (Phase 2 backend)
Extend the composable gallery filter with platform / untagged / no_artist / date_from / date_to, AND-composed with the existing tag/artist/media/sort params and threaded through scroll, timeline, and jump_cursor. Add GalleryService.facets() + GET /api/gallery/facets returning live counts scoped to the current filter with per-group minus-self semantics: platform counts (COUNT(DISTINCT image) incl. a null unsourced bucket), curation-flag counts (untagged / no_artist), and effective_date min/max bounds. The UNSOURCED_PLATFORM sentinel makes filesystem-imported content reachable via the platform facet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
16e0268da7 |
fix(modal): next arrow clears metadata panel + arrows work in empty tag input
Image viewer (#609): - The next (▶) arrow was offset from the viewport edge (right:16px) so it floated over the 320px metadata side panel. Offset it off a shared --fc-side-w var so it sits at the image's right edge instead; full-width again below 900px when the panel stacks under the image. - Arrow nav was fully disabled whenever a text field was focused. Now it yields to the caret ONLY when the field has text; an empty tag-entry field still navigates ←/→. Extracted to utils/textEntry.js (arrowNavAllowed). ESC behaviour unchanged (already closes the modal, overlay-aware). Test: arrowNavAllowed — empty/non-text → navigate, text present → don't. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
1f4ce8513b |
style: ruff I001 — keep _sync_engine as-import on its own line (combine-as-imports=false)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5a116ca9d0 |
style: ruff I001 — aliased _sync_session_factory sorts before get_sync_engine
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
ef3ee5aceb |
feat(maintenance): DB maintenance UI card + fix ruff I001
- Settings → Maintenance gains a "Database maintenance" card: a "Run VACUUM ANALYZE now" button (enqueues the maintenance task) plus a per-table bloat readout (live/dead/dead%/last vacuum) from /api/admin/maintenance/db-stats. - dbMaintenance store (loadStats / runVacuum) + test. - Fix ruff I001: combine the two _sync_engine imports onto one line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
914033db29 |
feat(maintenance): scheduled + manual DB VACUUM ANALYZE + bloat readout
The TABLESAMPLE showcase reads physical blocks (bloat-sensitive), and the periodic prune/backfill/recovery tasks churn dead tuples faster than autovacuum always keeps up — so explicit maintenance earns its keep here. - tasks.maintenance.vacuum_analyze: VACUUM (ANALYZE) over high-churn tables (VACUUM_TABLES) on an AUTOCOMMIT connection (VACUUM can't run in a txn). Scheduled weekly via Beat; also operator-triggerable. - _sync_engine.get_sync_engine(): expose the process engine for the autocommit connection. - GET /api/admin/maintenance/db-stats: per-table n_live/n_dead/dead_pct + last (auto)vacuum/analyze from pg_stat_user_tables — visibility, not a black box. - POST /api/admin/maintenance/vacuum: enqueue the task on demand. Tests: vacuum task runs + reports tables; db-stats shape; trigger queues. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d495605c12 |
style(gallery): hazey filter bar attached to the TopNav
- Filter bar gets the same obsidian translucent + backdrop-blur as the TopNav so the two read as one piece of chrome. - margin-top:-8px cancels the v-container's pt-2 so the bar sits flush at 64px even at scroll 0 — fixes the gap/separation when scrolled to top. - Inputs/toggles get a more-opaque backing so they stay legible on the haze. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
76d8ad42a8 |
fix(showcase): buffered producer/consumer for a steady cascade
The cascade "burped" — chunks appeared unevenly — because the old pipeline coupled display to fetch timing: it trickled each batch right after its fetch and assumed the next round-trip would land inside the ~240ms trickle window. When a fetch ran long (TABLESAMPLE hits random, sometimes-cold blocks; RTT jitter) the animation starved, then a clump burst in. Decouple the two: - Producer (_fill) races ahead fetching batches into a buffer up to a target depth, refilling when it dips below BUFFER_MIN. - Consumer (_drain) reveals one item every CADENCE_MS regardless of when fetches land; it only waits if the buffer genuinely starves. A small PRIME buffer precedes the drain so it doesn't starve at the front; the buffer (BUFFER_MIN×CADENCE runway) absorbs per-fetch jitter so images appear at an even pace. Public store API (loadInitial/shuffle/fetchPage/ images/loading/hasMore/isEmpty) unchanged — ShowcaseView/MasonryGrid need no change. Test (fake timers): fire-order + dedup, one-item-per-cadence rate limit, empty-library flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6d630d13d6 |
feat(gallery): pinned filter bar (Phase 1)
Gallery now has in-view filtering, styled like the app's sticky v-tabs
chrome (pinned at top:64px under TopNav).
- GalleryFilterBar: combined tag+artist autocomplete (searches
/api/tags + /api/artists), closable filter chips (multi-tag AND),
media toggle (All/Images/Videos), Newest/Oldest sort, Clear. Writes all
state to the URL via router.push.
- gallery store: filter is now { tag_ids, artist_id, media_type, sort,
post_id }; applyFilterFromQuery makes the URL the single source of truth
(deep-linkable, back-button works); chip labels resolved by id or
pre-noted on pick. Replaces the standalone tag chip + setTag/PostFilter.
- GalleryView: renders the bar (hidden in post-detail), syncs route.query
→ store on mount + every query change.
Also untracks the transient .claude/scheduled_tasks.lock committed in
|
||
|
bvandeusen
|
3f30327fa5 |
feat(gallery): composable scroll filter (multi-tag AND, media, sort)
Phase 1 backend for the gallery filter bar. Extends scroll/timeline/jump
from a single mutually-exclusive filter to a composable one:
- tag_ids: image must carry ALL of them (one correlated EXISTS per tag —
AND, no row multiplication), replacing the single-tag JOIN.
- artist_id composes with tags; media_type ('image'|'video') narrows by
mime; post_id stays the exclusive post-detail path.
- sort ('newest'|'oldest') flips the effective_date/id cursor comparison
and ordering; the cursor value is unchanged (direction comes from the
request). jump_cursor honors sort too.
- Shared _apply_scope helper applied across scroll/timeline/jump so the
timeline sidebar reflects the filtered set. API _parse_filters parses
tag_id (comma list), artist_id, media, sort.
Tests: multi-tag AND, media filter, sort reversal (service + API);
post_id-excludes-others; single tag_id back-compat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
4f9464d215 |
feat(gallery,tags): clear active filters
Two gaps where a filter couldn't be removed:
- Gallery: a tag_id filter (from clicking a tag) had no indicator or clear
control — only post_id did (PostInfoHeader). Add an "Tag: <name> ✕" chip
that clears the filter by dropping tag_id from the URL. New lightweight
GET /api/tags/<id> resolves the name; the store fetches it on filter set.
- Tags view: the kind chip-group used mandatory="false" — a STRING ("false"
is truthy in JS), which made the group mandatory so the active kind chip
couldn't be deselected. Fixed to :mandatory="false" so the filter clears.
Tests: GET /tags/<id> shape + 404; gallery store resolves filterTagName.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
e678d1dfdf |
feat(tags): fandom-edit UI in tags directory + image modal
Adds the missing UI to change a character tag's fandom, in both places:
- FandomSetDialog (shared): pick an existing fandom, create a new one, or
clear it; on a name collision in the target fandom it surfaces a merge
confirmation and resolves via setFandom(merge:true). Reuses the tags
store's fandom cache.
- TagCard kebab gains "Set fandom…" for character tags (→ TagsView opens
the dialog, reloads on success).
- TagPanel chip kebab gains "Set fandom…" for character tags (→ reloads the
modal's tag list on success).
- tags store: setFandom(tagId, fandomId, {merge}) action + test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
d9ab6e15c6 |
feat(tags): edit a character tag's fandom (backend)
No way existed to change which fandom a character tag belongs to after creation — PATCH /tags/<id> only renamed. - TagService.set_fandom(tag_id, fandom_id, merge=False): set / change / clear (fandom_id=None) a character's fandom, with the same validation as find_or_create. On a name collision in the target fandom it raises TagMergeConflict (→ 409, same shape as rename); merge=True resolves it by merging this tag INTO the existing character. - Extract _do_merge(source, target) from merge() so set_fandom can perform the deliberate CROSS-fandom merge the public merge() validation forbids. - PATCH /tags/<id> now accepts optional fandom_id (+ merge flag) alongside name, and returns fandom_id. Tests: set/change/clear, non-character + bad-ref rejection, collision raises, merge resolves; API set/clear + collision→merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
e05e0b9f37 |
perf(gallery): materialize indexed effective_date sort key
The gallery cursored on COALESCE(post.post_date, image_record.created_at) across the Post outer join — an expression spanning two tables that no index can serve, so every /scroll sorted a large slice of the library (and the old frontend fired ten serially). Materialize it: - image_record.effective_date column + ix_image_record_effective_date (effective_date DESC, id DESC); alembic 0035 backfills COALESCE(primary post's post_date, created_at) for existing rows. - gallery_service._effective_date_col() now returns the column, so scroll / timeline / jump / neighbors all order off the index instead of re-deriving the COALESCE. _neighbors reads record.effective_date directly (drops an extra Post lookup). - importer._apply_sidecar maintains it: when a primary post with a date is linked, effective_date = post.post_date; plain inserts keep the created_at-equivalent server default. Tests: sidecar import asserts effective_date == post.post_date; gallery ordering/timeline/jump test seeds set effective_date alongside created_at. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
56cc253009 |
feat(gallery): reveal tiles on image load + single initial fetch
The 5×10 metadata batching only staggered the cheap layer (JSON); thumbnails load as independent <img> requests and clustered, so tiles "popped in together" after a wait. Two changes: - GalleryItem reveals each tile when ITS OWN thumbnail fires @load (with an onMounted complete-check for cached thumbs), playing a showcase-style flip-up entrance. Tiles now cascade in natural load order instead of all at once. Honors prefers-reduced-motion. - gallery store does ONE initial fetch (limit=50) instead of 10 serial /scroll round-trips. Fewer RTTs, faster first paint; the reveal-on-load is what makes appearance progressive now. Infinite scroll pulls 25/trigger. Tests: GalleryItem gains is-loaded only after @load; loadInitial issues exactly one scroll request at the initial limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
576e16d14d |
fix(download): release DB connections across the gallery-dl subprocess
Backfill events were STILL stranding empty after the timeout-ladder fix.
Worker logs showed the salvage path working ("Download timeout for
anduo/patreon after 1170.0s (18 files written)") but then:
Retry in 3s: DBAPIError(ConnectionDoesNotExistError: connection was
closed in the middle of operation)
...succeeded in 0.149s <- in-flight guard no-op
Root cause: DownloadService held the async + sync DB connections checked
out across the entire (≤19.5-min backfill) gallery-dl subprocess. The
server reaps the idle connection, so phase 3's first query hits a dead
socket. That DBAPIError trips download_source's autoretry_for, the retry
re-enters _phase1_setup, sees the event still 'running', returns
in_flight and no-ops — leaving the event to be stranded empty by the
recovery sweep. pool_pre_ping was already on both engines but can't help
a *held* connection (it only validates on pool checkout).
Fix:
- DownloadService.download_source closes the async + sync sessions after
phase 1, before the subprocess, so phase 3 re-acquires a live
connection (matches the class's "Phase 2 — no DB connection" docstring).
- The per-task async engine switches to NullPool so phase 3 always opens
a fresh connection rather than a pooled one the server may have reaped.
Tests: assert connections are released before gdl.download runs and the
event still finalizes; assert the task engine uses NullPool. Also fixes a
stale 1800s->1170s comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|