Operator-confirmed on a fresh build: both the tag-chip and suggestion
kebabs still never opened. The prior 8326e54 'fix' only wrapped them in a
<span @click.stop> — inert for SuggestionItem (no parent capture) — and
never addressed why the `#activator`/`v-bind="props"` click failed to
toggle the menu inside the teleported ImageViewer modal. The dialogs in
that same modal open via v-model and work, so drive the menus the same way:
- The activator (v-btn / v-icon) toggles a reactive flag with @click.stop
(which also shields the chip's close button / any parent).
- The v-menu binds that flag (v-model / :model-value) and uses
activator="parent" with :open-on-click="false" purely for positioning,
so opening no longer depends on Vuetify's activator-click path.
- TagPanel tracks a single openTagId (one chip menu open at a time).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Not all characters belong to a fandom (original characters, unsorted).
The create flow forced every new character through FandomPicker, whose
only outcomes were 'Use this fandom' (disabled until one is picked) or
Cancel (which aborts the whole creation) — there was no way to confirm a
character with no fandom.
- FandomPicker: add a 'No fandom' action that emits confirm(null).
- TagAutocomplete.onFandomChosen: pass fandom_id: null when null is
emitted.
Backend already supported this end to end (Tag.fandom_id nullable, the
CHECK only forbids fandom_id on non-character kinds, tag_service
find_or_create defaults fandom_id=None, API reads body.get). A fandom can
still be assigned later from the chip kebab's 'Set fandom…'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The phash_threshold knob (controls whether edits/variants of an image are
dropped as near-duplicates on import) was buried at the bottom of the import
filters form and labelled opaquely, so it read as 'missing'. Hoist it to the
TOP of the form as a 'Near-duplicate sensitivity' section: a labelled slider
(Exact / Strict / Default / Loose stops, 0-16) for the gist + the precise
number field, both bound to phash_threshold, with copy that says plainly to
lower it if variants are being dropped.
Also swap the import-tab order to filters → trigger → recent-tasks (filters on
top per operator); the task list stays directly under the trigger for hit/miss
feedback adjacency.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Post cards no longer expand the whole card on click (the old two-click path
to the images). The card is compact-only now:
- Hero / rail thumbs / the +N tile are buttons that open the post-scoped image
modal (modal.open(id, { postImageIds })) so you view big + arrow through ALL
the post's images. The feed caps thumbnails at 6, so for posts with more we
lazily getPostFull to get the complete id list; +N opens at the first hidden
image.
- The description is the ONLY in-place expansion: a Show more / Show less toggle
shown only when the text is actually truncated (server description_truncated
flag OR a measured CSS-clamp overflow, ResizeObserver-guarded). Expanding
loads description_full when server-truncated and renders it unclamped.
- Attachments: download chips now render inline in the compact card (the feed
already carries download_url), since the expanded view is gone.
Removes PostImageGrid.vue (the mosaic, now unused). Tests cover show-more
visibility + image-click opening the scoped modal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The transparency / single-color audit cards held the run + poll timer in
local component state, so navigating away destroyed both and onMounted never
reconnected — the Celery scan kept running and writing LibraryAuditRun, but
the UI forgot it. Now each card, on mount, fetches its rule's latest run
(GET /api/cleanup/audit?rule=<rule>&limit=1) and rehydrates: shows progress +
resumes polling if still running, or shows the completed result (ready/applied/
error) so the operator can act on it after returning. Adds the ?rule= filter
to the audit-history endpoint + cleanup store latestAuditForRule().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wipe every general + character tag so the operator can re-tag from scratch via
the Camie auto-suggest, while PRESERVING fandoms, series (+ series_page order),
and each image's stored tagger_predictions (so suggestions repopulate
immediately). One set-based DELETE FROM tag WHERE kind IN ('general','character')
— the five tag-referencing tables all cascade, so applications + aliases +
allowlist + rejections + centroids clear automatically; series tags aren't
deleted so series survive; Tag.fandom_id is SET NULL so fandoms are untouched.
Reuses the established dry-run-preview -> confirm pattern: cleanup_service.
reset_content_tagging() + POST /api/admin/tags/reset-content +
TagMaintenanceCard section with a backup-first warning and a red confirm
showing exact counts (tags by kind + image applications). Irreversible except
via DB backup restore; the wipe only fires when the operator confirms.
Tests: service dry-run counts + live delete preserves fandom/series/series_page
while content tags + their image_tag cascade away; API dry-run wiring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'See all similar' takeover fetched /api/gallery/similar fine (200, ~100
results) but the grid showed nothing: GalleryGrid renders ONLY by iterating
store.dateGroups, and similar-mode returns date_groups=[] (results are ranked
by cosine distance, not chronological). Zero groups → zero tiles despite
store.images being full. Add a flat fallback: when there are no date groups
but images exist, render them as one ungrouped list in ranked order (no date
headers). The modal Related strip was unaffected (it renders its images
directly). Test locks both the flat and grouped paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vuetify's auto card-stack was too verbose (one subscription filled the whole
phone screen) and the expanded sources still needed lateral scroll. Replace it
below 600px (useDisplay) with a custom compact-card list: each subscription is
a 2-line card (name + health + expand chevron, then platform chips + sources
count + last activity) so several fit per screen. Expanding shows the action
row + each source as a STACKED SourceCard (new) — platform/url/enabled/last/
next/errors/actions laid out vertically, no horizontal scroll. The mobile
cards drive the same selected/expanded key arrays as the desktop data-table,
so selection and bulk actions are unchanged. Desktop keeps the v-data-table.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The subscriptions v-data-table (select + expand + 6 cols + a nested 8-col
sources sub-table) horizontally-scrolled on phones. Set mobile-breakpoint=600
so Vuetify stacks each subscription row into a label:value card below 600px;
the custom item slots (platform chips, health dot, action buttons) render as
card rows. The expanded sources detail reclaims its desktop indent on mobile
and keeps its own horizontal scroll for the wide source columns.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PostsFilterBar didn't wrap and its artist/platform fields had inline
min-widths (240/180px) a media query can't override → horizontal overflow on
phones. Moved widths to classes, added flex-wrap, and <600px each field takes
a full-width row. SubscriptionsTab's status/search inline max-widths likewise
moved to classes; <600px they go full-width and the v-spacer is dropped so the
search isn't shoved around.
Verified as already-fine (sweep false positives, no change): PostCard (default
body is a stacked column; only goes row at container >=800px), SeriesReaderView
(already has a <=768px block: nav drawer 150px, quick-nav stacks). The
subscriptions v-data-table scrolls horizontally within its own wrapper, so it
doesn't widen the page — a true mobile card layout is a larger follow-up if
wanted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The top nav packed brand + health + pipeline chip + ~7 inline links + an
action slot into one flex row, colliding/overflowing on phones (operator:
'almost unusable'). Below 768px the links now fold into a hamburger v-menu;
below 480px the brand text hides (glyph still brands). Plus the primary
browsing path:
- BulkEditorPanel: fixed 320px -> min(320px, 90vw) so it can't swallow the screen.
- GalleryFilterBar: <600px gives search its own full-width row (its 200px
min-width was jamming the wrapping bar); sort grows.
- GalleryFacetPanel: <480px wraps groups + lets the side-by-side date inputs
grow full-width.
- ArtistsView grid: minmax(min(440px,100%),1fr) so a card never overflows
(single column on phones).
- GalleryView: hide the year/month timeline strip <600px.
ImageViewer already stacks its side panel below the image <900px (left as-is).
Secondary surfaces (Posts/Subscriptions filter bars, SubscriptionsTab table,
SeriesReader, PostCard) still need a mobile pass — follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Showcase reveal cadence 80ms -> 160ms (slower, more deliberate one-at-a-time
cascade per operator). Bump showcase.spec timer advances to cover 60x160ms.
- Gallery filter bar now uses the EXACT gradiated-obsidian frost + blur as
TopNav (was a flat rgba(...,0.55) block), so the two read as one continuous
piece of chrome with images visibly scrolling under both; the nav's
transparent bottom edge against the bar's opaque top leaves a faint seam that
separates them at the very top of scroll.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Modal 'Related' strip (RelatedStrip.vue) — top-12 similar thumbs, fetched on
its own DEFERRED, single-flighted path (200ms after the modal is up) so it
never blocks or slows the modal; collapses silently on empty/slow/error and is
hidden when the image has no embedding (has_embedding flag). 'See all similar'
closes the modal and navigates the gallery to ?similar_to=<id>.
Gallery store: similar_to filter field + loadSimilar() (ranked, hasMore=false,
no timeline); applyFilterFromQuery routes similar-mode to /similar with the
scope filters composed; cloneFilter/filterToQuery carry similar_to. Filter bar:
clearable 'Similar to #id' chip, sort hidden in similar-mode; timeline sidebar
hidden too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GalleryService.similar() ranks images by pgvector cosine distance to a source
image's precomputed SigLIP embedding — no query-time ML inference. Composes
with the Phase-1/2 scope filters (AND) but replaces the date sort (always
nearest-first, bounded top-N, no cursor). Returns None for a missing source
(→404), [] for a source with no embedding (video / pending ML); excludes self
and NULL-embedding rows. New GET /api/gallery/similar?similar_to=<id>&limit=N.
Image-detail payload gains has_embedding so the UI can hide the surface.
Alembic 0036 adds an HNSW vector_cosine_ops index on siglip_embedding (1152<2000
dims) so the search is sub-50ms ANN instead of a full scan; one-time ~30-60s
build over existing embeddings on deploy. Shared _gallery_images/_image_json
helpers de-dup the scroll/similar builders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With fsync-off the whole integration suite runs in ~45s (was ~13min across
shards), so the 3-way split only triplicated the ~2min fixed overhead
(container + install + migrate) and consumed 3 of 6 runner slots for no
wall-clock gain. Merge intapi/intimp/intcore into one `integration` job:
spin up once, install once, migrate once, run `pytest -m integration` over
the whole suite. Frees 2 runner slots (6 jobs -> 4) and drops ~140 lines of
near-duplicate YAML.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Option 1 (pooling the teardown connection) left teardowns at ~1.5-2s/test, so
the cost is the per-test TRUNCATE's commit forcing an fsync, not the connect
handshake. Each shard now ALTER SYSTEM SETs fsync/synchronous_commit/
full_page_writes off + pg_reload_conf() right after deps install, before
alembic — sighup/user-context GUCs apply with no restart. The DB is ephemeral
(rebuilt per run) so fsync-off is safe; the step is non-fatal so a perms
surprise can't red a shard. Speeds up every test's commit (setup inserts +
the teardown TRUNCATE), stacking on the pooled engine from the prior commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The autouse integration teardown created a fresh SQLAlchemy engine + Postgres
connection for EVERY test, then disposed it — --durations showed the 15
slowest ops in both long shards were all ~1.5-2s teardowns (the connect+SCRAM
handshake, not test logic). Hoist the truncate engine to a session-scoped,
pool_pre_ping'd fixture so the pooled connection is reused across teardowns;
the TRUNCATE+restore still runs per test, so isolation is unchanged. Lazy
create_engine means the no-DB unit job instantiates but never connects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The buffered cascade revealed tiles on an 80ms timer regardless of image
load, so the flip-in animation played on a gray placeholder and the thumbnail
popped in afterward. Worse, MasonryGrid ALSO applied a per-index
animation-delay (index×70ms) that compounded on top of the insert cadence,
so the cascade visibly dragged and desynced as it grew.
Now the producer preloads each queued thumbnail (decode pipelined ahead) and
the consumer awaits that decode before pushing the item — every tile animates
in fully loaded, strictly one at a time. Drop the compounding CSS stagger;
the store's one-item-at-a-time push is the sole pacer, so each tile animates
the instant it mounts. New utils/preloadImage.js (load+decode+timeout gate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite facets() common/plat_scope as dict literals (C408). Open the refine
panel via a watch on hasRefineFilters rather than reading filter state at
bar-setup time — the parent applies the URL query in its onMounted, after the
bar child has set up, so the initial read was always the default (empty) state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'Refine ▾' toggle to the gallery filter bar that expands a full-width
GalleryFacetPanel below it, inside the same sticky hazey chrome. The panel
offers platform chips (with live counts + a 'No platform' unsourced bucket),
two count-badged curation-flag toggles (Untagged / No artist), and a from/to
date range bounded by the facet min/max.
Store gains the platform/untagged/no_artist/date_from/date_to filter params
(URL-mirrored, AND-composed) and a panel-gated, single-flighted loadFacets()
that fetches /api/gallery/facets scoped to the active filter. Shared
cloneFilter/filterToQuery helpers keep the bar and panel writing one URL
format. The panel auto-opens on deep-link when refine filters are present and
refetches counts (debounced) on every filter change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the composable gallery filter with platform / untagged / no_artist /
date_from / date_to, AND-composed with the existing tag/artist/media/sort
params and threaded through scroll, timeline, and jump_cursor.
Add GalleryService.facets() + GET /api/gallery/facets returning live counts
scoped to the current filter with per-group minus-self semantics: platform
counts (COUNT(DISTINCT image) incl. a null unsourced bucket), curation-flag
counts (untagged / no_artist), and effective_date min/max bounds. The
UNSOURCED_PLATFORM sentinel makes filesystem-imported content reachable via
the platform facet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Image viewer (#609):
- The next (▶) arrow was offset from the viewport edge (right:16px) so it
floated over the 320px metadata side panel. Offset it off a shared
--fc-side-w var so it sits at the image's right edge instead; full-width
again below 900px when the panel stacks under the image.
- Arrow nav was fully disabled whenever a text field was focused. Now it
yields to the caret ONLY when the field has text; an empty tag-entry field
still navigates ←/→. Extracted to utils/textEntry.js (arrowNavAllowed).
ESC behaviour unchanged (already closes the modal, overlay-aware).
Test: arrowNavAllowed — empty/non-text → navigate, text present → don't.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Settings → Maintenance gains a "Database maintenance" card: a "Run VACUUM
ANALYZE now" button (enqueues the maintenance task) plus a per-table bloat
readout (live/dead/dead%/last vacuum) from /api/admin/maintenance/db-stats.
- dbMaintenance store (loadStats / runVacuum) + test.
- Fix ruff I001: combine the two _sync_engine imports onto one line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The TABLESAMPLE showcase reads physical blocks (bloat-sensitive), and the
periodic prune/backfill/recovery tasks churn dead tuples faster than
autovacuum always keeps up — so explicit maintenance earns its keep here.
- tasks.maintenance.vacuum_analyze: VACUUM (ANALYZE) over high-churn tables
(VACUUM_TABLES) on an AUTOCOMMIT connection (VACUUM can't run in a txn).
Scheduled weekly via Beat; also operator-triggerable.
- _sync_engine.get_sync_engine(): expose the process engine for the
autocommit connection.
- GET /api/admin/maintenance/db-stats: per-table n_live/n_dead/dead_pct +
last (auto)vacuum/analyze from pg_stat_user_tables — visibility, not a
black box.
- POST /api/admin/maintenance/vacuum: enqueue the task on demand.
Tests: vacuum task runs + reports tables; db-stats shape; trigger queues.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Filter bar gets the same obsidian translucent + backdrop-blur as the
TopNav so the two read as one piece of chrome.
- margin-top:-8px cancels the v-container's pt-2 so the bar sits flush at
64px even at scroll 0 — fixes the gap/separation when scrolled to top.
- Inputs/toggles get a more-opaque backing so they stay legible on the haze.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cascade "burped" — chunks appeared unevenly — because the old pipeline
coupled display to fetch timing: it trickled each batch right after its
fetch and assumed the next round-trip would land inside the ~240ms trickle
window. When a fetch ran long (TABLESAMPLE hits random, sometimes-cold
blocks; RTT jitter) the animation starved, then a clump burst in.
Decouple the two:
- Producer (_fill) races ahead fetching batches into a buffer up to a
target depth, refilling when it dips below BUFFER_MIN.
- Consumer (_drain) reveals one item every CADENCE_MS regardless of when
fetches land; it only waits if the buffer genuinely starves.
A small PRIME buffer precedes the drain so it doesn't starve at the front;
the buffer (BUFFER_MIN×CADENCE runway) absorbs per-fetch jitter so images
appear at an even pace. Public store API (loadInitial/shuffle/fetchPage/
images/loading/hasMore/isEmpty) unchanged — ShowcaseView/MasonryGrid need
no change.
Test (fake timers): fire-order + dedup, one-item-per-cadence rate limit,
empty-library flag.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gallery now has in-view filtering, styled like the app's sticky v-tabs
chrome (pinned at top:64px under TopNav).
- GalleryFilterBar: combined tag+artist autocomplete (searches
/api/tags + /api/artists), closable filter chips (multi-tag AND),
media toggle (All/Images/Videos), Newest/Oldest sort, Clear. Writes all
state to the URL via router.push.
- gallery store: filter is now { tag_ids, artist_id, media_type, sort,
post_id }; applyFilterFromQuery makes the URL the single source of truth
(deep-linkable, back-button works); chip labels resolved by id or
pre-noted on pick. Replaces the standalone tag chip + setTag/PostFilter.
- GalleryView: renders the bar (hidden in post-detail), syncs route.query
→ store on mount + every query change.
Also untracks the transient .claude/scheduled_tasks.lock committed in
3f30327 and gitignores it.
Tests: store parses query → composable scroll params, post_id exclusivity,
newest-sort omitted, label pre-seed, single initial fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 backend for the gallery filter bar. Extends scroll/timeline/jump
from a single mutually-exclusive filter to a composable one:
- tag_ids: image must carry ALL of them (one correlated EXISTS per tag —
AND, no row multiplication), replacing the single-tag JOIN.
- artist_id composes with tags; media_type ('image'|'video') narrows by
mime; post_id stays the exclusive post-detail path.
- sort ('newest'|'oldest') flips the effective_date/id cursor comparison
and ordering; the cursor value is unchanged (direction comes from the
request). jump_cursor honors sort too.
- Shared _apply_scope helper applied across scroll/timeline/jump so the
timeline sidebar reflects the filtered set. API _parse_filters parses
tag_id (comma list), artist_id, media, sort.
Tests: multi-tag AND, media filter, sort reversal (service + API);
post_id-excludes-others; single tag_id back-compat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two gaps where a filter couldn't be removed:
- Gallery: a tag_id filter (from clicking a tag) had no indicator or clear
control — only post_id did (PostInfoHeader). Add an "Tag: <name> ✕" chip
that clears the filter by dropping tag_id from the URL. New lightweight
GET /api/tags/<id> resolves the name; the store fetches it on filter set.
- Tags view: the kind chip-group used mandatory="false" — a STRING ("false"
is truthy in JS), which made the group mandatory so the active kind chip
couldn't be deselected. Fixed to :mandatory="false" so the filter clears.
Tests: GET /tags/<id> shape + 404; gallery store resolves filterTagName.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the missing UI to change a character tag's fandom, in both places:
- FandomSetDialog (shared): pick an existing fandom, create a new one, or
clear it; on a name collision in the target fandom it surfaces a merge
confirmation and resolves via setFandom(merge:true). Reuses the tags
store's fandom cache.
- TagCard kebab gains "Set fandom…" for character tags (→ TagsView opens
the dialog, reloads on success).
- TagPanel chip kebab gains "Set fandom…" for character tags (→ reloads the
modal's tag list on success).
- tags store: setFandom(tagId, fandomId, {merge}) action + test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No way existed to change which fandom a character tag belongs to after
creation — PATCH /tags/<id> only renamed.
- TagService.set_fandom(tag_id, fandom_id, merge=False): set / change /
clear (fandom_id=None) a character's fandom, with the same validation as
find_or_create. On a name collision in the target fandom it raises
TagMergeConflict (→ 409, same shape as rename); merge=True resolves it by
merging this tag INTO the existing character.
- Extract _do_merge(source, target) from merge() so set_fandom can perform
the deliberate CROSS-fandom merge the public merge() validation forbids.
- PATCH /tags/<id> now accepts optional fandom_id (+ merge flag) alongside
name, and returns fandom_id.
Tests: set/change/clear, non-character + bad-ref rejection, collision
raises, merge resolves; API set/clear + collision→merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gallery cursored on COALESCE(post.post_date, image_record.created_at)
across the Post outer join — an expression spanning two tables that no
index can serve, so every /scroll sorted a large slice of the library
(and the old frontend fired ten serially). Materialize it:
- image_record.effective_date column + ix_image_record_effective_date
(effective_date DESC, id DESC); alembic 0035 backfills
COALESCE(primary post's post_date, created_at) for existing rows.
- gallery_service._effective_date_col() now returns the column, so scroll
/ timeline / jump / neighbors all order off the index instead of
re-deriving the COALESCE. _neighbors reads record.effective_date
directly (drops an extra Post lookup).
- importer._apply_sidecar maintains it: when a primary post with a date is
linked, effective_date = post.post_date; plain inserts keep the
created_at-equivalent server default.
Tests: sidecar import asserts effective_date == post.post_date; gallery
ordering/timeline/jump test seeds set effective_date alongside created_at.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 5×10 metadata batching only staggered the cheap layer (JSON);
thumbnails load as independent <img> requests and clustered, so tiles
"popped in together" after a wait. Two changes:
- GalleryItem reveals each tile when ITS OWN thumbnail fires @load (with
an onMounted complete-check for cached thumbs), playing a showcase-style
flip-up entrance. Tiles now cascade in natural load order instead of all
at once. Honors prefers-reduced-motion.
- gallery store does ONE initial fetch (limit=50) instead of 10 serial
/scroll round-trips. Fewer RTTs, faster first paint; the reveal-on-load
is what makes appearance progressive now. Infinite scroll pulls 25/trigger.
Tests: GalleryItem gains is-loaded only after @load; loadInitial issues
exactly one scroll request at the initial limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill events were STILL stranding empty after the timeout-ladder fix.
Worker logs showed the salvage path working ("Download timeout for
anduo/patreon after 1170.0s (18 files written)") but then:
Retry in 3s: DBAPIError(ConnectionDoesNotExistError: connection was
closed in the middle of operation)
...succeeded in 0.149s <- in-flight guard no-op
Root cause: DownloadService held the async + sync DB connections checked
out across the entire (≤19.5-min backfill) gallery-dl subprocess. The
server reaps the idle connection, so phase 3's first query hits a dead
socket. That DBAPIError trips download_source's autoretry_for, the retry
re-enters _phase1_setup, sees the event still 'running', returns
in_flight and no-ops — leaving the event to be stranded empty by the
recovery sweep. pool_pre_ping was already on both engines but can't help
a *held* connection (it only validates on pool checkout).
Fix:
- DownloadService.download_source closes the async + sync sessions after
phase 1, before the subprocess, so phase 3 re-acquires a live
connection (matches the class's "Phase 2 — no DB connection" docstring).
- The per-task async engine switches to NullPool so phase 3 always opens
a fresh connection rather than a pooled one the server may have reaped.
Tests: assert connections are released before gdl.download runs and the
event still finalizes; assert the task engine uses NullPool. Also fixes a
stale 1800s->1170s comment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backfill downloads stranded with empty logs + a generic "stranded by
recovery sweep" error. Root cause: the backfill gallery-dl subprocess
timeout (1170s) exceeded download_source's Celery soft_time_limit (900s),
so SoftTimeLimitExceeded preempted subprocess.TimeoutExpired. The
TimeoutExpired path (which captures partial stdout/stderr and finalizes
the event) never ran, the event was left 'running', and phase 3 never
decremented backfill_runs_remaining — so the source re-ran and
re-stranded every tick (Anduo #39912).
Two layers:
1. Raise download_source limits (soft 900→1350, hard 1200→1500) so both
subprocess budgets (870 tick / 1170 backfill) sit below the soft
limit with phase-3 persist headroom. Promote to module constants and
guard the invariant with a test.
2. Catch SoftTimeLimitExceeded in download_source and finalize the
in-flight event with a real reason, mirror phase-3 source-health, and
decrement backfill so a chronically-slow source self-heals to tick
mode. The existing celery_signals handler only covered TaskRun, not
DownloadEvent — that was the gap.
Updates stale 900/1200 references in gallery_dl.py + maintenance.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bvandeusen