Operator-flagged 2026-06-01: with the failing-sources panel expanded,
all rows have the same flat low-opacity background, no visual track
from the artist name on the left to the Logs/Retry buttons on the
right. Hard to tell which row a button belongs to when scanning down
a list of 17.
Three pure-CSS changes (no DOM, no logic):
- 3px gap between rows (was 2px) — slightly more breathing room
- Even rows get a darker surface tint (zebra striping)
- Hover paints the whole row in a low-opacity error tint, so moving
the cursor toward Logs/Retry lights up the whole horizontal band
and the eye traces back to the artist name automatically
`test_set_backfill_runs_arms_source` was pinning the pre-auto-arm initial
value (0) when checking that the override works. Now that create() pre-arms
enabled sources to 3, the assertion is stale — the test was already
verifying the override path, the pre-assertion just decorated it.
Drop the pre-assertion; keep the override check. Other tests use raw
Source(...) constructors (default to 0 via the column server_default),
not SourceService.create(), so they're unaffected.
A freshly created subscription has no gallery-dl archive yet, so the
first poll in tick mode would walk forever — exit:20 doesn't trip
until 20 contiguous archive-hits, and there are none. The wall-clock
cap kicks in mid-walk, and the partial-success classifier (plan #544)
gracefully labels it status=ok, but the operator still wonders why
their new subscription isn't grabbing the back-catalog.
Pre-arm `backfill_runs_remaining = 3` on create() when `enabled=True`.
Same default as the manual "Deep scan" button, same auto-decrement
and auto-reset rules — once the queue drains (clean exit + zero new
files) or the budget runs out, tick mode resumes naturally.
Disabled sources (sidecar synthetics with `url='sidecar:<platform>:<slug>'`
that arrive disabled = False, or operator-added sources that start
disabled deliberately) skip the pre-arm — they're never polled, no
budget to waste.
Operator-flagged 2026-06-01 (DaferQ patreon, event #38919). Video posts
hosted on Mux carry a JWT that encodes a playback restriction policy
(`playback_restriction_id` in the token). The token signature alone is
not sufficient — Mux's CDN ALSO checks Referer/Origin on every fetch.
gallery-dl's HEAD probe to `stream.mux.com/<id>.m3u8?token=...` returns
200 (token is valid, HEAD sends no Referer that Mux's policy rejects),
but when yt-dlp follows up with the actual manifest GET it sends its own
default Referer and Mux 403s. yt-dlp retries 4 times, gives up, the
single video fails — every other image in the same post still downloads.
Static `downloader.ytdl.raw-options.http_headers` block now pins Referer
and Origin to `https://www.patreon.com` so yt-dlp's manifest fetch
clears the policy check.
Headers-only restrictions are now handled. Mux IP-range restrictions
(if a creator opts into them) remain unfixable from our worker — those
would need a Patreon-region IP. Per-video PARTIAL classifier (shipped
in plan #544 last commit) already handles the residual case: a run that
grabs all images and fails 1 video classifies as status=ok rather than
flipping the whole event red.
Routine subscription polls walked the entire post history every tick
even when nothing had changed, because gallery-dl's default `skip: True`
continues iterating archived posts. A creator with ~550 archived posts
(Knuxy patreon) saturates the 870s wall-clock cap before completing,
even with zero downloads needed. Plus, a tier-limited run that
downloaded hundreds of files but ran out the clock should be a
warning, not an error.
Two coupled changes, both operator-flagged 2026-06-01:
* **Tick mode (default, cron polls).** New `TICK_SKIP_VALUE = "exit:20"`
asks gallery-dl to exit after 20 contiguous archived items. Fresh
subscriptions + new-content cases still walk normally; established
subscription with zero new content exits in ~30s of HEAD requests
instead of pegging the timeout. 20 (not 5) gives headroom against
paywall warnings interleaving with archived items.
* **Backfill mode (explicit, operator-triggered).** Sticky for N runs
via new `Source.backfill_runs_remaining` (alembic 0031). While > 0,
downloads use `skip: True` + 1800s timeout. Auto-decrements per run
with early-reset to 0 when a clean run finds zero files (queue
drained). N defaults to 3 — multiple runs give the system enough
budget to finish a deep walk across timeout boundaries. New
`POST /api/sources/{id}/backfill` arms the source; "Deep scan"
button on each SourceRow (chip shows remaining count) wires it.
Plus partial-success classifier: non-zero gallery-dl exit + ≥1 file
downloaded + no source-level error fires `ErrorType.PARTIAL`, which
download_service maps to `status=\"ok\"`. The run did real work; the
next tick continues via gallery-dl's archive. No more red events for
"timed out mid-walk after downloading 300 files."
Retires `SourceConfig.skip_existing` — skip value is now derived from
the source state and passed as a separate `skip_value` parameter
through download() / _build_config_for_source(). `GD_DEFAULTS` drops
the now-dead key (was inert data after this refactor).
Tests cover:
* tick + backfill skip-value emission in _build_config_for_source
* PARTIAL classifier branch + TIER_LIMITED-wins-over-PARTIAL ordering
* SourceService.set_backfill_runs validation + persistence
* /api/sources/{id}/backfill 200/400/404 paths
* download_service auto-decrement / auto-reset / tick-mode-no-touch
* PARTIAL → status=ok in the orchestrator (no consecutive_failures bump)
gallery_service._provenance_clause artist branch was correlating its bare
Post reference to the outer query's primary_post_id outer-join, so the
artist filter silently matched zero rows for images with no primary post.
Alias Post inside the EXISTS subquery so SQLAlchemy adds it to the inner
FROM rather than treating it as a correlated outer table.
Five sidecar/import tests still asserted that a synthetic Source row
appears after a filesystem import. Alembic 0030 retired that behavior;
the Post sits null-source and the artist linkage lives on Post.artist_id.
Updated test_sidecar_creates_provenance, test_reimport_same_post_idempotent,
test_sidecar_artist_used_when_no_folder_artist, test_supersede_applies_new_file_sidecar,
and test_apply_sidecar_recovers_from_integrity_error to assert
post.source_id IS NULL + post.artist_id linkage instead.
Operator-asked 2026-06-01 after the Dymkens orphan investigation
(Scribe plan #540). The pre-2030 sidecar-synthetic Source pattern
(`sidecar:<platform>:<slug>` enabled=false rows) existed solely to
satisfy `Post.source_id NOT NULL`, and leaked into the Subscriptions
UI as phantom subscriptions. Now the data model says what's true:
filesystem-imported content with no live subscription has NULL
source_id, full stop.
## Schema (alembic 0030)
- `post.artist_id` — NEW NOT NULL FK to artist (CASCADE). Backfilled
from source.artist_id in the migration. Indexed for the artist-filter
queries.
- `post.source_id` — NOT NULL → nullable; FK ondelete CASCADE → SET
NULL. Deleting a Source detaches its Posts instead of destroying
archived content (subscription ends, archive stays).
- `image_provenance.source_id` — same nullable + SET NULL.
- Partial unique index `uq_post_artist_external_id_null_source` on
(artist_id, external_post_id) WHERE source_id IS NULL — guards
filesystem-import dedup since the existing source-bound unique
ignores NULLs (Postgres treats NULL != NULL).
- Sidecar synthetic Sources deleted: NULL out FKs in post,
image_provenance first, then DELETE FROM source WHERE url LIKE
'sidecar:%'. The Dymkens cleanup.
## Model + service changes
- `Post.source_id` → `Mapped[int | None]`; new `Post.artist_id`
denormalized.
- `ImageProvenance.source_id` → `Mapped[int | None]`.
- Importer: `_source_for_sidecar` (synthetic-creating) →
`_lookup_source_for_sidecar` (returns None when no subscription).
`_find_or_create_post` takes required `artist_id`; matches on
(source_id, external_post_id) for source-bound posts or
(artist_id, external_post_id) for NULL-source posts.
- Service queries switched off the Source detour to use Post.artist_id
directly: post_feed_service.scroll/around/get_post (LEFT JOIN to
Source so NULL-source posts surface); artist_service date_row/
activity/post_count; provenance_service.for_image/for_post (LEFT
JOIN); gallery_service._provenance_exists_where_artist via
Post.artist_id instead of ImageProvenance.source_id → Source.
- `_to_dict` and provenance dict-builders emit `"source": null` for
NULL-source rows.
## Frontend
- `ProvenancePanel.vue` + `PostCard.vue`: render `e.source?.platform
?? 'filesystem import'` so NULL-source posts get a clear
"filesystem import" affordance instead of a NaN crash.
## Tests
- `test_importer_upsert_helpers`: removed the four synthetic-anchor
tests; added `_find_or_create_post_idempotent_with_null_source`
(dedup via the partial unique index) and
`_lookup_source_for_sidecar_returns_*` (existing-subscription +
none cases). The existing `_find_or_create_post_idempotent` now
also passes `artist_id` and asserts it.
- 8 other test files updated: every direct `Post(...)` construction
gains `artist_id=<artist>.id`. The `_seed_post` helper in
`test_post_feed_service` looks up artist_id from the source row so
callsites stay one-arg.
## Verification on deploy
After alembic 0030 runs:
- `SELECT COUNT(*) FROM source WHERE url LIKE 'sidecar:%'` → 0.
- `SELECT COUNT(*) FROM post WHERE source_id IS NULL` → count of
filesystem-imported posts (Dymkens + any other historical).
- Every `post.artist_id` non-null; consistent with source.artist_id
for source-bound rows.
- Subscriptions tab: no Dymkens phantom row.
- Artist detail → Posts/Gallery: Dymkens's content still reachable
via Post.artist_id.
- Provenance panel renders "filesystem import" chip for NULL-source
posts; PostCard same.
## Out of scope
- UI to manage/delete orphan NULL-source Posts. Data model is right;
UI follows if operator wants it.
Operator-asked 2026-06-01: each row in the "X sources are failing"
rollup needs a button to surface the last run's stdout/stderr/error
without leaving the Downloads tab to find the matching event row.
Wired:
- `downloadsStore.loadLastForSource(sourceId)` two-steps via
`GET /api/downloads?source_id=N&limit=1` (most recent event) then
the existing `loadOne(id)` for the full detail payload. Returns
null if no events exist (toast warns).
- `FailingSourcesCard` row: new text button `Logs` with
`mdi-text-box-search-outline`, per-row spinner via `logLoadingIds`,
emits `view-logs` with the source record.
- `DownloadsTab.onViewFailingLogs` is the handler — same
`DownloadDetailModal` instance the event-row clicks use.
Operator-flagged 2026-06-01: clicking a Provenance post title from the
view modal opens the post in the posts feed scoped to that artist
(`/posts?post_id=N&artist_id=A`), but the older/newer infinite-scroll
loaded UNFILTERED global posts instead of staying in the artist's
stream. Root cause: the posts store's loadAround/loadOlder/loadNewer
sent only `{around, cursor, direction}` — never the `artist_id` /
`platform` filters. Backend `/api/posts` accepts them on every path
(post_feed_service.scroll filters via `Source.artist_id`); the
frontend just wasn't passing them.
Fix:
- `posts.js` `loadAround(postId, filters)` now takes the filter
snapshot and stores it. `_aroundParams(extra)` mixes the active
filters into around/cursor calls.
- `PostsView.vue` `loadAroundAndAnchor` passes `artist_id` +
`platform` from the route query.
Two operator-flagged UX gaps from 2026-06-01:
1. **Esc trapped inside the tag-entry field**
The autofocused TagAutocomplete input made Esc-to-close unreachable
because ImageViewer's keydown handler bailed early on
`isTextEntry(ev.target)` for every key including Escape. Now Escape
closes the modal from text inputs too — except when a Vuetify
overlay is open (FandomPicker, autocomplete dropdown, suggestion
3-dot menu), in which case that overlay's own Esc-handling fires
instead of closing the whole modal mid-interaction. Detected via
`.v-overlay--active`. Arrow keys still gate on isTextEntry so the
tag input handles typing without navigating images.
2. **Provenance cards expanded the side panel unbounded**
When an image had many ImageProvenance entries the cards stack
pushed the Tags section below the fold. Wrapped the cards in a
`.fc-prov__cards` container with max-height 270px (≈2.5 cards) and
thin overflow scrollbar. Title stays anchored at top; Tags panel
sits at a consistent position below regardless of provenance count.
`${GITHUB_SHA:0:7}` substring expansion is bash-only; the runner
executes the step via dash/BusyBox sh and errored with
`Bad substitution` (act_runner workflow shell, observed on the
65386f0 main-push run). Switched to
`SHORT_SHA=$(printf '%s' "$GITHUB_SHA" | cut -c1-7)` which works in
both shells. Both build-web and build-ml tag-determination steps
updated.
Two operator-asked modal UX changes (Scribe plan #514):
1. **Post title click → artist-scoped posts feed**
- The bold post title in ProvenancePanel is now the primary click
target. Click navigates to /posts?post_id=N&artist_id=A; the
PostsView already filters on `artist_id`, so the user lands in
that creator's stream, not the global one.
- The redundant "View post" link is removed. "Show description"
stays as the only action link below the meta line.
- Title is styled as a button-link: accent color, hover underline,
focus ring for keyboard nav (family rule 24 — UI quality bar).
2. **Suggestion rows look like buttons**
- SuggestionItem becomes a chip-card: visible border, hover/focus
background, unified container for name + score + actions
(operator's "nothing visual to unify the buttons to their object"
complaint).
- Accept is an explicit tonal pill button labeled "Accept"
(operator's pick over whole-row-clickable). 3-dot menu retains
alias/dismiss, now `variant="outlined"` so it reads as a button.
- Score is a fixed-width monospace pill on the right.
- "+ new" badge upgraded to a pill chip with accent border so
it's visibly an annotation, not part of the tag name.
Four coupled operator-asked changes to the view modal (Scribe plan #509):
1. **Autofocus tag entry on modal open** — TagAutocomplete grabs focus
in onMounted/nextTick so the caret is in the input the moment the
modal renders. No click needed to start typing.
2. **General suggestions expanded by default** — SuggestionsPanel's
general-category group now mounts with `:default-open="true"`.
Operator can collapse if too noisy, but the v1 frame shows them.
3. **Lower general threshold default 0.95 → 0.50** — MLSettings.
suggestion_threshold_general default matches character. Alembic
0029 also bumps the existing singleton row's value if it's still
at the old 0.95. Operator can re-tune from Settings → ML.
4. **Retire `copyright` + `artist` as ML suggestion categories** —
neither feeds a Tag.kind (`artist` retired in FC-2d-vii-c, never
really existed as a copyright tag-kind). They were surfaced in the
suggestions pipeline + threshold settings UI but had no follow-
through. Drop from SURFACED_CATEGORIES, suggestions._threshold_for,
ml_admin GET/PATCH allowlist, MLSettings columns (alembic 0029
drops the two columns), frontend CATEGORY_ORDER + CATEGORY_LABELS,
SuggestionsPanel.peopleCats, AliasPickerDialog kind-check, and
MLThresholdSliders rows.
Out of scope (intentional): `tag_kind` Postgres enum still includes
`artist` for historic Tag row queryability (per the model comment);
no operator pain reported, no enum-shrink needed.
Tests:
- test_surfaced_categories asserts {character, general}, excludes
artist + copyright.
- test_threshold_for_artist_is_unsurfaced extended to cover copyright.
- test_get_and_patch_settings asserts new 0.50 default and the absent
artist + copyright keys in the GET payload.
Adds an immutable per-commit docker tag to every main-push build:
`git.fabledsword.com/bvandeusen/fabledcurator{,-ml}:c-<short_sha>`,
alongside the existing floating `:main` + `:latest`. Implements the
new family release-posture rule "Tags are milestones, not gates —
commit-SHA images are the rollback unit" so rollback to any commit
on main is `docker pull …:c-<sha>` with no release ceremony required.
Behavior change summary:
- main-push: was {:main, :latest} → now {:main, :latest, :c-<short_sha>}
- tag-push (opt-in vYY.MM.DD only, no .N): unchanged
- safety-net dev: unchanged
No code changes; the rule is about how the tag list is constructed.
Tag-push workflows stay as-is — vYY.MM.DD milestone cuts can still
fire them when the operator wants a labeled checkpoint.
Operator-asked 2026-05-31 (during sidecar synthetic anchor cleanup):
"the add source/subscription button idea to the firefox extension so
it can tell me if a source/artist is added or not and offer an option
to add it if it isn't." Plan tracked in Scribe task #507.
## Backend
- `ExtensionService.probe(url)` — read-only resolution. Reuses
`_derive` for platform+slug, then 2 SELECTs. Returns one of:
- `source_match` (exact (artist, platform, url) Source exists)
- `artist_match` (artist exists, this URL isn't a Source yet;
collapses the sidecar-synthetic-only case from v26.06.01.0)
- `new` (neither exists)
- `unknown_platform` (URL didn't match any artist-page regex)
- `GET /api/extension/probe?url=...` route with `X-Extension-Key`
auth posture matching `/quick-add-source`. Read-only, side-effect
free.
- 6 backend tests in tests/test_api_extension.py covering each state
+ auth + invalid URL.
## Extension
- `api.js`: `probeSource(url)` mirroring `quickAddSource` shape.
- `background.js`: `PROBE_SOURCE` + `OPEN_ARTIST_PAGE` handlers. The
latter strips the `/api` suffix from configured `apiUrl` (placeholder
format per options.html) and opens `${base}/artist/{slug}` in a new
tab via `browser.tabs.create`.
- `content-script.js`: probe-first render — on page-load and SPA
navigation, asks the backend for the URL's state and renders the
chip in the matching color/copy on FIRST paint instead of flashing
generic "Add" and updating after. Click handler branches:
`source_match` → OPEN_ARTIST_PAGE; `artist_match`/`new` → existing
ADD_AS_SOURCE flow (then re-probes so the chip flips green
immediately, no wait for next nav).
- `content-script.css`: three state-color modifiers
(--new, --artist-match, --source-match) on the FC parchment-on-slate
palette. Sage for already-added, amber for artist-exists, accent
orange for new.
## Versioning
- `extension/manifest.json` + `extension/package.json` → 1.0.6.
build.yml's sign-extension job will fire on push to main since no
`ext-1.0.6` Forgejo/Gitea release exists yet — exercises the
regenerated AMO keys end-to-end.
## Behavior on the sidecar-synthetic case
Filesystem-imported "Dymkens"-style artist with only a sidecar
synthetic Source: probe returns `artist_match` (not `new`), so the
chip reads "+ Add Patreon source to Dymkens" rather than offering to
recreate the artist. Clicking adds the real Source; existing
`_source_for_sidecar` preference logic (v26.06.01.0) routes future
gallery-dl Posts to the real one.
Three coupled operator-reported pains from the 2026-05-31 download
event audit:
1. `[patreon][warning] Not allowed to view post N` was bubbling up as
an error event, bumping consecutive_failures and parking the source
in "needs attention." The classifier's tier-gated branch was gated
on `return_code in (1, 4)`. Gallery-dl returns a different exit
code for mixed-failure runs (e.g. paywall warnings + a missing
yt-dlp dep flipping the exit bits), so the branch never fired and
the path fell through to UNKNOWN_ERROR. Widen the gate: when no
source-level error fired AND tier-gated warnings are present,
classify as TIER_LIMITED regardless of return code.
2. Knuxy event #38275 (2026-05-31) ran 30 min and finalized with
"stranded by recovery sweep (no terminal status after time_limit)"
+ empty stdout/stderr. Root cause: subprocess.run timeout (900s)
and Celery soft_time_limit (900s) raced; when Celery won, SIGKILL
wiped the in-memory captured output and the DownloadEvent ended up
empty-logged 18 minutes later when the sweep finalized it. Drop
gallery-dl's default subprocess timeout to 870s — a 30s margin
shy of Celery's soft limit — so subprocess.TimeoutExpired always
wins the race and captures the partial stdout/stderr via the
existing handler.
3. `[downloader.ytdl][error] Cannot import yt-dlp or youtube-dl` was
firing on every video attachment, causing per-item download
failures that masked legitimate tier-gated classification.
Add yt-dlp>=2025.1 to requirements.txt. Once it's in the image,
video posts download normally and the per-item failure noise
disappears.
Tests added:
- pure tier-gated stderr with exit code 128 → TIER_LIMITED + success
- mixed tier-gated + yt-dlp + per-item failures → still TIER_LIMITED
Two coupled bugs surfaced 2026-05-31 by the Subscriptions UI showing
"phantom" subscriptions like `sidecar:patreon:dpmaker`:
1. `SourceService.list()` returned every Source, no filter on URL.
alembic 0022 (2026-05-26) consolidated old per-post-URL Sources into
one canonical row per (artist, platform); when no real campaign URL
was salvageable it rewrote the canonical to `sidecar:<plat>:<slug>`
enabled=false as a disabled anchor. The UI then listed those
anchors as if they were polls — disabled, but visible. Fix: `list()`
excludes `url LIKE 'sidecar:%'` by default; `include_synthetic=True`
opts back in for admin tooling.
2. `importer._source_for_sidecar` picked the lowest-id Source for
(artist, platform). When alembic 0022 had rewritten a per-post row
into a synthetic anchor (lower id) AND the operator later added the
real subscription (higher id), every gallery-dl download silently
attached its Post to the SYNTHETIC instead of the real Source. Fix:
prefer a non-`sidecar:%` URL when one exists; fall back to the
synthetic; only create a new synthetic when nothing exists for
(artist, platform).
alembic 0028 is the data half: for every (artist, platform) with both
a synthetic AND a real Source, pre-merge Post+ImageProvenance
collisions on the canonical, bulk-repoint Posts/ImageProvenance/
DownloadEvent.source_id onto the real Source, and delete the
synthetic. Lone synthetics (no real twin) are left intact — they
anchor real imported content the operator may still want; the
list-filter hides them so they no longer surface as phantoms.
Today's platform-cooldown commit (61ce1ce) only filtered the scan tick
— manual /api/sources/<id>/check still bypassed it. Operator-flagged
2026-05-30: clicked "Retry failed" on a Patreon failure pile and saw
every one go 'queued' without realising the cooldown wasn't in the
loop. Bulk retry with N sources on a cooled-down platform bowls right
back into the rate limit the cooldown is trying to prevent.
**Backend (`/api/sources/<id>/check`):**
- Reads optional `?force=true` query flag.
- Without force: queries `active_platform_cooldowns` (renamed from the
private `_platforms_in_cooldown` since it's now a cross-module API).
If the source's platform is in cooldown, returns **202** with
`{status: 'deferred', platform, cooldown_until}` — no event created,
no dispatch.
- With force: cooldown skipped entirely.
- In-flight guard always applies (no point creating duplicate pendings).
**Frontend (`sourcesStore.checkNow(id, {force=false})`):** new optional
`force` flag → adds `?force=true` to the URL.
**Frontend (`DownloadsTab`):**
- `onRetrySource` (single-source RETRY click): passes `force: true` →
explicit operator override, useful for rapid auth-fix testing.
- `onRetryAll` (RETRY ALL + MaintenanceMenu "Retry failed"): no force →
cooldown respected. Tallies `deferred` alongside `queued` /
`already_running`; toast reads e.g. *"5 queued, 12 deferred
(cooldown), 3 already running"*. That count is the operator's
diagnostic answer for "is rate-limit the cause of most failures?"
(12-of-20 deferred → yes; 0 deferred → no).
**Auto-resume:** no new sweep needed. Deferred sources still have stale
`last_checked_at`, so the next scan tick after the cooldown AppSetting
expires picks them up via `select_due_sources` (which already filters
on `active_platform_cooldowns`).
Tests: two new — deferred-on-cooldown returns 202 with the right body
and no dispatch; force=true overrides the cooldown and creates the
event normally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-flagged 2026-05-30 (round 3): the all-parallel fetch was fast
but could let later chunks arrive ahead of earlier ones — even when
each chunk is a random sample, that "later chunk loads first" risk made
the load order non-deterministic. And the original goal behind asking
for batching was a faster first-image-on-screen, which neither sequen-
tial nor parallel really addressed cleanly.
Switched the loadInitial flow to a PIPELINE:
- Only one fetch in flight at any moment (in-order arrival, no race).
- The NEXT fetch kicks off as soon as the current one resolves (NOT
after its trickle finishes), so the next RTT overlaps the visible
trickle window — round-trips are hidden behind the animation cadence.
- PAGE 5 → 3 + INITIAL_BATCHES 12 → 20 (total still 60). Smaller chunk
→ first chunk's items appear sooner (a chunk of 3 trickles in 240ms,
well within one RTT, so by the time chunk 2 is in-hand the first
trickle is just finishing).
Trickle, sequence-token guard, and infinite-scroll behaviour unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-flagged 2026-05-30 (round 2): the batched-loads change improved
consistency but still felt "chunky" — 5 items appeared together, then a
~200 ms API round-trip pause, then another 5. And infinite-scroll fired
too late when a single tall image (long manga page) made one masonry
column much taller than its siblings.
**Cadence (showcase store):**
- Fire all INITIAL_BATCHES fetches in PARALLEL — collapses the per-
batch round-trip gap so all the data arrives in ~1 RTT instead of 12
sequential RTTs.
- Trickle each response's items into images.value one at a time with
APPEND_DELAY_MS = 80ms between each (≈ the MasonryGrid stagger
animation, 70ms). User sees a smooth steady stream.
- fetchPage (the infinite-scroll path) uses the same trickle so its
5-item appends also cascade one-by-one instead of popping together.
- Sequence token guards against a fast shuffle / mount-then-shuffle
interleaving two trickles into the same images.value.
- Dropped useAsyncAction here — the parallel-fetch-then-trickle flow
doesn't fit its single-wrap-call shape cleanly; inline loading/error
state is clearer.
**Infinite-scroll trigger (MasonryGrid):**
- Pass `rootMargin: '2400px'` to useInfiniteScroll (was the 600px
default). The masonry sentinel sits at the bottom of the container,
whose height = MAX(column heights). A tall image in one column pushes
the sentinel ~2× viewport below where the user is actually reading
(the bottom of the SHORTER columns). 2400px ≈ 2-3 screen-heights of
pre-emptive trigger, comfortable for typical tall manga heights.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Every archive import was failing immediately with "AssertionError:
daemonic processes are not allowed to have children" (operator-flagged
2026-05-30 — import_archive_file crashes in 49ms with the assertion).
Celery's prefork pool runs tasks in daemon processes; Python's
multiprocessing module refuses to let daemons spawn children, which is
exactly what probe_archive was doing via mp.get_context("spawn")
.Process. The Layer-3 crash-isolation feature added 2026-05-28 was
effectively a hard-blocker on the very import path it was meant to
protect.
Switched probe_archive to subprocess.run — no daemon restriction, still
isolates the probe (a probe segfault/OOM exits non-zero, doesn't kill
the worker). The probe body lifted to a tiny runner module
(_archive_probe_runner) that imports the unchanged _run_probe helper
and prints a single JSON line; parent parses stdout, returns the
ProbeResult exactly as before (timeout, signal, OOM, clean-rejection,
ok — all preserved).
cwd for the subprocess is the repo root derived from __file__ parents
so `python -m backend.app.utils._archive_probe_runner` resolves both in
the Celery container and pytest, regardless of where the worker was
launched.
Test refactor: test_archive_probe_target_bomb_guard →
test_run_probe_bomb_guard. Same in-process call to the (renamed) probe
body so the monkeypatched cap still takes effect; the real subprocess
path is exercised by the existing test_probe_archive_valid_zip /
test_probe_archive_corrupt_zip_clean_rejection tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The showcase/gallery/artist/series/post-feed APIs were constructing
thumbnail URLs from (sha256, mime). The MIME-based extension predicate
("png if image/png or image/gif else jpg") DISAGREED with the
thumbnailer's actual on-disk extension predicate ("png if alpha else
jpg"). Result: every PNG source without transparency 404'd (URL asked
.png, disk had .jpg); every WebP/AVIF source with transparency 404'd
(URL asked .jpg, disk had .png) — despite the thumbnail file existing
on disk.
The backfill task couldn't catch these because backfill checks the
ACTUAL thumbnail_path stored on the record (correct), not the URL the
browser fetches (broken derivation). So records with valid on-disk
thumbnails kept showing as broken in the UI no matter how many times
backfill ran.
Operator-flagged 2026-05-30: "the generate thumbnails function appears
to not catch all of the failed thumbnail cases" — turned out to not be
a backfill bug at all.
Fix: thumbnail_url now takes (thumbnail_path, sha256, mime) and returns
the stored path verbatim — Quart serves /images/* 1:1 from the volume
(frontend.py:20-36), so the URL IS the disk path. Falls back to the old
sha256+mime derivation only when thumbnail_path is NULL (thumbnailer
hasn't run yet); that URL will 404 in the browser until backfill catches
it, same as before the path was tracked.
All 8 callers updated: showcase_service, gallery_service (2 sites),
artist_service, series_service, post_feed_service, tag_directory_service,
artist_directory_service. The four sites whose query was raw-tuple now
also SELECT ImageRecord.thumbnail_path.
Net effect: every record that has a valid on-disk thumbnail will now
render correctly, regardless of which extension the thumbnailer chose,
without any DB migration or backfill rerun needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The showcase-store change in adeee64 didn't actually land — only
gallery.js + ShowcaseView.vue made it into the commit. Without this,
ShowcaseView mounts loadInitial() which doesn't exist yet, so the
showcase blows up. Landing the matching store edit now.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Showcase and gallery were fetching the full initial batch (60 / 50
items) in a single API call. Operator-flagged 2026-05-30: items should
stream in as small batches so they render progressively rather than
blocking on one big response.
- showcase store: PAGE 60 → 5, INITIAL_BATCHES = 12 (5 × 12 = 60, same
total). loadInitial() fetches PAGE chunks in sequence; shuffle()
delegates. fetchPage() still returns one PAGE-sized chunk so
infinite-scroll also pulls 5 per trigger.
- gallery store: loadMore() limit 50 → PAGE (5), INITIAL_BATCHES = 10.
loadInitial() loops loadMore() up to INITIAL_BATCHES times, stopping
early when nextCursor becomes null (end of data).
- ShowcaseView: onMounted calls loadInitial() instead of fetchPage().
Gallery/setTagFilter/setPostFilter already routed through loadInitial.
Net visual effect: each ~5-item batch lands and renders independently;
combined with MasonryGrid's animateFromIndex logic, the showcase now
cascades batches in as they arrive instead of one big pop-in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-flagged 2026-05-30: "the fail state of timeouts doesn't show
anything other than that the task timedout and was cleaned up. I can't
tell why it ran over or if it was stuck failed or there was just that
much to get."
The TimeoutExpired branch was returning a DownloadResult with no stdout,
no stderr, no files_downloaded, and a generic "Download timed out after
N seconds" message — even though subprocess.TimeoutExpired carries the
partial output gallery-dl emitted before being killed.
Now:
- Capture e.stdout / e.stderr (coerced str if bytes; "" if None).
- Count files_downloaded from partial stdout via _count_downloaded_files.
- Surface a tail-of-stderr hint in error_message so the UI summary tells
the operator at a glance whether it was "lots of content" (high count,
clean stderr), "stuck retrying" (any count, 429-spam stderr), or "hung
silent" (zero count, "no stderr output").
- Promote error_type to RATE_LIMITED when the partial stderr matches
RATE_LIMIT_PATTERNS — gallery-dl spinning on retries through the whole
900s window is the timeout-shaped tail of a real rate limit, and the
platform cooldown should kick in for the same reason.
Existing test_download_timeout strengthened to also assert empty-partial
case stays correctly TIMEOUT-classified with no preserved output.
New test_download_timeout_preserves_partial_output_and_classifies covers
the rich-partial-output → RATE_LIMITED promotion path.
DownloadEvent.metadata already flows stdout/stderr/run_stats from
DownloadResult via _phase3_persist — no UI change needed; the existing
DownloadDetailModal will surface the captured output automatically once
the build redeploys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
select_due_sources returned rows in undefined order (Postgres-determined,
typically PK). At tick rates that outpace download-queue throughput, a
freshly-rerun source could keep getting re-queued ahead of one that's
still waiting for its first attempt this cycle. Operator-flagged
2026-05-30:
> if there are 8 hours before a source is due again and 40 full time
> downloads can happen in that period that means that there's a chance
> the first one to fire gets back into the download queue before item 41
> has a chance to get downloaded.
Added `ORDER BY last_checked_at ASC NULLS FIRST, id` to the due-source
SELECT. Never-checked sources go first, then longest-since-checked, then
ties broken by id. Combined with Celery's FIFO `download` queue, the
oldest-overdue source in each tick now reaches a worker before any
fresher one.
Test pins the ordering: a NULL-last_checked source, a 4-hour-overdue
source, and a 2-min-overdue source come back in that exact order from
select_due_sources.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
61ce1ce added platform_cooldowns to scheduler_status(), and two pinned
key-set assertions tied to the old shape failed in CI. Updated:
- tests/test_api_sources.py::test_schedule_status_shape — direct
/api/sources/schedule-status response.
- tests/test_api_system_activity.py::test_summary_returns_rollup_shape
— nested under body["scheduler"] in the summary endpoint.
[[feedback-plan-grep-pinned-tests]] miss — should have grepped tests/
for `last_tick_at` / `auto_sources` before adding the new key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The scan tick fired download_source.delay() for every due source without
grouping by platform; with multiple download workers, N due Patreon
sources could all hit Patreon's API in parallel and rate-limit each
other. Per-source consecutive_failures backoff REACTS to that (slows the
offender across cycles) but didn't PREVENT the first-tick burst.
When DownloadService._update_source_health sees a source error
classified as ErrorType.RATE_LIMITED, it now stamps an AppSetting row
`platform_cooldown:<platform>` with the cooldown expiry (now + 15 min,
PLATFORM_RATE_LIMIT_COOLDOWN_SECONDS). select_due_sources queries every
platform_cooldown:* key at the start of each tick and excludes every
source whose platform is in active cooldown. scheduler_status surfaces
active cooldowns as platform_cooldowns: {platform: expires_iso} so the
TopNav pipeline chip / activity summary can display them.
INSERT...ON CONFLICT DO UPDATE for the upsert so two workers racing
RATE_LIMITED responses on the same platform don't let one's
IntegrityError roll back the other's event-finalize transaction
(stranding the event for the recovery sweep). Atomic at the SQL level.
Tests cover: select_due_sources skips a platform in cooldown; other
platforms unaffected during single-platform cooldown; expired cooldown
rows don't filter; set_platform_cooldown is upsert-safe under repeated
calls.
Operator-flagged 2026-05-30 ("running multiple workers I don't know how
we'd keep the downloader from hitting a rate limit on a source").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SourceConfig.timeout defaulted to 3600s (1 hour), but download_source's
Celery task has soft_time_limit=900s and hard time_limit=1200s. So
gallery-dl never hit its own subprocess.run timeout — Celery always
killed it first. The hard SIGKILL leaves no terminal flip on the
DownloadEvent, which then sat pending/running until the recovery sweep
flipped it to error at 30 min from start. From the operator's seat that
was a ~30–40 min "hang" on every retry of a broken source.
Pinning the default to 900s (matching Celery's soft_time_limit) lets
subprocess.run raise TimeoutExpired cleanly inside Celery's window, and
the existing `except subprocess.TimeoutExpired` branch
(gallery_dl.py:635) captures it as a clean error_type='timeout' with a
real message. The DownloadEvent flips to error in ~15 min instead of
waiting on the recovery sweep at 30.
Per-source bumps still live in source.config_overrides for legitimately
long first-syncs. The new _DEFAULT_GDL_TIMEOUT_SECONDS constant carries
the rationale and the Celery-soft-limit dependency in its comment.
Operator-flagged 2026-05-30 on 59-source strand pile retries.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The maintenance dropdown in Subscriptions → Downloads was wired to the
filesystem-import pipeline (POST /api/import/retry-failed +
POST /api/import/clear-stuck) — the subtitles even said so ("Re-enqueue
every failed import task"), but it was contextually misplaced. From the
Downloads view "Retry failed" queued nothing the operator could see
because the action operated on import_task rows, not download_event
rows. Import-pipeline maintenance is already reachable from Settings →
Imports (ImportTaskList.vue), so removing the import wiring loses
nothing.
Rewired:
- "Retry failed" → bulk-retries the failing-sources list, same loop as
FailingSourcesCard's RETRY ALL (sourcesStore.checkNow per source).
Subtitle now matches: "Re-queue every currently failing source".
- "Force recovery sweep" → triggers recover_stalled_download_events on
demand via a new POST /api/downloads/recover-stalled endpoint. The
sweep also runs every 5 min on Beat; this is the manual fallback so
the operator doesn't have to wait for the next tick to clear newly
stranded events.
MaintenanceMenu is now stateless — emits retry-failed and recover-
stalled. DownloadsTab owns the handlers (reuses the existing
onRetryAll; new onRecoverStalled with a delayed refresh so swept rows
land in the failing rollup).
Operator-flagged 2026-05-29 — "the retry failed button in the
maintenance dropdown doesn't appear to queue anything but manual
requeues works."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
104cac5's override sits at the same specificity as Vuetify's default
(`.v-tooltip > .v-overlay__content` on both sides). The fix relied on
source order — app.css imported after vuetify/styles in main.js — but
Vite's production bundler reorders node_modules CSS into the final
stylesheet unpredictably, so source order isn't a reliable winner.
!important on the two contrast properties (background + color) forces
the slate-on-parchment pair regardless of stylesheet load order. The
cosmetic border + shadow don't need it (Vuetify doesn't set them, so
nothing's competing).
Operator re-flagged 2026-05-29: tooltips still rendered light-on-light
on the deployed :latest after PR #33 — 104cac5 was in the bundle but
not winning. Also updated the file header so the source-order claim
isn't carried forward as gospel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The scan tick (scan.py:_tick_due_sources_async) inserts
DownloadEvent(status='pending') and fires download_source.delay(). If the
task dies before finalizing the event — worker OOM/SIGKILL, lost task, or
a gallery-dl that didn't unwind on the 1200s hard time_limit — the event
stays in-flight forever. Every later tick then skips the source via the
in-flight guard (scan.py:168), so Source.last_checked_at is never written
and the operator sees "last check never" in the Subscriptions health
column, permanently.
cleanup_old_download_events only prunes terminal events (by design); no
existing sweep covered the pending/running case. Operator confirmed
2026-05-29 with a diagnostic query: all 43 "never checked" sources were
stranded behind stale in-flight events (eligible_stuck_inflight = 43,
every other bucket zero).
New recover_stalled_download_events task (Beat every 5 min):
- Flips DownloadEvent rows pending/running > 30 min (10 min past the
download_source 1200s hard kill, so legitimately-running tasks are
never touched) to status='error' with a sentinel message.
- Bumps each affected Source's consecutive_failures ONCE per source —
backoff is 2^N on that counter so per-event bumps would needlessly
inflate the next interval — sets last_error, stamps last_checked_at.
UPDATE...RETURNING source_id avoids a SELECT-then-UPDATE-WHERE-IN that
would hit the psycopg 65535-param ceiling on a large strand pile.
Net: the 43 currently-stranded sources unstick on the first sweep after
deploy, their health dots flip amber instead of unchecked, and the next
scan tick re-queues them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The plain-JS frontend re-declares two backend enum value-sets by hand:
download-event statuses (downloadStatus.js) and platform keys
(platformColor.js). With no TS codegen, drift is silent — the same class
as the last_verified_at field mismatch.
tests/test_fe_be_contract.py parses both JS mirrors and asserts they equal
the backend canon (downloads._ALLOWED_STATUSES, platforms.known_platform_keys())
so a change on either side fails CI on whichever moved. Backend is the
source of truth. Documented the invariant in both JS file headers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The IntersectionObserver-on-a-sentinel infinite-scroll pattern was copy-
pasted in 7 places. New composables/useInfiniteScroll(sentinelRef, cb)
owns the observer lifecycle (attach on mount, re-attach when the ref
changes, disconnect on unmount). Migrated GalleryGrid, MasonryGrid
(gallery+showcase), ArtistPostsTab, ArtistsView, TagsView, SeriesManageView.
PostsView left manual on purpose — its anchored mode does bidirectional
scroll-position preservation that doesn't fit the simple composable.
I4(a) (timestamps) is effectively already done: the UTC displays were
converted earlier; the remaining toLocaleString uses already render local
time, so they're left as-is rather than churned for format-only consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New /api/system/activity/summary aggregates scheduler health + per-queue
pending depths + running count + 24h failure count in one cached call (safe
to poll app-wide). PipelineStatusChip lives in the TopNav on every page: a
compact running/queued/failing chip with a scheduler-health dot that expands
to a popover (scheduler, busy queues, counts, link to downloads). Polls the
summary every 8s, paused when backgrounded. Reuses the queue-cache read via
_queues_cached(). + API test for the summary shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The row template renders status + platform (PlatformChip) + time + counts;
the artist isn't displayed there. Assert on 'Completed'/'Patreon' instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds @vue/test-utils + happy-dom and mounts CredentialCard, DownloadEventRow,
PostCard, ActiveDownloadsPanel, QueueStatusBar, asserting they render without
throwing and surface key content — catching the dead-binding / render-error
class (e.g. the last_verified_at regression, guarded explicitly). Vuetify
components are left unresolved (Vue renders unknown elements + slots), so no
Vuetify-plugin setup is needed; only RouterLink is stubbed. Per-file
happy-dom env via docblock keeps the existing node-env specs untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ruff is pre-installed in the ci-python image, so a new `lint` job runs it
with no dependency install and fails in seconds — surfacing the common lint
bounce class without waiting on the backend job's ~30-60s wheel install.
Dropped the now-redundant ruff step from backend-lint-and-test (same job
name, required-checks unchanged).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Provenance "View post" deep-links to /posts?post_id=X, which now opens the
feed centered on that post with infinite load in BOTH directions.
Backend: PostFeedService.scroll gains a direction (older|newer); new
around(post_id) returns a window of newer + the post + older with a cursor
for each end. /api/posts accepts ?around= and ?direction=. + API tests.
Frontend: posts store gains loadAround/loadOlder/loadNewer (older appends,
newer prepends) with per-end cursors; PostsView's anchored mode scrolls to
the post, observes top + bottom sentinels, and preserves scroll position on
upward prepend so the page doesn't jump. Normal feed mode unchanged.
Closes the remaining half of the post-navigation work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each maintenance button (ML backfill, centroid recompute → ml queue;
thumbnail backfill → thumbnail queue) now shows a status bar with the live
pending count for its Celery queue, so the operator can see work is already
queued/running before re-triggering and piling on. MaintenancePanel polls
/api/system/activity/queues every 4s; QueueStatusBar reads the depth and
turns warning-colored when the queue is busy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New ActiveDownloadsPanel pinned at the top of the Downloads tab shows what's
running (pulsing dot + live mm:ss timer counting from started_at) and what's
queued, so the operator can see activity at a glance without the running
filter. Backed by store.loadActive() which fetches running + pending
independent of the feed filter; refreshed every 4s by the existing live poll.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Download event times showed raw UTC wall-clock (iso.slice). Added
formatDateTime()/formatLocalDate() (local tz, robust to naive vs tz-aware
ISO) and applied them to the download row + detail modal datetimes and the
credential/artist date displays. formatPostDate stays UTC (date-only,
locale-stable, unit-tested).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaced the subtle 12px fade with a perspective rotateX(-28deg) tilt that
flips up and settles flat with a slight overshoot, staggered 70ms so tiles
cascade in one at a time. Makes the showcase read as an experience rather
than a quiet fade. Tunable (tilt/stagger/duration); reduced-motion safe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FC already matched IR's staggered entry fade-in; this adds the missing
piece — hover zoom (scale 1.03) + brighten (1.1) on each thumbnail, with
overflow:hidden so the scaled image clips to the rounded card. Disabled
under prefers-reduced-motion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Post titles arrived as stored HTML (e.g. <strong>…</strong>) rendering as
literal markup. New toPlainText() strips tags; titles render plain + bold
(ProvenancePanel and PostCard).
- Removed "View original post" from the provenance panel (modal) — the
open-original button lives on the PostCard (the post view).
- Provenance "View post" now navigates to the /posts feed (post_id query),
not the gallery image grid. (Feed in-context landing lands next.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The whole view scrolled instead of just the subscription list. Made the
hub a viewport-height flex column (tabs stay fixed) with the v-window as
the single internal scroll container; the per-tab sticky control bars now
pin to the window top (top:48px -> 0). Operator-flagged 2026-05-28.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ArtistView and SubscriptionsView both two-way-bound a tab ref with the
?tab= query param via identical tab->URL and URL->tab watchers. Extracted
useTabQuery(validTabs, defaultTab) — defaultTab accepts a string or a
function (ArtistView's default is postCount-dependent), and resolve() is
exposed so ArtistView can re-apply it after the artist loads. Both views
drop their now-orphaned ref/useRouter imports.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_find_or_create_source, _source_for_sidecar, and _find_or_create_post each
repeated the SELECT → savepoint-INSERT → on-IntegrityError rollback+re-SELECT
pattern. Extracted _get_or_create(stmt, factory): the statement is reused for
the scalar_one_or_none lookup and the scalar_one post-conflict re-fetch, so
all three are reproduced exactly. Centralizing the race-safe pattern in one
place also reduces the risk of the copies drifting (the bug class banked
2026-05-26).
Left _upsert_artist (no savepoint by design) and the ImageProvenance void
ensure-exists block (no return / no re-select) alone — they don't fit.
The rest of the ingest pipeline was already DRY: sidecar parsing lives in
utils/sidecar.py, per-platform quirks in the platforms package, and
_safe_ext/_categorize_error/_build_config are each single-instance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `select(ImportSettings).where(id == 1)).scalar_one()` singleton load was
repeated 15× across services, API, and 5 task modules. Added async load() +
sync load_sync() classmethods on the model and migrated all 15 full-row sites
(callers already imported ImportSettings, so no new imports; dropped download's
now-orphaned select import). Left maintenance.py's deliberate column-select
(import_scan_path only) as-is.
Rest of the service layer was already adequately DRY — the Record/to_dict
pattern is only 2 instances and the savepoint find-or-create recovery is
correctly per-entity, so neither was forced into a shared abstraction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
download/migration/scan each defined an identical _async_session_factory()
(fresh per-invocation async engine — async connections are event-loop-bound
so each asyncio.run() task needs its own engine, unlike the process-wide
_sync_engine). Moved it to tasks/_async_session.py; the 3 files import it
and drop their now-orphaned sqlalchemy.ext.asyncio / get_config imports
(migration keeps AsyncSession for a type hint). Call-site try/finally
dispose left as-is to avoid re-indenting the critical task bodies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8 blueprints each defined an identical _bad() (two variants: with/without
detail). Extracted error_response() into api/_responses.py; each blueprint
now imports it `as _bad` so call sites are unchanged. The detail-aware
canonical subsumes both variants. Left settings.py's distinct _bad_int and
the inline jsonify error sites (not duplicated helpers).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New composables/useAsyncAction.js owns the loading/error/try-finally
lifecycle. Migrated 11 stores: credentials, downloads, sources, posts
(error=raw) + ml, artistDirectory, tagDirectory, showcase, suggestions,
seriesReader, modal (error=message). The errorAs option preserves each
store's existing error shape so store.error keeps the same type for
components (~50 consumption sites unchanged). Stores whose catch also
reset data (suggestions/seriesReader/modal) clear it upfront instead.
Deliberately NOT migrated (special control flow, would change behavior):
artist (conditional 404 catch + dual loading states), migration (rethrows),
gallery (inflight-id stale-response guard), and the Shape-B no-catch /
loaded-guard / keyed-cache stores.
Net -77 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removing the create_app import left 2 blank lines before pytestmark in 9
files where ruff isort wants 1. Also stripped two pre-existing
whitespace-only blank lines surfaced by the file change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removed the app/client fixtures duplicated across 36 test files (two
variants: separate app + client(app), and a self-contained client() that
called create_app inline) and the now-unused create_app imports. Both
fixtures now live once in conftest.py. test_suggestions_bulk keeps its
import (builds the app inline in two tests); test_health drops its local
client + unused pytest_asyncio.
Net -415 lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
D1 scheduler visibility: AppSetting last-tick stamp on every Beat tick +
GET /api/sources/schedule-status (last_tick_at/next_due_at/due_now/auto_sources)
+ SchedulerStatusBar on the Subscriptions tab (re-polled every 30s).
D2 failing-source rollup: ?failing=true on the sources list + FailingSourcesCard
on Downloads with per-source and bulk "retry" (re-runs the feed via /check).
D3 activity sparkline: GET /api/downloads/activity hourly buckets + CSS bar
chart by the stat chips (failures stacked in error color); refreshes on live poll.
D4 credential staleness: surface last_verified age + "re-verify recommended"
warning past 30d; also fixes the dead last_verified_at field-name mismatch so
the verification row renders at all.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-flagged 2026-05-28: the Subscriptions/Downloads dashboards were
full-bleed and thin on filtering. Chosen via AskUserQuestion.
**Layout**
- SubscriptionsView capped at a centered max-width 1600px (covers all
three subtabs) so rows aren't a mile wide on ultrawide monitors.
- Sticky control headers on both tabs (top: 48px, below the top nav,
opaque bg) so filters/stat-chips stay reachable while scrolling a
long list.
**Downloads filters (all four requested)**
- Stat chips are now clickable filters: click Queued/Running/Completed/
Failed/Skipped to filter the list to that status; the active chip is
outlined + elevated; re-click clears.
- Free-text search box over the loaded events (artist / platform /
error substring).
- Artist filter: the filter popover's numeric "Source ID" field is
replaced with an artist autocomplete (sources.autocompleteArtist →
artist_id, which /api/downloads already supports). Pill shows the
artist name.
- "Show no-change scans" toggle (default OFF): hides status=ok/skipped
rows with 0 files (the scheduled scans that found nothing) so real
downloads + failures stand out.
**Subscriptions**
- "Needs attention" quick-filter chip: one click to show only artists
with sources that have errors OR have never been checked; chip shows
the count and disables the status dropdown while active.
Frontend-only — backend filter params (status/artist_id/date) and the
/api/downloads endpoint already supported everything.
Operator-flagged 2026-05-28: the download event detail modal had no way
to copy the error / stdout / stderr text for researching a failure
(parity with the ErrorDetailModal Copy button shipped in v26.05.26.0).
Added per-block Copy buttons (Error, Errors & warnings, Raw stdout, Raw
stderr — the stdout/stderr ones sit in the expansion-panel title with
@click.stop so they don't toggle the panel) plus a "Copy all
diagnostics" button in the footer that assembles a single block: header
line (event id / platform / artist / status / timestamps) + error +
errors&warnings + full stdout + stderr, ready to paste into an issue.
All routed through utils/clipboard.js copyText() — the navigator.clipboard
→ execCommand fallback that works on the plain-HTTP homelab origin
(per feedback_no_secure_context_apis). Each copy shows a confirmation
toast.
Operator-flagged 2026-05-28, two asks.
**1. Credential Verify (was missing vs GS — and now actually verifies).**
GS's Verify was a stub (`TODO: implement actual verification` — just
stamped last_verified). FC does a real check, which matters given the
recent auth pain (subscribestar age cookie, HF host-only PHPSESSID):
- GalleryDLService.verify(url, platform, cookies_path, auth_token) runs
gallery-dl in `--simulate --range 1-1` mode (no download) against the
URL with the materialized credentials, then reuses _categorize_error:
returncode 0 / NO_NEW_CONTENT → valid; AUTH_ERROR → invalid; other →
inconclusive (reason surfaced). 45s timeout.
- POST /api/credentials/<platform>/verify picks an enabled Source for
the platform to probe, runs verify, and on success stamps
credential.last_verified (new CredentialService.mark_verified).
Returns {valid: bool|null, reason, last_verified?}. valid=null means
untestable (no credential, or no enabled source to point at).
- CredentialCard gains a Verify button (on credentialed cards) + a
result chip (Verified ✓ / Failed / Untestable) and a toast with the
reason. SettingsTab reloads on @verified so last_verified refreshes.
**2. Live download-activity feedback.** The Downloads tab was static —
no way to tell if downloads were succeeding without manually hitting
Refresh. It now auto-polls: stats every 4s, and the event list too
while anything is queued/running. Polling pauses when the tab is
backgrounded (document.hidden) and the list reload is skipped on idle
ticks to stay light. A pulsing "● live" indicator next to the stat
chips shows when auto-refresh is active (queued+running > 0); honors
prefers-reduced-motion.
Tests: verify endpoint — untestable with no credential, untestable with
no enabled source, valid+stamped on success (gallery-dl mocked), and
auth-failure reported without stamping.
Operator-confirmed 2026-05-28: the BlenderKnight:* tags are `archive`
kind (caught by the kind purge), but the `source:patreon`-style tags
are IR's old `source` kind that fell back to `general` during migration
(FC's enum has no `source` kind) — so they can't be matched by kind.
Broadened the purge to a two-rule match and renamed it for accuracy
(all dev-only, unreleased):
- cleanup_service.purge_tags_by_kind → purge_legacy_tags. Predicate is
now `kind IN (archive, post, artist) OR name LIKE 'source:%'`
(LEGACY_NAME_PREFIXES). Preview classifies each row into by_kind OR
by_prefix (source:* counts once under the prefix bucket regardless of
its general kind).
- endpoint /tags/purge-retired-kinds → /tags/purge-legacy. dry-run
returns by_kind + by_prefix + count + sample.
- store purgeRetiredKindTags → purgeLegacyTags.
- Tag Maintenance card copy + breakdown updated to show both buckets;
button reads "Preview/Delete legacy tags".
Tests updated + extended: dry-run reports by_kind {archive,post,artist}
AND by_prefix {source:*}, plain general/character tags survive; commit
deletes both the kind-matched and source:*-matched rows and leaves the
rest.
Operator-flagged 2026-05-28, two asks.
**1. Provenance posts ate the panel.** Each post card rendered its full
description (180px scroll box) inline, so a few posts pushed everything
else off-screen. ProvenancePanel now collapses the description by
default behind a per-post "Show description ▾ / Hide ▴" toggle (state
keyed by provenance_id, reset when the viewed image changes). Cards stay
compact — platform/date/title/meta/actions — and the operator expands
only the descriptions they want.
**2. Purge tags of retired/system kinds.** The IR migration left
`archive`/`post`/`artist`-kind tags (e.g. `BlenderKnight:Hannah_BJ_Loops`)
that FC no longer creates — the tag input only makes
character/fandom/series/general, and provenance + artists are their own
systems now. (meta/rating were already hard-deleted by alembic 0023.)
- cleanup_service.purge_tags_by_kind(kinds=PURGEABLE_TAG_KINDS) — counts
(dry_run) or deletes tags whose kind ∈ (archive, post, artist).
CASCADE clears the image_tag / alias / allowlist / etc. rows.
- POST /api/admin/tags/purge-retired-kinds (Tier-A, dry-run preview
returns per-kind counts + sample names — the preview IS the
verification of exactly what'll be deleted before committing).
- Tag Maintenance card gets a second section: "Preview retired-kind
tags" → per-kind breakdown + sample → "Delete N retired-kind tag(s)".
Tests: dry-run counts by kind (general survives), commit deletes only
the retired kinds (general + character survive, retired count → 0).
NOTE: a dry-run preview will show exactly which kinds/counts are
present. If the operator's noisy tags turn out to be `general` (e.g. an
IR `source:patreon` that fell back to general during migration), they
won't be caught by the kind purge — the preview makes that visible so
we can decide on a name-based pass separately.
Operator-flagged 2026-05-28: tooltips (e.g. the action buttons in
Subscriptions → Subscriptions) render near-white-on-near-white,
unreadable.
Cause: Vuetify's default v-tooltip pairs `on-surface-variant` text with
an `surface-variant` background. FC's theme deliberately maps
`on-surface-variant` to vellum (#C2BFB4 — a light cream, the correct
muted-text token for captions/hints on the dark page) but never defines
`surface-variant`, so Vuetify auto-generates a light-ish tooltip
background. Light text on light bg.
Fix is tooltip-specific so it doesn't disturb the (correctly light)
muted-text token elsewhere. New app-global stylesheet
frontend/src/styles/app.css, imported in main.js AFTER vuetify/styles
(equal-specificity rule wins by source order), overrides
`.v-tooltip > .v-overlay__content` to a dark elevated panel
(surface-bright = slate #2C313A) with high-contrast parchment text
(#E8E4D8) + a subtle border + shadow. Applies to every tooltip in the
app, so the fix is consistent rather than per-component.
Layer 2 — remediate a corrupt file by re-fetching a fresh copy from its
source, bounded to a single attempt. Operator-requested 2026-05-28.
New backend/app/services/refetch_service.py:
- resolve_refetch_source: parse the failed file's sidecar → platform,
derive the artist from the import path, find an ENABLED Source with a
real feed URL for (artist, platform). Returns None for filesystem-only
imports, missing sidecars, or `sidecar:<platform>:<slug>` synthetic
anchors (not pollable).
- attempt_refetch: if not already refetched AND a Source resolves,
delete the corrupt file (so gallery-dl's skip_existing re-fetches it),
set ImportTask.refetched=True, and trigger ONE download_source
re-check. Bounded by `refetched` so source-side corruption can't loop.
Wiring:
- Manual endpoint POST /api/import/tasks/<id>/refetch (only on 'failed'
tasks). Returns refetch_queued / no_source / already_refetched /
not_found / not_failed.
- Auto path in recover_interrupted_tasks: for each poison-pill row, if
env FC_AUTO_REFETCH_CORRUPT=1, attempt_refetch (default OFF — the
manual button is the primary path; auto is opt-in since re-fetch
deletes a file + re-runs the downloader).
- Frontend: a cloud-refresh icon button on failed rows in ImportTaskList
→ stores.import.refetchTask → toast keyed on the result status.
Filesystem imports with no upstream return no_source — the operator's
only remediation there is replacing the file on disk, surfaced clearly
in the toast.
Tests: 404 unknown task, 400 non-failed task, no_source when
unresolvable, and the full resolvable-source path (file deleted,
refetched flag set, one download_source dispatched, second call is a
no-op). The resolvable test repoints the migration-seeded
import_settings(id=1) scan path rather than inserting a conflicting row.
Layer 3 — prevent the hard worker crash rather than just recovering from
it. The realistic process-crash vectors (operator's observed slow/heavy
tasks) are video decode and archive extraction; images decode in-process
and Pillow raises-and-skips cleanly, and a subprocess per image would
wreck deep-scan throughput, so images are intentionally not probed.
New backend/app/utils/safe_probe.py (leaf module, lazy heavy imports so
the spawned child stays light):
- probe_video(path): validates the container + first video stream via
ffprobe (a separate binary — a decoder crash kills only ffprobe, not
the worker). Returns width/height, which the importer didn't capture
for videos before. crashed=True only on ffprobe timeout.
- probe_archive(path): an uncompressed-size bomb guard
(MAX_ARCHIVE_UNCOMPRESSED_BYTES = 4 GiB) plus the format integrity
test (zipfile.testzip / rarfile.testrar / py7zr.test) run in a
spawned child process. A decompression-bomb OOM or native-lib
segfault on a malformed archive shows up as a non-zero child exit
code → crashed=True, never a dead worker.
ProbeResult.crashed distinguishes a HARD failure (subprocess killed /
timed out — the poison-pill signature → caller returns terminal
'failed') from a CLEAN rejection (corrupt-but-handled, bomb cap,
integrity mismatch → caller's choice of skipped/attached).
Wired:
- importer._import_media video branch: probe_video before the pipeline;
crash → failed, clean reject → invalid_image skip, ok → capture dims.
- importer._import_archive: probe_archive before extract_archive; crash
→ failed, clean reject → still preserve the archive as a
PostAttachment (matches extract_archive's fail-soft contract).
- ml.tag_and_embed video branch: probe_video before sampling 10 frames,
so a corrupt video is rejected (status='bad_video') instead of
crashing the ml-worker on frame decode.
Tests (test_safe_probe.py): valid/corrupt zip via probe_archive, direct
_inspect_archive size+integrity, in-process _archive_probe_target bomb
guard (monkeypatch can't reach a spawned child, so the target is called
directly), and a non-video → ok=False that's robust to ffprobe presence
in CI.
Layer 1 of the import-task resilience work (operator-requested
2026-05-28). The recover_interrupted_tasks sweep re-queues rows stuck
in 'processing' — correct for a worker crash, but without a cap a row
that RELIABLY hard-crashes the worker (OOM/segfault/SIGKILL on a
corrupt or oversized input) loops forever: re-queue → crash → re-queue,
burning a worker slot every 5 min. A caught exception flips to terminal
'failed' and never enters this loop; only process-killing inputs do.
- alembic 0026: import_task.recovery_count (int, default 0) +
import_task.refetched (bool, default false — backs Layer 2).
- recover_interrupted_tasks now runs a poison-pill UPDATE FIRST: stuck
rows whose recovery_count has already reached MAX_RECOVERY_ATTEMPTS-1
are marked 'failed' with a diagnostic ("crashed or stalled the worker
N times … likely a corrupt or oversized input … inspect/replace the
file, then retry via /api/import/retry-failed") instead of re-queued.
The re-queue pass then handles the remaining stuck rows and bumps
recovery_count. Shared stuck_predicate (and_/or_) keeps the
media-5min / archive-40min split.
- MAX_RECOVERY_ATTEMPTS=3 (two recoveries then give up).
The failed poison pill surfaces in the existing import-failures view
with its file path, directly answering "help me identify them."
Test test_recover_interrupted_poison_pill_caps_at_max pins both
branches: a row at the cap is failed (not re-enqueued, diagnostic
present), a row one short is re-queued + incremented.
Operator-flagged 2026-05-28: import_media_file on target 1645019 hit
SoftTimeLimitExceeded at exactly 5.0 min. Their diagnosis was correct —
the timeout covered the WHOLE archive, not per object. Importer._import_archive
(importer.py:409) runs the full per-member pipeline (sha256 + pHash +
dedup query + copy + provenance) for EVERY media member inline, all
under import_media_file's single 300s soft limit. A single media file
is sub-second; a multi-hundred-member archive blows the budget. They
shared one task name and one timeout.
**Split archive into its own task**
- New `import_archive_file` task: same body as import_media_file
(dispatch is by file-kind inside Importer.import_one) but
soft=30min / hard=35min. Shared `_run_import_task` helper holds the
flip-to-processing + resilience-contract wrapper; both tasks call it.
- New `enqueue_import(task_id, task_type)` router — single source of
truth for media-vs-archive dispatch. Used by all three enqueue sites:
scan_directory, /api/import/retry-failed, recover_interrupted_tasks.
- scan_directory now sets ImportTask.task_type = "archive" when
is_archive(entry) (the model field already existed, anticipating
this; scan was hardcoding "media").
- import_archive_file routes to the existing 'import' queue via the
task_routes `import_file.*` wildcard — no worker config change.
**Archive-aware recovery sweeps**
Both sweeps would otherwise preempt a legitimately-running archive:
- recover_interrupted_tasks (ImportTask 'processing' sweep): now
task-type-aware. Media stays at STUCK_THRESHOLD_MINUTES (5); archives
get ARCHIVE_STUCK_THRESHOLD_MINUTES (40 = 5-min buffer past the
35-min hard limit). Single UPDATE with an OR predicate over the two
(task_type, cutoff) pairs; requeue routes via enqueue_import.
- recover_stalled_task_runs (TaskRun 'running' sweep): now supports
per-task-name overrides (TASK_STUCK_THRESHOLD_MINUTES) layered above
the per-queue overrides added for ml. import_archive_file gets 40 min
while the 'import' queue stays at the 5-min default for single-file
imports. Precedence: task_name → queue → default, each pass excluding
rows claimed by a higher-precedence pass so every row is touched once.
**Tests**
- test_import_archive_file_registered
- test_recover_stalled_task_runs_archive_task_uses_longer_threshold —
pins that a 10-min archive task-run survives, a 50-min one is flagged,
and a same-queue 10-min media import is flagged at the default.
- _make_task_run gains queue= + task_name= params.
After deploy: archive imports get a 30-min budget and aren't preempted
by either sweep; single-file imports keep their tight 5-min detection.
Operator-flagged 2026-05-28: tag_and_embed on image 6288 (an mp4) was
marked failed by recover_stalled_task_runs at the 5-min sweep tick
while still legitimately running. The error_type='RecoverySweep' /
"no completion signal received within 5 min" message was misleading
— the worker was busy, not stuck.
Root cause is two interacting limits, both undersized for video work:
tag_and_embed: soft_time_limit=300, time_limit=420
(sized for the image branch, ≈2 GPU ops)
recovery sweep: STUCK_THRESHOLD_MINUTES = 5 across all queues
The video branch samples 10 frames via ffmpeg, then runs tagger +
embedder on EACH frame — ~20 GPU ops vs 2 for an image. A loaded
ml-worker can take 5-10 min on a long video, which trips both
limits well before the task naturally finishes.
**Two-part fix**
1. `tag_and_embed` time limits bumped to soft=900 (15 min) / time=1200
(20 min). Sized for the video path's worst case; image runs return
in seconds and don't care.
2. New `QUEUE_STUCK_THRESHOLD_MINUTES` override dict in maintenance.py.
Queues with legitimately-long-running tasks (currently just `ml` at
25 min — 5-min buffer past the new hard kill) get their own
threshold; queues not in the dict use the default 5 min. The sweep
now issues one UPDATE per distinct threshold value, with
`queue.notin_(override_queues)` on the default pass so each row is
touched at most once.
Tests:
- _make_task_run helper accepts `queue=` (defaults to "default") so
existing tests use the default-threshold path.
- New test `test_recover_stalled_task_runs_ml_queue_uses_longer_threshold`
pins both directions: a 10-min-old ml row survives (fresh by 25-min
override), a 30-min-old ml row gets flagged.
After deploy, operator's mp4 ML jobs run to completion without
spurious RecoverySweep failures.
Operator-flagged 2026-05-27: the Downloads subtab "doesn't feel like a
dashboard" — status was a tiny mdi icon at the far left, platform chip
was neutral-tonal, errors were plain orange text floating on the right,
and all 28 rows from the same hour visually had the same priority.
**Row restyle (A):**
- 4px colored left-edge bar by status (success/error/info/warning/grey)
— visually scannable at the edge without parsing the chip text
- Status chip with text label (Completed/Failed/Running/Queued/Skipped)
+ leading icon, tonal-colored. Replaces the bare mdi-icon.
- Platform chip swapped to the color-coded subscriptions/PlatformChip
(Patreon=red mdi-patreon, SubscribeStar=amber, HentaiFoundry=purple,
Discord=indigo, Pixiv=blue, DeviantArt=green).
- File count: tonal info chip when > 0, dim middle-dot when 0 (so
scheduled "no-change" scans don't dominate the column visually).
- Error: red tonal pill chip with leading icon, truncated to 60 chars
with full text in the title tooltip. Replaces plain text.
- Per-row actions (hidden at 50% opacity, fade to full on row hover):
Retry (only when status=error AND source_id known — hits
POST /api/sources/<id>/check via the existing sources.checkNow),
Details (opens the detail modal), Open artist (navigates to the
artist page). Clicks stop-propagation so they don't bubble to the
row click.
**Date-grouped sections (B):**
- Events are bucketed into four sections: Today / Yesterday /
Last 7 days / Earlier. Empty buckets are skipped. Buckets boundaries
are computed against the operator's local-time start-of-day so
"Today" matches their intuition.
- Each section has a collapsible header with a row-count chip + a
red "failed in this section" chip when any failures are in scope.
- Within each section, status='error' rows are pinned to the top
(operator's eye lands on failures first; successful scans flow
below).
- Collapsed state persists across refresh within the SubscriptionsView
lifetime (reactive object, default all-expanded).
DownloadEventRow grid widened to accommodate the status chip + actions
column. PolyMasonry-style ellipsis on the artist link prevents long
names from breaking the layout.
No new endpoints; the Retry path reuses the existing /api/sources/<id>/check
flow (the source-check endpoint was already in place, just not wired
into a per-row button).
Operator-flagged 2026-05-27: walk the whole project for the same shape
as the min-dim Delete-button silent failure (crypto.subtle TypeError
on plain HTTP). FC runs over plain HTTP per the homelab posture;
Secure-Context-gated browser APIs are undefined on the production
origin.
**Audit results across `frontend/src/`:**
crypto.subtle.digest — 2 sites:
- MinDimensionCard (fixed 2026-05-27)
- BulkEditorPanel (THIS FIX)
navigator.clipboard — 1 site, already guarded:
- utils/clipboard.js writeText with execCommand fallback
serviceWorker / mediaDevices / Push / Web USB|HID|Bluetooth|Serial /
cookieStore / queryLocalFonts / WebAuthn / geolocation
— NOT USED, nothing to fix
Extension scripts (background.js) use crypto.subtle but run from
moz-extension:// which IS a Secure Context — left as-is.
**BulkEditorPanel double bug**
The bulk-delete UI on the gallery selection had been broken since
FC-3k shipped, in two ways:
1. `crypto.subtle.digest` swallowed TypeError on plain HTTP — modal
never opened. Same symptom as min-dim.
2. Even on HTTPS, the modal's `kind="images-selection"` produced
`delete-images-selection-<sha8>` while the backend expected
`delete-images-<sha8>`. The two would never match.
Fix:
- Backend `/api/admin/images/bulk-delete` dry-run response now returns
`confirm_token` (the canonical `delete-images-<sha8>` string).
Integration test `test_bulk_delete_dry_run_returns_counts` pinned to
assert the new field.
- DestructiveConfirmModal gains an `expectedTokenOverride` prop. When
set, it bypasses the `${action}-${kind}-${runId}` formula and uses
the explicit string. This decouples the UI label (`kind`) from the
wire-format token (server-provided), so future endpoints can use a
kind-specific label without their kind name leaking into the token.
- BulkEditorPanel passes `:expected-token-override="bulkProjected?.confirm_token"`
— no client-side crypto, no kind-prefix mismatch.
- MinDimensionCard refactored to the same explicit pattern (was
slicing the 8-char suffix off the backend's token and passing it
through `runId`; now passes the full backend token via
`expected-token-override` directly). Cleaner; one source of truth.
**Banked memory**
`feedback_no_secure_context_apis.md` documents the full table of
Secure-Context-gated APIs, which ones FC currently uses, and how each
is handled. Indexed in MEMORY.md. Sites for the audit also listed in
the memory for future drift-checking.
No other Secure-Context-gated APIs found in `frontend/src/`. The same
shape won't recur unless someone adds a new dependency on one — at
which point the banked memory should fire.
**showcase R-key + entry animation**
Restores two behaviors lost during the FC-2 IR→Vue port. Operator-flagged
2026-05-27.
- ShowcaseView listens for keydown 'r'/'R' on window. Triggers
`store.shuffle()`. Skips when an input/textarea/contenteditable is
focused or a Vuetify overlay is open (the dialog/menu sets
`.v-overlay--active` on the body).
- MasonryGrid gains an opt-in `animateFromIndex` prop (default
`Number.POSITIVE_INFINITY` = off). When set, items with index ≥ the
threshold animate in with a stagger fade-in: 12px translateY,
0.25s ease, 60ms per item, capped by `prefers-reduced-motion`.
Stagger uses original-items-array index (resolved via an `idxById`
Map) so the reading order is preserved even after the masonry
distributes items across columns.
- ShowcaseView watches `store.images.length`: shrink-or-zero baseline
⇒ `animateFromIndex=0` (animate everything on initial load /
shuffle); grow ⇒ baseline=prevCount (animate only the appended
tail on infinite-scroll). Other MasonryGrid consumers (ArtistView's
Gallery tab) don't pass the prop, so they keep their current
no-animation behavior.
Direct port of IR's `app/static/js/showcase.js` keyboard handler +
`app/static/style.css` itemFadeIn keyframe.
**min-dim Delete: crypto.subtle TypeError fix**
The Delete button on the Cleanup → Minimum Dimensions card was
silently no-op'ing. Root cause: `crypto.subtle` is Secure-Context-gated
(undefined on plain-HTTP origins per the homelab posture). The card's
`onDeleteClick` computed the Tier-C confirm token via
`crypto.subtle.digest('SHA-256', ...)`, which threw TypeError before
`showModal.value = true`. The promise rejected, the click handler had
no `.catch`, the modal never opened — exactly the operator's reported
symptom.
Same shape as the v26.05.26.0 `navigator.clipboard` fix on the
ErrorDetailModal Copy button.
Fix: backend `/api/cleanup/min-dimension/preview` now returns
`confirm_token` (the canonical `delete-min-dim-<sha8>` string) in its
response. Frontend reads it from the preview response and feeds the
8-char suffix to DestructiveConfirmModal's `runId` prop — no
client-side crypto needed. Single source of truth.
Integration test `test_min_dimension_preview_returns_count` pinned to
also assert `body["confirm_token"]` matches the server-side compute.
Operator-requested 2026-05-27: centralize the per-platform quirks that
had been accumulating across credential_service, sidecar, and platforms
into a single per-platform module so adding/updating quirks becomes
"edit one file."
**Layout**
services/platforms/
base.py PlatformInfo dataclass + module-default key
chains + shared helpers (str_id_value, str_field)
__init__.py PLATFORMS dict + public API (auth_type_for,
known_platform_keys, to_dict,
external_post_id_keys_for, description_keys_for)
patreon.py metadata only — the reference platform, no quirks
subscribestar.py metadata + augment_cookies (18+ agreement) +
derive_post_url (synthetic /posts/<post_id>)
hentaifoundry.py metadata + augment_cookies (host-only PHPSESSID
duplicate) + derive_post_url (/pictures/user/...)
pixiv.py metadata + derive_post_url (/artworks/<id>)
discord.py metadata + derive_post_url
(channels/<server>/<channel>/<message>)
deviantart.py metadata only — un-audited; quirks to be added
when an operator first exercises DA
**PlatformInfo extensions**
Existing fields preserved. Four new optional fields:
external_post_id_keys: tuple[str, ...] | None
Override the sidecar external_post_id lookup chain. None falls
back to DEFAULT_EXTERNAL_POST_ID_KEYS in base.py
("post_id", "id", "index", "message_id") — covers every current
platform.
description_keys: tuple[str, ...] | None
Override the description body lookup chain. None falls back to
DEFAULT_DESCRIPTION_KEYS ("content", "description", "caption",
"message") — Discord's "message" body field is covered by the
default's trailing entry.
derive_post_url: Callable[[dict], str | None] | None
Synthesize the post permalink from sidecar metadata. None = trust
the bare `url` / `post_url` field (patreon, deviantart).
subscribestar/pixiv/hf/discord override this because their `url`
is the file CDN URL.
augment_cookies: Callable[[str], str] | None
Post-process the materialized cookies.txt before gallery-dl
consumes it. None = no-op. Used by subscribestar (age cookie) and
hentaifoundry (host-only PHPSESSID duplicate).
**Consumer changes**
- credential_service._augment_cookies(platform, netscape) shrunk from a
per-platform-conditional dispatcher (~80 lines of inlined helpers) to
a 5-line lookup: `info.augment_cookies(netscape) if info and
info.augment_cookies else netscape`. The platform-specific helper
bodies moved verbatim into the per-platform modules.
- sidecar.parse_sidecar similarly delegates: external_post_id chain via
external_post_id_keys_for(category), description chain via
description_keys_for(category), post_url via
PLATFORMS[category].derive_post_url. The _DERIVED_URL_PLATFORMS set
and inline _derive_post_url body both gone. Added a shared `_first_id`
helper for bool-safe id coercion.
**Public API preserved**
PLATFORMS, PlatformInfo, auth_type_for, known_platform_keys, to_dict
are all re-exported from the package's __init__.py. test_platforms_registry
test_credential_service, and test_sidecar_util pass without changes
because the behavior is identical; only the implementation moved.
**Adding a new platform**
1. Create services/platforms/<name>.py with `INFO = PlatformInfo(...)`
and any of the four optional hooks.
2. Import it in services/platforms/__init__.py + add to the PLATFORMS
tuple-comprehension.
3. Done. sidecar parsing, cookie materialization, /api/platforms all
pick it up automatically.
Operator-flagged 2026-05-27: HF source check 401'd on
`HEAD /?enterAgree=1` even with valid login cookies. Root cause is the
combination of (1) gallery-dl's HF extractor checking
`self.cookies.get("PHPSESSID", domain="www.hentai-foundry.com")` with
`requests`' EXACT domain matching, and (2) the extension's cookies.js
forcibly rewriting every captured cookie to a leading-dot subdomain-wide
form. HF's PHPSESSID is browser-stored as host-only on
`www.hentai-foundry.com`; the rewrite re-anchored it to
`.hentai-foundry.com`, which `cookies.get(...)` no longer matches even
though the cookie is still sent on actual HTTP requests (RFC 6265
subdomain rules). The extractor falls into its unauthenticated
`?enterAgree=1` fallback, which 401s (Cloudflare or HF's anti-bot HEAD
gating).
Two-part fix, no operator action required for existing stored cookies:
1. **Backend** (`credential_service._augment_cookies`) — refactored from
the subscribestar-only single function into a per-platform dispatcher.
New `_augment_hentaifoundry` parses the materialized netscape file
and, for each `.hentai-foundry.com` entry whose name is PHPSESSID or
YII_CSRF_TOKEN, appends a host-only duplicate
(`www.hentai-foundry.com\tFALSE\t...`). Originals preserved. Three
new tests pin: injection fires + originals preserved; idempotent
when host-only already exists; doesn't touch unrelated cookies
(e.g. `_ga`).
2. **Extension** (`cookies.js`) — `toNetscapeFormat` now respects
`c.hostOnly` from the browser instead of blindly forcing a
leading-dot subdomain-wide form. Host-only cookies are written with
the bare host + FALSE flag; non-host-only cookies retain the
leading-dot + TRUE form. Forward-compat — fresh captures from
v1.0.5+ no longer need the backend's host-only duplication.
Extension bumped 1.0.4 → 1.0.5; manifest + package.json in lockstep.
After deploy: the next HF source check on the operator's already-stored
cookies will succeed because the materialized cookies.txt now contains
host-only PHPSESSID. No browser re-export needed.
Operator-flagged 2026-05-27: subscribestar source check aborted with
`AbortExtraction: HTTP redirect to .../age_confirmation_warning`. The
captured `_personalization_id` cookie in the browser-stored file had
expired (annual rotation), and the user could not realistically refresh
it: SubscribeStar's frontend JS uses localStorage to suppress the
age-confirmation popup once dismissed, so a logged-in revisit doesn't
re-show the popup and the server-side cookie is never re-issued.
gallery-dl's own login flow (which FC doesn't exercise — cookies come
from the extension instead) sidesteps this by manually setting
`18_plus_agreement_generic=true` on `.subscribestar.adult`. The server
accepts that as the age-confirmation marker.
`credential_service._augment_cookies(platform, netscape)` mirrors that
behavior: when the materialized cookies file is for subscribestar and
the age cookie isn't already present, append a synthetic line for
`.subscribestar.adult` with name=`18_plus_agreement_generic` value=`true`
and a far-future expiry. No-op for other platforms; no-op if the cookie
is already present (idempotent for manual pastes / extension captures
that happen to include it).
Three new tests pin: (a) injection fires for subscribestar, preserves
existing cookies; (b) idempotent when already present (no double
injection); (c) does NOT fire for non-subscribestar platforms (Patreon
etc. don't get a foreign-domain cookie).
Not a curator handling bug per se — the extension faithfully captured
what the browser had. This is mirroring a documented gallery-dl
workaround so the cookies-via-extension auth path doesn't degrade as the
server-side cookie expires.
Operator-flagged: the claim that 'a future deep-scan via the new parser
will fix HF and Discord post_url' was conditional on the operator
actually running a deep-scan, which they might not do for ages. Until
then HF posts stay at post_url=NULL (HF sidecars have no `url` field)
and Discord posts stay pointing at cdn.discordapp.com/attachments/...
(the file URL, not the message permalink).
The migration was already opening sidecar files for SubscribeStar.
Generalizing the loop to also handle HF and Discord is a tiny addition
that closes the gap without operator intervention.
Per-platform Part 1 logic now:
subscribestar — read sidecar.post_id, overwrite external_post_id +
post_url with the derived /posts/<post_id> permalink.
hentaifoundry — read sidecar.user + .index, overwrite post_url with
/pictures/user/<u>/<i>. external_post_id (= index) unchanged.
discord — read sidecar.server_id + .channel_id + .message_id,
overwrite post_url with the discord.com/channels/.../<m> triple.
external_post_id (= message_id) unchanged.
Part 2 (SubscribeStar fragment merge) and Part 3 (pure-SQL Pixiv
post_url backfill) unchanged.
Posts whose related ImageRecord paths don't resolve on disk (orphan
filesystem state) are reported per-platform in the migration output —
those still need a future deep-scan, but the in-DB-with-on-disk-files
common case is now fully covered by the migration alone.
bvandeusen
