Commit Graph
21 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 fc136006b7 feat: what you pay for, against what FC actually follows (387 C4)
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
Build images / build-web (push) Successful in 1m18s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m11s
Build images / promote (push) Skipped
CI / integration (push) Successful in 3m4s
Reconciliation in Subscriptions, asymmetric on purpose. Subscriptions FC does not follow get a per-row add; sources the roster cannot account for are REPORT ONLY (operator decision) and link to the list on the same page. No one-click disable, so no disabled-reason column and no migration.

The membership<->source join lands as a SHARED resolver in membership_roster, not inline here: E4 now uses it as its negative check, so the two features cannot give different answers to 'is this membership already tracked?'. Keys on the exact cached campaign id FIRST and the URL handle only as fallback, because the id is written only after a source has been walked once. Read via any <platform>_campaign_id override rather than naming Patreon's, per rule 169.

The report-only bucket is gated on roster freshness and carries a per-row basis, so 'your membership says former patron', 'we know this id and it is absent', and 'we only have a handle' stay three different sentences. has_paid_access None never reads as lapsed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-11 21:38:03 -04:00
bvandeusenandClaude Opus 5 a708f5e9db feat: the front door says whether ingestion is working (milestone 387 step B3)
CI / extension-version (push) Successful in 4s
CI / lint (push) Successful in 4s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 27s
CI / backend-lint-and-test (push) Successful in 34s
Build images / build-web (push) Successful in 1m17s
Build images / smoke-web (push) Skipped
CI / integration (push) Failing after 2m8s
Build images / build-ml (push) Successful in 2m18s
Build images / promote (push) Skipped
The step phase A was building toward. A1 made the gated count true, A2
made it a durable state, A3 made it visible in Subscriptions — but
Subscriptions is where you go once you already suspect something. This
is the line that reaches someone who wasn't looking.

A thin grey strip above the feed, front door only: last check, sources
failing, sources you can't see. Only the actionable items take a
colour, and nothing renders at zero — a permanent "0 failing" trains
you to skip the line, which would hide the real number when it appears.

Two predicates, defined once. The ribbon counts and the surfaces it
links to have to agree on what "failing" and "no access" MEAN, or the
ribbon says 3 and the card shows 4. They live in db_helpers, which
exists for exactly this reason (its docstring: divergent copies are how
the race bugs crept in). Not in source_service, because
scheduler_service needs them too and source_service already imports
scheduler_service — the other direction is a cycle.

Counting deliberately spans all ENABLED sources rather than the
auto_check subset scheduler_status already walks: a source erroring on
a manual-only artist is still erroring. Disabled sources count for
nothing, which is what makes issue 1285 the real escape hatch for a sub
you stopped paying for.

Extends the existing schedule-status endpoint rather than adding a
parallel aggregate — the store already fetches it. Two scalar COUNTs.

The status filter is now URL-addressable, which it had to be for the
ribbon's links to land anywhere: a count that drops you on an
unfiltered list makes the reader redo the filtering the ribbon just
did. Mirrors how artistFilter already reads from route.query.

Front-door-only via a route prop, not a route.name check, so the view
doesn't need to know what it's mounted as and the router states the
intent in one place. Inside Browse's Posts tab you're looking FOR
something and the hub is one click away.

The fetch is swallowed on mount by design (rule 164): this is an aside,
and the feed must render whether or not the status call succeeds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-09 22:55:14 -04:00
bvandeusenandClaude Opus 5 6bb18050a4 feat: no-access is visible per source, and findable (milestone 387 step A3)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 2m40s
Build images / build-ml (push) Successful in 2m47s
Build images / build-web (push) Successful in 1m35s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
A3 of milestone 387, completing phase A. A1 made the count true, A2
made it a durable state; this makes it something the operator can see
without going looking.

Turned out smaller than filed, because A2 revealed why the existing
`tier_limited` palette entry in FailingSourcesCard had never rendered:
the chip was being cleared by the same successful run that produced it.
The colour was already chosen.

Where it surfaces:

- SourceHealthDot gains a `no-access` grade. Deliberately its own grade
  rather than folded into healthy (which hides it) or warning (which
  sends the operator hunting for a break that isn't there). A source
  with real failures still grades as failing whether or not it is also
  gated.
- SourceRow gets an info-coloured lock chip in the status cell, which
  was empty for these sources — they have zero failures. Placed ahead
  of the backfill states: "we can't see this creator" is the more
  useful thing to say than which walk phase it is in, and unlike those
  it does not resolve on its own.
- A "No access" status filter, deliberately separate from "Has errors".
  Without it a gated source is invisible in a long list, because it
  correctly stays out of the failing rollup.

Left OUT of NeedsAttentionCard on purpose. That card's only affordance
is Retry, and you cannot retry your way into a subscription tier —
issue 1285 already gives the real escape hatch, since disabling a
source clears its state. Nothing structural needed changing: the card
is fed by consecutive_failures > 0, which a tier-limited source never
has.

The count lives on the download event, not the source, so `list()`
joins it in with one DISTINCT ON query — selecting the run_stats
sub-object rather than whole metadata blobs, which carry up to 500KB of
truncated stdout each. Scoped to tier-gated rows only, so a healthy
library issues no extra query at all. Absent stays None rather than 0,
and both UI surfaces phrase the state without a number when it is
missing instead of printing a fabricated zero.

Also covers A1's live gated count, which shipped untested, and extends
the mount helper with slot stubs: SourceHealthDot puts the dot in a
NAMED slot, and unresolved Vuetify components render default slots
only — so those assertions would have found an empty wrapper and
passed vacuously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
2026-09-09 21:26:48 -04:00
bvandeusenandClaude Fable 5 dc7fa6eae2 feat(ia): wave 3 — Subscriptions landing answers 'what needs me, what came in?'
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 38s
CI / frontend-build (push) Successful in 20s
CI / integration (push) Successful in 3m30s
Daily-use reorder of the Subscriptions tab: needs-attention strip first
(FailingSourcesCard moves up from below the Downloads fold — a broken
subscription was invisible unless you went looking), then a new Recent
arrivals card (real downloads only, no-change scans filtered out, artist
links), then the source list. Both cards render nothing when there's nothing
to say.

Retry logic moves into the downloads store (retrySource / retryAllFailing) so
the needs-attention card and the Downloads maintenance menu share one
implementation — single-retry forces past cooldown, bulk keeps cooldown
enforcement, same tally shape. The card's Logs button deep-links into the
Downloads tab pre-filtered (?source_id now watched, not just read on mount).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 17:52:00 -04:00
bvandeusenandClaude Opus 4.8 65ec29ba9b feat(ingest): Recapture mode — re-grab post bodies/links + localize on-disk inline images (#830)
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 44s
CI / integration (push) Successful in 3m21s
CI / lint (push) Successful in 4s
A plain backfill gates post-body capture on the seen-ledger, so a post whose
media is already on disk AND whose post key is already seen never gets its body
recaptured (operator-flagged: Industrial Lust description missing). Recovery
recaptures unconditionally but re-downloads the whole source.

New 'recapture' walk mode (4th beside tick/backfill/recovery): bypasses the
post-record gate so EVERY post's body + external links are re-captured
(detail-fetching empty bodies) WITHOUT re-downloading on-disk media; and
surfaces already-present media via a separate non-deleting relink channel so the
importer backfills ImageRecord.source_filehash for inline-image localization.

- ingest_core: recapture mode + recapture_records gate bypass + relink collect
- patreon_downloader: recapture surfaces seen-on-disk as skipped_disk(path),
  never refetches seen-missing media, still downloads genuinely-new
- importer.relink_source_filehash: NULL-only sha256 backfill, never unlinks
- download_service: mode derivation + phase-3 relink loop + lifecycle clear
- source_service/api: start_recapture + backfill_recapture field + action
- frontend: Recapture kebab action + 'Recapturing' badge across SourceActions/
  Row/Card/SubscriptionsTab + sources store
- tests across ingester/downloader/importer/source_service/api/download_service

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 20:58:40 -04:00
bvandeusenandClaude Opus 4.8 e4e35163ab feat(subs): subscriptions UX batch — error reasons, single-source rows, health sort, bulk backfill
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 3m2s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 26s
Operator-requested follow-ups:
- #1 Failure reason on hover: the red error-count chip now shows source.last_error
  in a tooltip (desktop row + mobile card), so the cause (e.g. the new
  "vanity=cw" message) is visible without opening Downloads.
- #2 Collapse the single-source case: a subscription with exactly one source now
  shows that source's URL + its own actions (Check / Backfill / ⋮ / Edit) inline
  on the artist row — no expand needed for the common case. Multi-source keeps
  the artist-level actions + expandable per-source table.
- #3 Sort by health: the Health column is sortable on a numeric rank
  (never/ok/warn/fail) and the table defaults to worst-first, name as tiebreak.
- #4 Drop Preview: removed the low-value bounded-peek action from the menu and
  all its wiring (backend endpoint + store fn left in place, unused).
- #5 Backfill selected: a "Backfill" button in the bulk bar arms a run-until-done
  backfill on every enabled, not-already-running source in the selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 21:00:47 -04:00
bvandeusenandClaude Opus 4.8 b7d07324ee fix(subs): stop the lock/reload on source actions + regroup the row buttons
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 24s
CI / integration (push) Successful in 3m2s
Lock/reload: every inline source action (check / backfill / recover / toggle /
remove) ended by refetching the WHOLE subscription list (store.loadAll /
refresh), which blocked the UI and re-rendered the table — collapsing the
expanded row, which read as "locks then resets." The action APIs already return
the updated source, so the store now patches that one row in place
(_patchSource / _dropSource); the post-action loadAll/refresh calls are gone.
Toggling enabled, starting/stopping a backfill, recovering, and removing are now
instant and leave the expansion intact.

Button regroup (operator-flagged: tiny, mis-clickable, not grouped by function):
- New shared SourceActions.vue used by desktop SourceRow + mobile SourceCard.
- Frequent actions stay as size="small" buttons: Check, Backfill/Stop.
- Low-frequency / destructive actions move into a labelled overflow (⋮) menu —
  Preview backfill, Recover dropped near-duplicates, Remove source — so they
  can't be fat-fingered, and the labels spell out recover-vs-backfill.
- Edit moves next to the source URL (its identity), out of the action cluster
  where it sat beside Remove.
- Single-source Remove now confirms (it had no guard before).

Tests: store patches/drops in place without dropping the cache.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 20:48:01 -04:00
bvandeusenandClaude Opus 4.8 e82c2ee57b feat(subscriptions): dry-run backfill preview — B4 preview (plan #708)
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 24s
CI / integration (push) Successful in 3m1s
CI / frontend-build (push) Successful in 5m13s
Owning the walk lets an operator gauge "is this source worth a backfill?" before
arming one. ingest_core.Ingester.preview walks the first few feed pages and
counts media NOT already in the seen/dead ledgers, downloading nothing
(read-only). download_backends.preview_source resolves the campaign id + runs it
(native-only, mirrors verify_source_credential / run_download); POST
/api/sources/{id}/preview returns {total_new, posts_scanned, has_more, sample[]}
(409 on auth/drift/unresolvable, 400 for gallery-dl platforms). PatreonClient
gains post_meta(post) for the sample's title/date.

UI: a Patreon-only Preview button (mdi-eye-outline) on SourceRow + SourceCard
opens PreviewDialog — self-fetches with loading / error / empty / result states
and a "Start backfill" shortcut. Store action previewSource.

Tests: preview counts new media without downloading + samples only posts with
new items; page_limit caps the walk + flags has_more.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:18:08 -04:00
bvandeusenandClaude Opus 4.8 5b615b7ded feat(sources): pre-flight credential verify on backfill/recovery arm — #703 step 2
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 25s
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 2m59s
Before arming a deep walk on a native-ingester platform (Patreon — where
verify is one cheap API page), POST /sources/{id}/backfill {start|recover}
runs the shared verify_source_credential first and REFUSES (409 + reason)
only on a definitive rejection (verify→False, e.g. expired cookies). It
proceeds on valid (True) or inconclusive (None — a network blip must not
block). Gated to native platforms: gallery-dl verify is a slow --simulate
subprocess, too heavy for an arm action. The credential read happens in a
session that's CLOSED before the verify network call (no held conn).

Frontend: onBackfill/onRecover now read e.body.detail (ApiError carries the
reason in .body, not .detail) so the rejection text surfaces in the toast.

Tests: arm blocked on rejection (409, source not armed), proceeds on
inconclusive, stop never pre-flights, gallery-dl platform skips pre-flight.
An autouse fixture stubs verify to 'valid' for the existing backfill
endpoint tests so they stay network-free.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 23:21:05 -04:00
bvandeusenandClaude Opus 4.8 ec43e823e1 feat(patreon): recovery UI + gallery-dl cutover — build step 5 (plan #697)
CI / frontend-build (push) Successful in 23s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 10s
CI / integration (push) Successful in 2m59s
Final step of the native Patreon ingester: a first-class Recovery action,
and removal of the now-dead gallery-dl Patreon path.

Recovery (rules #23/#24/#27 — full product, with UI):
- source_service.start_recovery arms the #693 backfill state machine PLUS
  `_backfill_bypass_seen`, flipping download mode to recovery (bypass the
  seen-ledger to re-fetch dropped-and-deleted near-dups and re-evaluate
  under the current pHash threshold). Stop via the shared stop_backfill.
- SourceRecord exposes backfill_bypass_seen; POST /sources/{id}/backfill
  gains action="recover".
- Frontend: Recovery button (Patreon-only, mdi-backup-restore) on SourceRow
  + SourceCard; the running badge labels "Recovering (N)" vs "Backfilling
  (N)"; the Stop tooltip says "Stop recovery". sources.js recoverSource +
  SubscriptionsTab onRecover.

Cutover (rule #22 — no legacy):
- gallery_dl: removed PLATFORM_DEFAULTS["patreon"], the patreon
  files/cursor branch in _build_config_for_source, and the patreon/Mux
  yt-dlp Referer/Origin block (was patreon-specific and wrongly tagged the
  other platforms' yt-dlp fetches; native ingester owns it now).
- download_service: removed the dead campaign-id-retry helpers
  (_looks_like_campaign_id_failure / _CAMPAIGN_ID_FAILURE_PATTERN) and
  _effective_url. Vanity→campaign resolution + resume_cursor + the
  cursor/PARTIAL lifecycle stay — they serve the native ingester.

Tests: removed the two obsolete patreon-gallery-dl config tests (yt-dlp
Referer, resume-cursor); repointed the generic skip-value config tests to
subscribestar; added start_recovery + recover-endpoint coverage
(backfill_bypass_seen). gallery-dl stays for the other 5 platforms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 22:18:03 -04:00
bvandeusenandClaude Opus 4.8 618dafde85 feat(subscriptions): smarter-backfill UI — Start/Stop + state badge
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 11s
CI / frontend-build (push) Successful in 18s
CI / integration (push) Successful in 2m58s
Plan #693 (frontend). Backend landed in 96fffaf.

- sources store: setBackfill(runs) → startBackfill/stopBackfill ({action}).
- SubscriptionsTab: the deep-scan window.prompt for N runs becomes a
  Start/Stop toggle keyed on source.backfill_state.
- SourceRow + SourceCard: the 'backfill (N×)' chip becomes a state badge —
  Backfilling (chunk N) / Backfilled / Stalled — and the scan button
  toggles between Backfill (mdi-magnify-scan) and Stop (mdi-stop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 15:06:30 -04:00
bvandeusenandClaude Opus 4.8 928e3037f0 fix(ui): purpose-built mobile layout for subscriptions hub
CI / frontend-build (push) Successful in 59s
CI / integration (push) Successful in 2m56s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 10s
Vuetify's auto card-stack was too verbose (one subscription filled the whole
phone screen) and the expanded sources still needed lateral scroll. Replace it
below 600px (useDisplay) with a custom compact-card list: each subscription is
a 2-line card (name + health + expand chevron, then platform chips + sources
count + last activity) so several fit per screen. Expanding shows the action
row + each source as a STACKED SourceCard (new) — platform/url/enabled/last/
next/errors/actions laid out vertically, no horizontal scroll. The mobile
cards drive the same selected/expanded key arrays as the desktop data-table,
so selection and bulk actions are unchanged. Desktop keeps the v-data-table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 13:40:25 -04:00
bvandeusenandClaude Opus 4.8 b08b12eb8f fix(ui): subscriptions table → card layout on mobile
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 11s
CI / frontend-build (push) Successful in 1m8s
CI / integration (push) Successful in 2m56s
The subscriptions v-data-table (select + expand + 6 cols + a nested 8-col
sources sub-table) horizontally-scrolled on phones. Set mobile-breakpoint=600
so Vuetify stacks each subscription row into a label:value card below 600px;
the custom item slots (platform chips, health dot, action buttons) render as
card rows. The expanded sources detail reclaims its desktop indent on mobile
and keeps its own horizontal scroll for the wide source columns.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 12:53:40 -04:00
bvandeusenandClaude Opus 4.8 4fd6d4cc29 fix(ui): mobile pass 2 — Posts & Subscriptions filter bars
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 12s
CI / frontend-build (push) Successful in 34s
CI / integration (push) Successful in 2m58s
PostsFilterBar didn't wrap and its artist/platform fields had inline
min-widths (240/180px) a media query can't override → horizontal overflow on
phones. Moved widths to classes, added flex-wrap, and <600px each field takes
a full-width row. SubscriptionsTab's status/search inline max-widths likewise
moved to classes; <600px they go full-width and the v-spacer is dropped so the
search isn't shoved around.

Verified as already-fine (sweep false positives, no change): PostCard (default
body is a stacked column; only goes row at container >=800px), SeriesReaderView
(already has a <=768px block: nav drawer 150px, quick-nav stacks). The
subscriptions v-data-table scrolls horizontally within its own wrapper, so it
doesn't widen the page — a true mobile card layout is a larger follow-up if
wanted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 09:26:56 -04:00
bvandeusen 4bff1d8558 fix(audit-g4): status-enum miss batch
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 14s
CI / frontend-build (push) Successful in 19s
CI / intimp (push) Successful in 3m48s
CI / intapi (push) Successful in 7m37s
CI / intcore (push) Successful in 8m18s
Five extension-miss findings from the 2026-06-02 audit, where a status
value was added on one side but a downstream consumer didn't pick it up.

- download_service._phase3_persist: explicit branches for
  ImportResult.status in ('failed','refreshed'). For 'failed' (archive
  probe crash from _import_archive), unlink the source file so the
  filesystem scanner doesn't re-import and re-crash on the same
  archive forever. 'refreshed' is currently unreachable from the
  download path (no deep=True) but matches the importer's documented
  contract; treat as 'attached'.

- gallery-dl backfill auto-complete now gates on dl_result.success +
  no error_type, not just return_code==0 + files_downloaded==0.
  VALIDATION_FAILED exits the subprocess with returncode=0 and
  files_downloaded=0 when every file was quarantined, matching the
  prior predicate exactly and zeroing the operator's armed backfill
  budget on the FIRST quarantine run instead of decrementing.

- attach_in_place archive dispatch now threads artist + source_row
  through _import_archive (and _import_media for archive members)
  and _supersede. The path-walk fallback (_resolve_artist) is still
  used by filesystem-import; the download path now binds
  ImageProvenance to the explicit subscription Source instead of
  rediscovering by (artist_id, platform).

- Three FE handlers now recognize status:'deferred' from
  /api/sources/<id>/check: SubscriptionsTab.onCheck (was toasting
  "event #undefined"), SubscriptionsTab.checkAll (was counting
  deferred as queued), DownloadEventRow.onRetry (was saying
  "re-queued" when nothing was). Pattern matches DownloadsTab.onRetryAll
  which already had it.

- celery_signals._queue_for now maps backup/admin/library_audit
  prefixes to 'maintenance' (matching task_routes). TaskRun.queue
  was returning 'default' for those rows, so per-queue dashboard
  filters and per-queue threshold overrides (added in G3) silently
  missed them.
2026-06-02 16:04:59 -04:00
bvandeusen 19aece1fc4 feat(download): tick/backfill modes + partial-success classifier (plan #544)
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 16s
CI / frontend-build (push) Successful in 27s
CI / intimp (push) Successful in 3m42s
CI / intapi (push) Successful in 7m38s
CI / intcore (push) Successful in 8m20s
Routine subscription polls walked the entire post history every tick
even when nothing had changed, because gallery-dl's default `skip: True`
continues iterating archived posts. A creator with ~550 archived posts
(Knuxy patreon) saturates the 870s wall-clock cap before completing,
even with zero downloads needed. Plus, a tier-limited run that
downloaded hundreds of files but ran out the clock should be a
warning, not an error.

Two coupled changes, both operator-flagged 2026-06-01:

* **Tick mode (default, cron polls).** New `TICK_SKIP_VALUE = "exit:20"`
  asks gallery-dl to exit after 20 contiguous archived items. Fresh
  subscriptions + new-content cases still walk normally; established
  subscription with zero new content exits in ~30s of HEAD requests
  instead of pegging the timeout. 20 (not 5) gives headroom against
  paywall warnings interleaving with archived items.
* **Backfill mode (explicit, operator-triggered).** Sticky for N runs
  via new `Source.backfill_runs_remaining` (alembic 0031). While > 0,
  downloads use `skip: True` + 1800s timeout. Auto-decrements per run
  with early-reset to 0 when a clean run finds zero files (queue
  drained). N defaults to 3 — multiple runs give the system enough
  budget to finish a deep walk across timeout boundaries. New
  `POST /api/sources/{id}/backfill` arms the source; "Deep scan"
  button on each SourceRow (chip shows remaining count) wires it.

Plus partial-success classifier: non-zero gallery-dl exit + ≥1 file
downloaded + no source-level error fires `ErrorType.PARTIAL`, which
download_service maps to `status=\"ok\"`. The run did real work; the
next tick continues via gallery-dl's archive. No more red events for
"timed out mid-walk after downloading 300 files."

Retires `SourceConfig.skip_existing` — skip value is now derived from
the source state and passed as a separate `skip_value` parameter
through download() / _build_config_for_source(). `GD_DEFAULTS` drops
the now-dead key (was inert data after this refactor).

Tests cover:
* tick + backfill skip-value emission in _build_config_for_source
* PARTIAL classifier branch + TIER_LIMITED-wins-over-PARTIAL ordering
* SourceService.set_backfill_runs validation + persistence
* /api/sources/{id}/backfill 200/400/404 paths
* download_service auto-decrement / auto-reset / tick-mode-no-touch
* PARTIAL → status=ok in the orchestrator (no consecutive_failures bump)
2026-06-01 18:23:28 -04:00
bvandeusenandClaude Opus 4.7 42ddac9996 fix(subscriptions): fixed-height hub so only the tab content scrolls
The whole view scrolled instead of just the subscription list. Made the
hub a viewport-height flex column (tabs stay fixed) with the v-window as
the single internal scroll container; the per-tab sticky control bars now
pin to the window top (top:48px -> 0). Operator-flagged 2026-05-28.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:13:19 -04:00
bvandeusenandClaude Opus 4.7 eebc8e2413 refactor(dry-F2): centralize shared UI primitives (relative-time, toast, download-status)
- utils/date.js: add formatRelative(iso, {future,nullText}); migrate 6 sites
  (SourceRow, SubscriptionsTab, SourceHealthDot, SchedulerStatusBar +
  thin adapters in BackupRunsTable/SystemActivityTab for their '—' null text).
  PostCard (30d->absolute) and CredentialCard (mo/y buckets) intentionally
  keep bespoke formatters.
- utils/toast.js: toast(opts) wraps the globalThis.window?.__fcToast?.(...)
  incantation; migrate 63 call sites across 24 files.
- utils/downloadStatus.js: single source for the download-event status enum
  -> label/color/icon; collapse the 3 duplicate maps (DownloadStatChips,
  DownloadsFilterPopover, DownloadEventRow).

Net -33 lines. Platform metadata was already centralized in platformColor.js.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 11:20:07 -04:00
bvandeusenandClaude Opus 4.7 2358cedf3e feat(dashboards): scheduler health strip, failing-source rollup, 24h activity sparkline, credential staleness nudge
D1 scheduler visibility: AppSetting last-tick stamp on every Beat tick +
GET /api/sources/schedule-status (last_tick_at/next_due_at/due_now/auto_sources)
+ SchedulerStatusBar on the Subscriptions tab (re-polled every 30s).

D2 failing-source rollup: ?failing=true on the sources list + FailingSourcesCard
on Downloads with per-source and bulk "retry" (re-runs the feed via /check).

D3 activity sparkline: GET /api/downloads/activity hourly buckets + CSS bar
chart by the stat chips (failures stacked in error color); refreshes on live poll.

D4 credential staleness: surface last_verified age + "re-verify recommended"
warning past 30d; also fixes the dead last_verified_at field-name mismatch so
the verification row renders at all.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 08:30:14 -04:00
bvandeusen a459d21a65 feat(dashboards): 1600px max-width, richer Downloads filters, needs-attention + sticky headers
Operator-flagged 2026-05-28: the Subscriptions/Downloads dashboards were
full-bleed and thin on filtering. Chosen via AskUserQuestion.

**Layout**
- SubscriptionsView capped at a centered max-width 1600px (covers all
  three subtabs) so rows aren't a mile wide on ultrawide monitors.
- Sticky control headers on both tabs (top: 48px, below the top nav,
  opaque bg) so filters/stat-chips stay reachable while scrolling a
  long list.

**Downloads filters (all four requested)**
- Stat chips are now clickable filters: click Queued/Running/Completed/
  Failed/Skipped to filter the list to that status; the active chip is
  outlined + elevated; re-click clears.
- Free-text search box over the loaded events (artist / platform /
  error substring).
- Artist filter: the filter popover's numeric "Source ID" field is
  replaced with an artist autocomplete (sources.autocompleteArtist →
  artist_id, which /api/downloads already supports). Pill shows the
  artist name.
- "Show no-change scans" toggle (default OFF): hides status=ok/skipped
  rows with 0 files (the scheduled scans that found nothing) so real
  downloads + failures stand out.

**Subscriptions**
- "Needs attention" quick-filter chip: one click to show only artists
  with sources that have errors OR have never been checked; chip shows
  the count and disables the status dropdown while active.

Frontend-only — backend filter params (status/artist_id/date) and the
/api/downloads endpoint already supported everything.
2026-05-28 08:10:04 -04:00
bvandeusen 9322c984fd feat(subs-hub): collapse /credentials + /downloads into /subscriptions hub with three GS-style subtabs
Replaces the three top-level routes with a single `/subscriptions` parent
owning the whole download-pipeline domain. Internal tab state via `?tab=`
query param, mirroring ArtistView's pattern. TopNav auto-drops the two
removed entries (route-driven via meta.title). Bookmark-safe redirects
from `/credentials` and `/downloads` route into the appropriate subtab.

**Subtab 1 — Subscriptions (default).** Carries over the existing
artist-grouped expandable table; adds (a) status filter dropdown, (b)
bulk-select column with Enable/Disable/Delete-all actions, (c) GS-style
color-coded `PlatformChip` per distinct platform in the collapsed row.
Reuses SourceRow, SourceHealthDot, SourceFormDialog, ArtistCreateDialog.

**Subtab 2 — Downloads.** Full GS dashboard. Five colored stat chips up
top (Queued/Running/Completed/Failed/Skipped, sourced from new
`GET /api/downloads/stats?window_hours=`). Popover-style filter UI
(Status/Source/FromDate/ToDate) with active-filter pills below.
Maintenance menu wraps existing /api/import/retry-failed and
/api/import/clear-stuck endpoints; Export-failed-logs item disabled with
a "v2" tooltip. Per-row Retry preserved via existing DownloadEventRow.

**Subtab 3 — Settings.** Four sections: ExtensionKeyBar (top), GS-style
per-platform CredentialCard grid (md=6 v-row/v-col, dashed border if
unset / accent border if set, expandable how-to panel), Downloader card
(rate limit, validate_files), Schedule defaults card (default interval,
event retention, failure warning threshold). The Downloader and Schedule
sections were extracted out of components/settings/ImportFiltersForm.vue
— SettingsView's Import tab now owns only image-import filters.

**Backend:** new `GET /api/downloads/stats` returns
{pending, running, ok, error, skipped} count grouped by status over the
configurable window. Status keys stay raw from the ENUM; UI does the
display-label mapping. Two integration tests pin the response shape +
window_hours validation.

**Util:** `frontend/src/utils/platformColor.js` — single source of truth
for the six platforms' color + icon + label, mirroring GS's palette
(patreon=red mdi-patreon, subscribestar=amber mdi-star,
hentaifoundry=purple mdi-palette, discord=indigo mdi-discord,
pixiv=blue mdi-alpha-p-box, deviantart=green mdi-deviantart). Unknown
platform falls back to grey + mdi-web.

Deferred (explicit non-goals): subscription import/export, "Trigger Due
Now" scheduler-tick button (needs new backend endpoint), Export Failed
Logs CSV dump.
2026-05-27 13:02:24 -04:00