feat: finish the Discord switchover — recapture on every native source, backfills that run, gallery-dl's Discord config retired (milestone 428)
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 21s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Failing after 2m19s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 21s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Failing after 2m19s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
- Recover and Recapture show on every native source. The menu gated them on
a copied platform list ('patreon', 'subscribestar') that went stale when
Discord moved over. Sources now carry `native_ingester` from the backend's
own predicate.
- A running backfill is due on every scheduler tick. Nothing queued a
backfill's next chunk: each one waited for the source's regular interval,
so an armed backfill sat idle until the next check (8h at the default) and
a five-chunk walk took most of two days. The in-flight guard and the
platform lock keep one chunk at a time. A failing source falls back to its
backoff, and a stalled or out-of-budget walk stops being due. It also runs
when the artist has auto-check off, since the operator started it by hand.
- gallery-dl no longer carries Discord: its naming constants, platform
defaults, sidecar-mirroring postprocessor and token injection are gone.
The naming test moves to the native downloader and still renders against
the real gallery-dl sidecar fixture. That is the guard that the files
gallery-dl wrote are found on disk rather than fetched again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -18,6 +18,7 @@ from ..models import (
|
||||
Source,
|
||||
)
|
||||
from .db_helpers import failing_sources_clause
|
||||
from .download_backends import uses_native_ingester
|
||||
from .gallery_dl import ErrorType
|
||||
from .membership_reconcile import KEPT_KEY, STOPPED_KEY
|
||||
from .membership_roster import gated_reasons_for_sources
|
||||
@@ -125,6 +126,11 @@ class SourceRecord:
|
||||
"backfill_posts": self.backfill_posts,
|
||||
"tier_gated_count": self.tier_gated_count,
|
||||
"gated_reason": self.gated_reason,
|
||||
# Recover / recapture exist only on the native ingester. Sent so the
|
||||
# UI asks the backend's own predicate instead of keeping a copy of
|
||||
# the platform list — the copy said "patreon, subscribestar" for a
|
||||
# day after Discord went native (milestone 428).
|
||||
"native_ingester": uses_native_ingester(self.platform),
|
||||
}
|
||||
|
||||
|
||||
@@ -551,8 +557,8 @@ class SourceService:
|
||||
whole source); the two flags are mutually exclusive, so arming recapture
|
||||
clears bypass_seen. Clears prior cursor/chunk/stall state so it walks
|
||||
fresh from the top. The flag is cleared on completion (download_service)
|
||||
and on stop. Recapture is Patreon-only (the native ingester's post-record
|
||||
capture); inert elsewhere. The UI gates the action to Patreon sources."""
|
||||
and on stop. Recapture needs the native ingester's post-record capture,
|
||||
so the UI offers it on native sources only (`native_ingester`)."""
|
||||
source = (await self.session.execute(
|
||||
select(Source).where(Source.id == source_id)
|
||||
)).scalar_one_or_none()
|
||||
|
||||
Reference in New Issue
Block a user