feat(subscriptions): newly added enabled sources start in backfill mode
A freshly created subscription has no gallery-dl archive yet, so the first poll in tick mode would walk forever — exit:20 doesn't trip until 20 contiguous archive-hits, and there are none. The wall-clock cap kicks in mid-walk, and the partial-success classifier (plan #544) gracefully labels it status=ok, but the operator still wonders why their new subscription isn't grabbing the back-catalog. Pre-arm `backfill_runs_remaining = 3` on create() when `enabled=True`. Same default as the manual "Deep scan" button, same auto-decrement and auto-reset rules — once the queue drains (clean exit + zero new files) or the budget runs out, tick mode resumes naturally. Disabled sources (sidecar synthetics with `url='sidecar:<platform>:<slug>'` that arrive disabled = False, or operator-added sources that start disabled deliberately) skip the pre-arm — they're never polled, no budget to waste.
This commit is contained in:
@@ -87,6 +87,11 @@ class SourceRecord:
|
||||
|
||||
_EDITABLE = {"enabled", "url", "config_overrides", "check_interval_override", "platform"}
|
||||
|
||||
# Plan #544 follow-up: newly created enabled sources pre-arm backfill so
|
||||
# their first N polls walk gallery-dl's full post history with the longer
|
||||
# timeout (matches the manual "Deep scan" button's default).
|
||||
_NEW_SOURCE_BACKFILL_RUNS = 3
|
||||
|
||||
|
||||
class SourceService:
|
||||
def __init__(self, session: AsyncSession):
|
||||
@@ -204,10 +209,21 @@ class SourceService:
|
||||
select(func.count(Source.id)).where(Source.artist_id == artist_id)
|
||||
)).scalar_one()
|
||||
|
||||
# Plan #544 follow-up: a freshly added subscription has no archive
|
||||
# yet, so the first few polls would walk the full post history in
|
||||
# tick mode and trip exit:20 after ~20 contiguous archive hits —
|
||||
# except there are none yet, so tick mode would walk forever and
|
||||
# blow the wall-clock cap. Pre-arm backfill so the initial syncs
|
||||
# use the longer timeout + skip:True walk. Tick mode resumes once
|
||||
# the budget is spent or the queue drains.
|
||||
# Disabled sources (incl. sidecar synthetics, url='sidecar:...')
|
||||
# are never polled, so leave their counter at 0.
|
||||
backfill_runs = _NEW_SOURCE_BACKFILL_RUNS if enabled else 0
|
||||
source = Source(
|
||||
artist_id=artist_id, platform=platform, url=url,
|
||||
enabled=enabled, config_overrides=config_overrides,
|
||||
check_interval_override=check_interval_override,
|
||||
backfill_runs_remaining=backfill_runs,
|
||||
)
|
||||
self.session.add(source)
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user