feat(download): tick/backfill modes + partial-success classifier (plan #544)

Routine subscription polls walked the entire post history every tick
even when nothing had changed, because gallery-dl's default `skip: True`
continues iterating archived posts. A creator with ~550 archived posts
(Knuxy patreon) saturates the 870s wall-clock cap before completing,
even with zero downloads needed. Plus, a tier-limited run that
downloaded hundreds of files but ran out the clock should be a
warning, not an error.

Two coupled changes, both operator-flagged 2026-06-01:

* **Tick mode (default, cron polls).** New `TICK_SKIP_VALUE = "exit:20"`
  asks gallery-dl to exit after 20 contiguous archived items. Fresh
  subscriptions + new-content cases still walk normally; established
  subscription with zero new content exits in ~30s of HEAD requests
  instead of pegging the timeout. 20 (not 5) gives headroom against
  paywall warnings interleaving with archived items.
* **Backfill mode (explicit, operator-triggered).** Sticky for N runs
  via new `Source.backfill_runs_remaining` (alembic 0031). While > 0,
  downloads use `skip: True` + 1800s timeout. Auto-decrements per run
  with early-reset to 0 when a clean run finds zero files (queue
  drained). N defaults to 3 — multiple runs give the system enough
  budget to finish a deep walk across timeout boundaries. New
  `POST /api/sources/{id}/backfill` arms the source; "Deep scan"
  button on each SourceRow (chip shows remaining count) wires it.

Plus partial-success classifier: non-zero gallery-dl exit + ≥1 file
downloaded + no source-level error fires `ErrorType.PARTIAL`, which
download_service maps to `status=\"ok\"`. The run did real work; the
next tick continues via gallery-dl's archive. No more red events for
"timed out mid-walk after downloading 300 files."

Retires `SourceConfig.skip_existing` — skip value is now derived from
the source state and passed as a separate `skip_value` parameter
through download() / _build_config_for_source(). `GD_DEFAULTS` drops
the now-dead key (was inert data after this refactor).

Tests cover:
* tick + backfill skip-value emission in _build_config_for_source
* PARTIAL classifier branch + TIER_LIMITED-wins-over-PARTIAL ordering
* SourceService.set_backfill_runs validation + persistence
* /api/sources/{id}/backfill 200/400/404 paths
* download_service auto-decrement / auto-reset / tick-mode-no-touch
* PARTIAL → status=ok in the orchestrator (no consecutive_failures bump)

backend/app/services/gallery_dl.py

@@ -39,9 +39,29 @@ class ErrorType(StrEnum):
     HTTP_ERROR = "http_error"
     UNSUPPORTED_URL = "unsupported_url"
     VALIDATION_FAILED = "validation_failed"
+    # Run made real progress (downloaded ≥1 file) but did not finish in the
+    # subprocess budget. Distinct from UNKNOWN_ERROR — the downstream status
+    # mapping classifies this as "ok" because the next tick continues.
+    PARTIAL = "partial"
     UNKNOWN_ERROR = "unknown_error"
 
 
+# Tick mode (routine cron polls): skip ≤20 contiguous already-archived
+# items, then exit gallery-dl. Established subscription with zero new
+# content exits in ~30s of HEAD requests instead of walking to the bottom
+# of the post history (which can be hours for prolific creators). 20 (not
+# 5) is operator-set headroom against any edge case where paywalled or
+# otherwise-non-downloadable items might interleave with archived ones —
+# 20 contiguous HEADs is still negligible.
+TICK_SKIP_VALUE = "exit:20"
+
+# Backfill mode (operator-triggered deep scan): walk the full history.
+# Source.backfill_runs_remaining > 0 selects this mode; the longer
+# timeout below absorbs creators with thousands of posts.
+BACKFILL_SKIP_VALUE = True
+BACKFILL_TIMEOUT_SECONDS = 1800
+
+
 # 30 seconds shy of download_source's Celery soft_time_limit (900s, see
 # tasks/download.py:32). subprocess.run MUST raise TimeoutExpired before
 # Celery raises SoftTimeLimitExceeded — otherwise Celery wins the race,
@@ -56,12 +76,19 @@ _DEFAULT_GDL_TIMEOUT_SECONDS = 870
 
 @dataclass
 class SourceConfig:
+    """Per-source overrides loaded from Source.config_overrides JSON.
+
+    Note: the gallery-dl `skip` value (tick vs backfill, see TICK_SKIP_VALUE /
+    BACKFILL_SKIP_VALUE) is NOT carried here — it derives from the
+    Source.backfill_runs_remaining column at the download_service layer
+    and is passed to _build_config_for_source as `skip_value`. Same for
+    the per-run subprocess timeout.
+    """
     content_types: list[str] = field(default_factory=lambda: ["all"])
     sleep: float | None = None
     sleep_request: float | None = None
     directory_pattern: str | None = None
     filename_pattern: str | None = None
-    skip_existing: bool = True
     save_metadata: bool = True
     timeout: int = _DEFAULT_GDL_TIMEOUT_SECONDS
 
@@ -73,7 +100,6 @@ class SourceConfig:
             sleep_request=data.get("sleep_request"),
             directory_pattern=data.get("directory_pattern"),
             filename_pattern=data.get("filename_pattern"),
-            skip_existing=data.get("skip_existing", True),
             save_metadata=data.get("save_metadata", True),
             timeout=data.get("timeout", _DEFAULT_GDL_TIMEOUT_SECONDS),
         )
@@ -245,7 +271,18 @@ class GalleryDLService:
         platform: str,
         source_config: SourceConfig,
         artist_slug: str,
+        skip_value: bool | str = BACKFILL_SKIP_VALUE,
     ) -> dict:
+        """`skip_value` controls gallery-dl's archive-walk behavior:
+            - True (BACKFILL_SKIP_VALUE): walk full post history, skipping
+              archived items but continuing past them. Used in backfill mode.
+            - "exit:20" (TICK_SKIP_VALUE): exit gallery-dl after 20
+              contiguous archived items. Used in tick (routine catch-up)
+              mode for fast no-op syncs on creators with deep history.
+            - False: don't skip — redownload everything (not used in FC).
+        The caller (download_service) chooses based on
+        Source.backfill_runs_remaining.
+        """
         config = json.loads(json.dumps(self._get_default_config()))  # deep copy
 
         destination = str(self.images_root / artist_slug / platform)
@@ -255,7 +292,7 @@ class GalleryDLService:
             config["extractor"]["sleep"] = source_config.sleep
         if source_config.sleep_request is not None:
             config["extractor"]["sleep-request"] = source_config.sleep_request
-        config["extractor"]["skip"] = source_config.skip_existing
+        config["extractor"]["skip"] = skip_value
 
         if source_config.save_metadata:
             config["extractor"]["postprocessors"] = [
@@ -394,6 +431,22 @@ class GalleryDLService:
                     f"Subscription tier does not grant access to {count} post{'s' if count != 1 else ''}",
                 )
 
+        # Partial-success: the subprocess exited non-zero (typically because
+        # the wall-clock timeout fired mid-walk), but it had downloaded ≥1
+        # file by then and no source-level error category fired. The work
+        # the run DID do is real; gallery-dl's archive will pick up where
+        # it left off on the next tick. Mapped to status="ok" downstream
+        # (download_service.py) so this doesn't flag the source as
+        # "needs attention." Operator-flagged 2026-06-01 after a Knuxy
+        # patreon run downloaded hundreds of files then ran red on timeout.
+        files_downloaded = self._count_downloaded_files(stdout)
+        if not has_actual_error and files_downloaded > 0:
+            return (
+                ErrorType.PARTIAL,
+                f"Downloaded {files_downloaded} file{'s' if files_downloaded != 1 else ''}; "
+                "run did not complete in budget — next tick will continue",
+            )
+
         return ErrorType.UNKNOWN_ERROR, f"Unknown error (return code: {return_code})"
 
     def _count_downloaded_files(self, stdout: str) -> int:
@@ -545,6 +598,7 @@ class GalleryDLService:
         source_config: SourceConfig | None = None,
         cookies_path: str | None = None,
         auth_token: str | None = None,
+        skip_value: bool | str = BACKFILL_SKIP_VALUE,
     ) -> DownloadResult:
         start_time = time.time()
         started_at = datetime.now(UTC).isoformat()
@@ -552,7 +606,9 @@ class GalleryDLService:
         if source_config is None:
             source_config = SourceConfig()
 
-        config = self._build_config_for_source(platform, source_config, artist_slug)
+        config = self._build_config_for_source(
+            platform, source_config, artist_slug, skip_value=skip_value,
+        )
 
         if cookies_path:
             config["extractor"]["cookies"] = cookies_path