fix(audit-g1): six one-liner drift fixes from 2026-06-02 audit
CI / lint (push) Successful in 5s
CI / backend-lint-and-test (push) Successful in 22s
CI / frontend-build (push) Failing after 27s
CI / intimp (push) Successful in 3m40s
CI / intapi (push) Successful in 8m6s
CI / intcore (push) Successful in 9m3s

- BACKFILL_TIMEOUT_SECONDS 1800→1170: keep the subprocess timeout
  30s below Celery's hard time_limit=1200 so SIGKILL doesn't beat
  TimeoutExpired (matched the tick 870s/900s rationale). Backfill
  runs that hit the cap let the next tick continue via the archive.

- recover_interrupted_tasks orphan UPDATE now stamps finished_at;
  without it cleanup_old_tasks' WHERE finished_at<cutoff never
  reaped orphan-swept rows. recover_stalled_task_runs also now sets
  duration_ms (matches celery_signals.finalize's millisecond math).

- ExtensionService.quick_add_source arms NEW_SOURCE_BACKFILL_RUNS=3
  on Source creation, mirroring SourceService.create. Without it,
  Firefox quick-add on a creator with >20 unsynced posts walked the
  full feed until subprocess timeout. Renamed the constant from
  _NEW_SOURCE_BACKFILL_RUNS so it can be imported cross-module.

- gallery_dl.verify() accepts TIER_LIMITED as auth-success alongside
  NO_NEW_CONTENT — the download path (line 712) already does, and
  TIER_LIMITED proves auth reached the post and was told it was
  tier-gated. Verify endpoint previously showed red on this and
  prompted operators to rotate working cookies.

- prune_unused_tags now runs a single DELETE with the NOT-IN
  predicate find_unused_tags uses, instead of SELECT-ids →
  DELETE-WHERE-IN. Removes the psycopg 65535-param cliff that
  would have surfaced on a tag explosion (>65k unused tags).

- credentials.upload() reflects the returned record into the store
  cache (`.set(platform, rec)`) instead of evicting it; previously
  the card briefly rendered "no credential" between upload and
  loadAll().
This commit is contained in:
2026-06-02 11:24:05 -04:00
parent 91be9df671
commit 3898ce7be4
6 changed files with 75 additions and 16 deletions
+19 -2
View File
@@ -58,8 +58,19 @@ TICK_SKIP_VALUE = "exit:20"
# Backfill mode (operator-triggered deep scan): walk the full history.
# Source.backfill_runs_remaining > 0 selects this mode; the longer
# timeout below absorbs creators with thousands of posts.
#
# 30 seconds shy of Celery's hard `time_limit=1200` on download_source
# (tasks/download.py:33). subprocess.run MUST raise TimeoutExpired
# before Celery SIGKILLs the worker — same rationale as the tick
# default at line 74. The audit (2026-06-02) caught this at 1800,
# guaranteeing SIGKILL on any backfill that ran to its subprocess
# budget: stdout/stderr lost, backfill_runs_remaining never
# decrements, recovery sweep stamps generic "stranded" 30 min later.
# Recreates the exact Knuxy #38275 failure mode the tick 870s default
# was added to prevent. backfill_runs_remaining=3 still gives ~58
# minutes of cumulative walk across three runs for prolific creators.
BACKFILL_SKIP_VALUE = True
BACKFILL_TIMEOUT_SECONDS = 1800
BACKFILL_TIMEOUT_SECONDS = 1170
# 30 seconds shy of download_source's Celery soft_time_limit (900s, see
@@ -844,7 +855,13 @@ class GalleryDLService:
),
)
etype, msg = self._categorize_error(proc.returncode, proc.stdout, proc.stderr)
if proc.returncode == 0 or etype == ErrorType.NO_NEW_CONTENT:
# TIER_LIMITED proves auth worked — gallery-dl reached the
# post, was told it's tier-gated. The download path treats
# this as success (line 712); verify must too, or operators
# rotate working cookies for no reason. Audit 2026-06-02.
if proc.returncode == 0 or etype in (
ErrorType.NO_NEW_CONTENT, ErrorType.TIER_LIMITED,
):
return True, "Credentials valid — the feed authenticated."
if etype == ErrorType.AUTH_ERROR:
return False, msg