audit-g1: six one-liner drift fixes from 2026-06-02 audit #49

Merged

bvandeusen merged 2 commits from dev into main 2026-06-02 13:29:17 -04:00

Conversation 0 Commits 2 Files Changed 7

+84 -20

bvandeusen commented 2026-06-02 13:29:12 -04:00

Owner

Copy Link

Copy Source

Summary

Group 1 of the 2026-06-02 multi-system drift audit (workflow wf_bbe3fdb1-e62, 83 agents, adversarially verified). Six small fixes, each on its own audit finding.

• G1.1 — BACKFILL_TIMEOUT_SECONDS capped at 1170s (under Celery's 1200s hard time_limit). Backfill subprocesses no longer get SIGKILLed before subprocess.TimeoutExpired fires, so stdout/stderr aren't lost and backfill_runs_remaining actually decrements. Re-creates the same rationale as the tick 870s/900s pairing.
• G1.2 — recover_interrupted_tasks orphan UPDATE now stamps finished_at; without it cleanup_old_tasks' WHERE finished_at < cutoff never reaped orphan-swept rows, leaving them as permanent table tenants. recover_stalled_task_runs also now stamps duration_ms so sweep-closed rows carry duration like normally-finalized rows.
• G1.3 — ExtensionService.quick_add_source arms NEW_SOURCE_BACKFILL_RUNS=3 on Source creation (mirroring SourceService.create). Without it, Firefox quick-add on a creator with >20 unsynced posts walked the full feed to subprocess timeout. Constant renamed (dropped leading underscore) to allow cross-module import.
• G1.4 — gallery_dl.verify() accepts TIER_LIMITED as auth-success alongside NO_NEW_CONTENT. The download path (line 712) already treats TIER_LIMITED as success — TIER_LIMITED proves auth reached the post and was told it was tier-gated. Operators no longer rotate working cookies because verify showed red.
• G1.5 — prune_unused_tags rewritten as a single DELETE with the same NOT-IN predicate find_unused_tags uses, instead of SELECT-ids → DELETE-WHERE-IN. Removes the psycopg 65535-parameter cliff that would have surfaced on a tag explosion.
• G1.6 — credentials.upload() reflects the returned record into the store cache (.set(platform, rec)) instead of evicting it. Eliminates the gap where the card briefly showed "no credential" between upload and the caller's follow-up loadAll().

Test Plan

• Trigger a long-running backfill (a creator with hundreds of posts) — confirm the event lands with status ok or partial, not error with empty logs; confirm backfill_runs_remaining decremented.
• Run an orphan ImportTask sweep (or wait for the periodic), then check cleanup_old_tasks after 7 days — orphaned rows should be reaped.
• Firefox extension Quick-Add a creator with >20 backlog posts — first tick should walk in backfill mode (longer timeout) instead of red-flagging.
• Verify credentials on a Patreon source where some posts are tier-gated — Settings card should go green, not "could not confirm."
• Open Credentials settings and upload a new cookie file — card should update immediately to show the new credential, no blank flash.

Audit reference

This PR closes Group 1 of plan task Scribe #551 (FC integrity-audit drift remediation). Groups 2-5 (async race fixes, lifecycle batch, status-enum miss batch, architectural debt) to follow.

🤖 Generated with Claude Code

## Summary Group 1 of the 2026-06-02 multi-system drift audit (workflow `wf_bbe3fdb1-e62`, 83 agents, adversarially verified). Six small fixes, each on its own audit finding. - **G1.1** — `BACKFILL_TIMEOUT_SECONDS` capped at 1170s (under Celery's 1200s hard `time_limit`). Backfill subprocesses no longer get SIGKILLed before `subprocess.TimeoutExpired` fires, so stdout/stderr aren't lost and `backfill_runs_remaining` actually decrements. Re-creates the same rationale as the tick 870s/900s pairing. - **G1.2** — `recover_interrupted_tasks` orphan UPDATE now stamps `finished_at`; without it `cleanup_old_tasks`' `WHERE finished_at < cutoff` never reaped orphan-swept rows, leaving them as permanent table tenants. `recover_stalled_task_runs` also now stamps `duration_ms` so sweep-closed rows carry duration like normally-finalized rows. - **G1.3** — `ExtensionService.quick_add_source` arms `NEW_SOURCE_BACKFILL_RUNS=3` on Source creation (mirroring `SourceService.create`). Without it, Firefox quick-add on a creator with >20 unsynced posts walked the full feed to subprocess timeout. Constant renamed (dropped leading underscore) to allow cross-module import. - **G1.4** — `gallery_dl.verify()` accepts `TIER_LIMITED` as auth-success alongside `NO_NEW_CONTENT`. The download path (line 712) already treats `TIER_LIMITED` as success — `TIER_LIMITED` proves auth reached the post and was told it was tier-gated. Operators no longer rotate working cookies because verify showed red. - **G1.5** — `prune_unused_tags` rewritten as a single DELETE with the same NOT-IN predicate `find_unused_tags` uses, instead of SELECT-ids → DELETE-WHERE-IN. Removes the psycopg 65535-parameter cliff that would have surfaced on a tag explosion. - **G1.6** — `credentials.upload()` reflects the returned record into the store cache (`.set(platform, rec)`) instead of evicting it. Eliminates the gap where the card briefly showed "no credential" between upload and the caller's follow-up `loadAll()`. ## Test Plan - [ ] Trigger a long-running backfill (a creator with hundreds of posts) — confirm the event lands with status `ok` or `partial`, not `error` with empty logs; confirm `backfill_runs_remaining` decremented. - [ ] Run an orphan ImportTask sweep (or wait for the periodic), then check `cleanup_old_tasks` after 7 days — orphaned rows should be reaped. - [ ] Firefox extension Quick-Add a creator with >20 backlog posts — first tick should walk in backfill mode (longer timeout) instead of red-flagging. - [ ] Verify credentials on a Patreon source where some posts are tier-gated — Settings card should go green, not "could not confirm." - [ ] Open Credentials settings and upload a new cookie file — card should update immediately to show the new credential, no blank flash. ## Audit reference This PR closes Group 1 of plan task `Scribe #551` (FC integrity-audit drift remediation). Groups 2-5 (async race fixes, lifecycle batch, status-enum miss batch, architectural debt) to follow. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 2 commits 2026-06-02 13:29:12 -04:00

fix(audit-g1): six one-liner drift fixes from 2026-06-02 audit ... 3898ce7be4

- BACKFILL_TIMEOUT_SECONDS 1800→1170: keep the subprocess timeout
  30s below Celery's hard time_limit=1200 so SIGKILL doesn't beat
  TimeoutExpired (matched the tick 870s/900s rationale). Backfill
  runs that hit the cap let the next tick continue via the archive.

- recover_interrupted_tasks orphan UPDATE now stamps finished_at;
  without it cleanup_old_tasks' WHERE finished_at<cutoff never
  reaped orphan-swept rows. recover_stalled_task_runs also now sets
  duration_ms (matches celery_signals.finalize's millisecond math).

- ExtensionService.quick_add_source arms NEW_SOURCE_BACKFILL_RUNS=3
  on Source creation, mirroring SourceService.create. Without it,
  Firefox quick-add on a creator with >20 unsynced posts walked the
  full feed until subprocess timeout. Renamed the constant from
  _NEW_SOURCE_BACKFILL_RUNS so it can be imported cross-module.

- gallery_dl.verify() accepts TIER_LIMITED as auth-success alongside
  NO_NEW_CONTENT — the download path (line 712) already does, and
  TIER_LIMITED proves auth reached the post and was told it was
  tier-gated. Verify endpoint previously showed red on this and
  prompted operators to rotate working cookies.

- prune_unused_tags now runs a single DELETE with the NOT-IN
  predicate find_unused_tags uses, instead of SELECT-ids →
  DELETE-WHERE-IN. Removes the psycopg 65535-param cliff that
  would have surfaced on a tag explosion (>65k unused tags).

- credentials.upload() reflects the returned record into the store
  cache (`.set(platform, rec)`) instead of evicting it; previously
  the card briefly rendered "no credential" between upload and
  loadAll().

fix(test): credentials.upload reflects record into cache (not invalidate) ... 80ef9bce48

Stale assertion pinned the buggy pre-G1.6 behavior (the test name
"upload invalidates the cache" literally describes the bug). The
audit's correction makes upload mirror the returned record into the
store so the card updates immediately — update the assertion to
match the corrected behavior.

bvandeusen merged commit 93e37681b7 into main 2026-06-02 13:29:17 -04:00

bvandeusen referenced this pull request 2026-06-02 19:09:44 -04:00

audit-g5 final + ML threshold default + kebab menu fix #54

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#49