audit-g4: status-enum miss batch #52

Merged
bvandeusen merged 1 commits from dev into main 2026-06-02 16:15:00 -04:00
Owner

Summary

Group 4 of the 2026-06-02 multi-system drift audit. Five extension-miss findings where a status value was added on one side but a downstream consumer didn't pick it up — each had a real or latent operator-impacting consequence.

  • download_service._phase3_persist now branches on ImportResult.status for 'failed' and 'refreshed'. For 'failed' (today only: archive probe crash/timeout from _import_archive), the source file is unlinked so the filesystem scanner doesn't re-import and re-crash on the same archive forever. 'refreshed' is currently unreachable from the download path (no deep=True) but the importer's contract documents it; treat as attached.

  • Backfill auto-complete is now correctly gated. Was: if return_code == 0 and files_downloaded == 0: backfill_runs_remaining = 0. VALIDATION_FAILED exits the subprocess with returncode=0 and files_downloaded=0 when every file was quarantined, matching the predicate exactly — zeroing the operator's armed backfill budget on the first quarantine run. Now requires dl_result.success and error_type is None so only genuinely-empty successful runs drain the counter.

  • attach_in_place archive dispatch threads artist/source_row through _import_archive, archive-member _import_media calls, and _supersede. The filesystem-import path still uses bare _import_archive(source) and falls back to _resolve_artist path-walk. The download path now binds ImageProvenance to the explicit subscription Source instead of rediscovering by (artist_id, platform) — fixes the dispatch asymmetry the audit flagged.

  • Three FE handlers now recognize status:'deferred' from /api/sources/<id>/check:

    • SubscriptionsTab.onCheck was toasting "event #undefined" because it assumed download_event_id was always present
    • SubscriptionsTab.checkAll was counting deferred sources as queued, inflating the success tally
    • DownloadEventRow.onRetry was toasting "re-queued" when nothing was
    • Pattern matches DownloadsTab.onRetryAll which already had it.
  • celery_signals._queue_for maps backup / admin / library_audit prefixes to 'maintenance' (matching task_routes). Was returning 'default' for those rows, so per-queue dashboard filters and the per-queue threshold overrides added in G3 silently missed them. The function comment said "keep in sync with task_routes" — the audit caught the drift.

Test Plan

  • Run a tick with a corrupt archive in the download tree — confirm the event lands status=error AND the archive file is removed from disk (won't get re-imported on the next scan)
  • Arm a backfill (backfill_runs_remaining=3), trigger a run that quarantines every downloaded file (VALIDATION_FAILED) — confirm the counter decrements to 2 instead of zeroing
  • Download an archive that contains media — confirm extracted images have ImageProvenance rows linked to the subscription Source (not a freshly-derived one)
  • Trigger a single-source check on a platform in cooldown — Subscriptions tab toast says "Check deferred — platform in cooldown", not "event #undefined"
  • Check-all on a group where some sources are cooldown'd — toast tally shows N queued, M deferred (cooldown)
  • Wait for a backup / library_audit / admin task to land — confirm TaskRun.queue is 'maintenance' not 'default'

Audit reference

Closes Group 4 of plan task Scribe #551. Group 5 (architectural debt — centroid two-sources-of-truth, ErrorType in FailingSourcesCard, hardcoded platform tuple, modal route-leak, race-poisoning, Fernet key safety, fandom cache invalidation, dead backfill_thumbnails Beat entry, cleanup cluster) is the last group.

🤖 Generated with Claude Code

## Summary Group 4 of the 2026-06-02 multi-system drift audit. Five extension-miss findings where a status value was added on one side but a downstream consumer didn't pick it up — each had a real or latent operator-impacting consequence. - **`download_service._phase3_persist`** now branches on `ImportResult.status` for `'failed'` and `'refreshed'`. For `'failed'` (today only: archive probe crash/timeout from `_import_archive`), the source file is unlinked so the filesystem scanner doesn't re-import and re-crash on the same archive forever. `'refreshed'` is currently unreachable from the download path (no `deep=True`) but the importer's contract documents it; treat as `attached`. - **Backfill auto-complete is now correctly gated.** Was: `if return_code == 0 and files_downloaded == 0: backfill_runs_remaining = 0`. `VALIDATION_FAILED` exits the subprocess with `returncode=0` and `files_downloaded=0` when every file was quarantined, matching the predicate exactly — zeroing the operator's armed backfill budget on the *first* quarantine run. Now requires `dl_result.success and error_type is None` so only genuinely-empty successful runs drain the counter. - **`attach_in_place` archive dispatch threads `artist`/`source_row` through `_import_archive`**, archive-member `_import_media` calls, and `_supersede`. The filesystem-import path still uses bare `_import_archive(source)` and falls back to `_resolve_artist` path-walk. The download path now binds `ImageProvenance` to the explicit subscription `Source` instead of rediscovering by `(artist_id, platform)` — fixes the dispatch asymmetry the audit flagged. - **Three FE handlers now recognize `status:'deferred'`** from `/api/sources/<id>/check`: - `SubscriptionsTab.onCheck` was toasting "event #undefined" because it assumed `download_event_id` was always present - `SubscriptionsTab.checkAll` was counting deferred sources as queued, inflating the success tally - `DownloadEventRow.onRetry` was toasting "re-queued" when nothing was - Pattern matches `DownloadsTab.onRetryAll` which already had it. - **`celery_signals._queue_for` maps `backup` / `admin` / `library_audit` prefixes to `'maintenance'`** (matching `task_routes`). Was returning `'default'` for those rows, so per-queue dashboard filters and the per-queue threshold overrides added in G3 silently missed them. The function comment said "keep in sync with task_routes" — the audit caught the drift. ## Test Plan - [ ] Run a tick with a corrupt archive in the download tree — confirm the event lands status=`error` AND the archive file is removed from disk (won't get re-imported on the next scan) - [ ] Arm a backfill (`backfill_runs_remaining=3`), trigger a run that quarantines every downloaded file (`VALIDATION_FAILED`) — confirm the counter decrements to 2 instead of zeroing - [ ] Download an archive that contains media — confirm extracted images have `ImageProvenance` rows linked to the subscription Source (not a freshly-derived one) - [ ] Trigger a single-source check on a platform in cooldown — Subscriptions tab toast says "Check deferred — platform in cooldown", not "event #undefined" - [ ] Check-all on a group where some sources are cooldown'd — toast tally shows `N queued, M deferred (cooldown)` - [ ] Wait for a backup / library_audit / admin task to land — confirm `TaskRun.queue` is `'maintenance'` not `'default'` ## Audit reference Closes Group 4 of plan task `Scribe #551`. Group 5 (architectural debt — centroid two-sources-of-truth, ErrorType in FailingSourcesCard, hardcoded platform tuple, modal route-leak, race-poisoning, Fernet key safety, fandom cache invalidation, dead backfill_thumbnails Beat entry, cleanup cluster) is the last group. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
bvandeusen added 1 commit 2026-06-02 16:14:54 -04:00
fix(audit-g4): status-enum miss batch
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 14s
CI / frontend-build (push) Successful in 19s
CI / intimp (push) Successful in 3m48s
CI / intapi (push) Successful in 7m37s
CI / intcore (push) Successful in 8m18s
4bff1d8558
Five extension-miss findings from the 2026-06-02 audit, where a status
value was added on one side but a downstream consumer didn't pick it up.

- download_service._phase3_persist: explicit branches for
  ImportResult.status in ('failed','refreshed'). For 'failed' (archive
  probe crash from _import_archive), unlink the source file so the
  filesystem scanner doesn't re-import and re-crash on the same
  archive forever. 'refreshed' is currently unreachable from the
  download path (no deep=True) but matches the importer's documented
  contract; treat as 'attached'.

- gallery-dl backfill auto-complete now gates on dl_result.success +
  no error_type, not just return_code==0 + files_downloaded==0.
  VALIDATION_FAILED exits the subprocess with returncode=0 and
  files_downloaded=0 when every file was quarantined, matching the
  prior predicate exactly and zeroing the operator's armed backfill
  budget on the FIRST quarantine run instead of decrementing.

- attach_in_place archive dispatch now threads artist + source_row
  through _import_archive (and _import_media for archive members)
  and _supersede. The path-walk fallback (_resolve_artist) is still
  used by filesystem-import; the download path now binds
  ImageProvenance to the explicit subscription Source instead of
  rediscovering by (artist_id, platform).

- Three FE handlers now recognize status:'deferred' from
  /api/sources/<id>/check: SubscriptionsTab.onCheck (was toasting
  "event #undefined"), SubscriptionsTab.checkAll (was counting
  deferred as queued), DownloadEventRow.onRetry (was saying
  "re-queued" when nothing was). Pattern matches DownloadsTab.onRetryAll
  which already had it.

- celery_signals._queue_for now maps backup/admin/library_audit
  prefixes to 'maintenance' (matching task_routes). TaskRun.queue
  was returning 'default' for those rows, so per-queue dashboard
  filters and per-queue threshold overrides (added in G3) silently
  missed them.
bvandeusen merged commit 89b48f8f35 into main 2026-06-02 16:15:00 -04:00
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#52