fix(tags): time-box + self-resume the tag standardization (stop the 40-min timeout)
normalize_tags_task timed out at the 40-min hard limit on a large back-catalog
(the first run recases the whole booru vocabulary) — operator-flagged, and it
monopolized the concurrency-1 maintenance queue while doing so.
normalize_existing_tags now takes time_budget_seconds: the live run stops
cleanly at the budget and reports {partial, remaining}. The task runs 600s
chunks and re-enqueues itself until nothing remains (idempotent — commits per
group, so the next chunk skips already-canonical groups). Short chunks let the
recovery sweep and other maintenance tasks interleave instead of being blocked
for 40 minutes.
Frontend: the Standardize button is now fire-and-forget ("Queued — runs in the
background; re-run Preview to confirm") instead of poll-until-done, which would
have falsely reported "complete" after the first chunk.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -123,6 +123,29 @@ async def test_live_merges_case_variants_and_repoints_images(db):
|
||||
assert assoc == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_time_budget_stops_partial_and_reports_remaining(db):
|
||||
"""A zero budget stops before the first group so a huge back-catalog can't
|
||||
run the task into the Celery time limit; it reports partial/remaining so the
|
||||
caller re-enqueues to continue (operator-flagged 40-min timeout 2026-06-07)."""
|
||||
await _raw_tag(db, "alpha", TagKind.general)
|
||||
await _raw_tag(db, "beta", TagKind.general)
|
||||
|
||||
summary = await normalize_existing_tags(db, dry_run=False, time_budget_seconds=0)
|
||||
|
||||
assert summary["partial"] is True
|
||||
assert summary["groups_processed"] == 0
|
||||
assert summary["remaining"] == summary["total_changes"] >= 2
|
||||
# Nothing recased yet — the budget cut before any work.
|
||||
assert await _count_named(db, "alpha", TagKind.general) == 1
|
||||
|
||||
# A full (unbudgeted) run finishes and reports nothing remaining.
|
||||
done = await normalize_existing_tags(db, dry_run=False)
|
||||
assert done["partial"] is False
|
||||
assert done["remaining"] == 0
|
||||
assert await _count_named(db, "Alpha", TagKind.general) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_idempotent_second_run_is_noop(db):
|
||||
await _raw_tag(db, "kafka", TagKind.character)
|
||||
|
||||
Reference in New Issue
Block a user