bvandeusen
f2e9ae07dc
scan_library_for_rule ran one 2-hour pass that timed out on large libraries and held the concurrency-1 maintenance queue the whole time, starving vacuum/backup/ normalize (operator-flagged — it was the dominant entry in the 24h failures). It now runs ~10-min chunks and re-enqueues itself until the library is exhausted, matching the operator's preferred pattern (reasonable timeout → retry queued → other things process between). New columns (alembic 0039): resume_after_id persists the keyset cursor so a chunk continues where the last left off; last_progress_at lets the recovery sweep tell a progressing multi- chunk audit from a dead one (it now measures staleness from last_progress_at, not started_at). Matches accumulate across chunks. soft/hard limits dropped 2h→15/16.7 min so the in-chunk budget fires first; a soft-limit backstop re-enqueues to resume instead of erroring the whole run. Tests: time-box → re-enqueue (status stays running); resume carries prior matches and appends new ones. Existing full-scan tests unchanged (small sets finish in one chunk). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
41 lines
1.2 KiB
Python
41 lines
1.2 KiB
Python
"""library_audit_run: resume cursor + progress timestamp for chunked scans
|
|
|
|
Revision ID: 0039
|
|
Revises: 0038
|
|
Create Date: 2026-06-07
|
|
|
|
scan_library_for_rule used to run one 2h pass that timed out on large libraries
|
|
and monopolized the concurrency-1 maintenance queue (operator-flagged). It now
|
|
runs short time-boxed chunks that re-enqueue: `resume_after_id` persists the
|
|
keyset cursor so the next chunk continues where it left off, and
|
|
`last_progress_at` lets the recovery sweep tell a progressing multi-chunk audit
|
|
from a genuinely stuck one.
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision: str = "0039"
|
|
down_revision: Union[str, None] = "0038"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column(
|
|
"library_audit_run",
|
|
sa.Column(
|
|
"resume_after_id", sa.Integer, nullable=False, server_default="0"
|
|
),
|
|
)
|
|
op.add_column(
|
|
"library_audit_run",
|
|
sa.Column("last_progress_at", sa.DateTime(timezone=True), nullable=True),
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("library_audit_run", "last_progress_at")
|
|
op.drop_column("library_audit_run", "resume_after_id")
|