fix: a stopped source is not a failing one, and cannot be deep-scanned (4279)
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m3s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m12s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m16s
CI / lint (push) Successful in 2s
CI / extension-version (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m3s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m12s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m16s
Ebi77 sat in the "1 source is failing" banner for six days with no action available, reading `stranded by recovery sweep (no terminal status after time_limit)`. Four things lined up: 1. The membership sweep did its job — saw `former_patron`, disabled the source, cleared its failure state. Clean at 02:50. 2. Twenty minutes later a deep scan was armed on it. `/backfill` had a credential pre-flight but NO `enabled` guard, while `/check` has carried one all along. The two trigger endpoints disagreed, and the ungated one is the one that arms the long walk. 3. Without a membership the walk cannot finish, never reaches a terminal status, and the recovery sweep strands it with consecutive_failures = 1. 4. Nothing could clear that. A disabled source is never scheduled, so no successful run resets the count; `SourceService.update` clears only on an explicit disable and it was already disabled; and the banner's Retry routes to `/check`, which refuses a disabled source. The card offered a button structurally incapable of acting on the only source it was showing. `failing_sources_clause()` now means "enabled AND erroring". That also settles a disagreement its two callers already had: the scheduler's count paired it with `enabled.is_(True)` and `SourceService.list(failing=True)` did not, so one counted Ebi77 and the other did not — exactly the drift the note above that function warns about, which is why the test belongs IN the predicate rather than beside it. The scheduler's now-duplicate clause is dropped so one place decides. `/backfill` gains the guard for start/recover/recapture. `stop` stays open on a disabled source, or arming becomes a one-way door. Migration 0101 clears failure state on sources that are already disabled — the predicate fixes what the surfaces report, not what the rows carry, and the rows are why the operator had no way out (lesson #4202). It matches what `update` already does on an explicit disable, so rows disabled by any other path come into line. Enabled sources are untouched: a real failure on a live source must keep showing, which the second new test pins. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -682,3 +682,83 @@ async def test_new_disabled_source_skips_backfill(db):
|
||||
enabled=False,
|
||||
)
|
||||
assert rec.backfill_runs_remaining == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_disabled_source_is_not_failing(db):
|
||||
"""#4279: "stopped because you no longer subscribe" is not "failing".
|
||||
|
||||
Ebi77 was stopped by the membership sweep as `former_patron`, then a deep
|
||||
scan armed on it got stranded by the recovery sweep. The banner showed it
|
||||
for six days with no action available: a disabled source is never
|
||||
scheduled (so no run clears the count), `update` only clears on an
|
||||
explicit disable (it was already disabled), and Retry routes to /check,
|
||||
which refuses a disabled source.
|
||||
"""
|
||||
artist = await _artist(db)
|
||||
svc = SourceService(db)
|
||||
rec = await svc.create(
|
||||
artist_id=artist.id, platform="patreon",
|
||||
url="https://patreon.com/stopped",
|
||||
)
|
||||
source = (await db.execute(
|
||||
select(Source).where(Source.id == rec.id)
|
||||
)).scalar_one()
|
||||
source.enabled = False
|
||||
source.consecutive_failures = 1
|
||||
source.last_error = "stranded by recovery sweep (no terminal status after time_limit)"
|
||||
await db.commit()
|
||||
|
||||
assert [r.id for r in await svc.list(failing=True)] == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_enabled_source_that_errors_is_still_failing(db):
|
||||
"""The other half — folding `enabled` in must not hide a real failure on
|
||||
a live source."""
|
||||
artist = await _artist(db)
|
||||
svc = SourceService(db)
|
||||
rec = await svc.create(
|
||||
artist_id=artist.id, platform="patreon", url="https://patreon.com/live",
|
||||
)
|
||||
source = (await db.execute(
|
||||
select(Source).where(Source.id == rec.id)
|
||||
)).scalar_one()
|
||||
source.enabled = True
|
||||
source.consecutive_failures = 2
|
||||
source.last_error = "auth failed"
|
||||
await db.commit()
|
||||
|
||||
assert [r.id for r in await svc.list(failing=True)] == [rec.id]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_failing_list_and_the_ribbon_count_agree(db):
|
||||
"""The two callers of `failing_sources_clause` disagreed before #4279:
|
||||
the scheduler's count paired it with `enabled.is_(True)`, the list did
|
||||
not, so one counted Ebi77 and the other did not. The predicate owns the
|
||||
whole definition now — assert the two surfaces match rather than trusting
|
||||
that they were both updated."""
|
||||
from backend.app.services.scheduler_service import scheduler_status
|
||||
|
||||
artist = await _artist(db)
|
||||
svc = SourceService(db)
|
||||
for url, enabled, fails in (
|
||||
("https://patreon.com/one", True, 3),
|
||||
("https://patreon.com/two", False, 1),
|
||||
("https://patreon.com/three", True, 0),
|
||||
):
|
||||
rec = await svc.create(
|
||||
artist_id=artist.id, platform="patreon", url=url,
|
||||
)
|
||||
s = (await db.execute(
|
||||
select(Source).where(Source.id == rec.id)
|
||||
)).scalar_one()
|
||||
s.enabled = enabled
|
||||
s.consecutive_failures = fails
|
||||
await db.commit()
|
||||
|
||||
listed = len(await svc.list(failing=True))
|
||||
counts = await scheduler_status(db)
|
||||
assert listed == 1
|
||||
assert counts["failing_sources"] == listed
|
||||
|
||||
Reference in New Issue
Block a user