Ebi77 sat in the "1 source is failing" banner for six days with no action
available, reading `stranded by recovery sweep (no terminal status after
time_limit)`. Four things lined up:
1. The membership sweep did its job — saw `former_patron`, disabled the
source, cleared its failure state. Clean at 02:50.
2. Twenty minutes later a deep scan was armed on it. `/backfill` had a
credential pre-flight but NO `enabled` guard, while `/check` has carried
one all along. The two trigger endpoints disagreed, and the ungated one is
the one that arms the long walk.
3. Without a membership the walk cannot finish, never reaches a terminal
status, and the recovery sweep strands it with consecutive_failures = 1.
4. Nothing could clear that. A disabled source is never scheduled, so no
successful run resets the count; `SourceService.update` clears only on an
explicit disable and it was already disabled; and the banner's Retry routes
to `/check`, which refuses a disabled source. The card offered a button
structurally incapable of acting on the only source it was showing.
`failing_sources_clause()` now means "enabled AND erroring". That also settles
a disagreement its two callers already had: the scheduler's count paired it
with `enabled.is_(True)` and `SourceService.list(failing=True)` did not, so
one counted Ebi77 and the other did not — exactly the drift the note above
that function warns about, which is why the test belongs IN the predicate
rather than beside it. The scheduler's now-duplicate clause is dropped so one
place decides.
`/backfill` gains the guard for start/recover/recapture. `stop` stays open on
a disabled source, or arming becomes a one-way door.
Migration 0101 clears failure state on sources that are already disabled — the
predicate fixes what the surfaces report, not what the rows carry, and the
rows are why the operator had no way out (lesson #4202). It matches what
`update` already does on an explicit disable, so rows disabled by any other
path come into line. Enabled sources are untouched: a real failure on a live
source must keep showing, which the second new test pins.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
bvandeusenandClaude Opus 5
5aa8e3d81b