bvandeusenandClaude Opus 5.5
a9bf4b7f16
After Steward or its database was down, every agent's age and every series' last sample were old, so all agent-offline rules fired and all rules went no-data at once, clearing a minute later. With per-account alert email that would have mailed everyone a dozen notices per restart. A periodic job now ticks a Watch (steward/core/watch.py) once it has reached the database; the Watch says since when runs have been unbroken (process start, or the end of a gap over 120 s). seconds_since_seen and the no-data sweep count silence from no earlier than that, so a source must stay quiet while Steward is watching before it counts as silent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
33 lines
1.2 KiB
Python
33 lines
1.2 KiB
Python
"""Since when Steward has been watching without a break (#5441): a job's
|
|
Watch starts at its first run, holds across runs a minute apart, and starts
|
|
again after a gap long enough to mean Steward was not watching."""
|
|
from datetime import datetime, timedelta, timezone
|
|
from types import SimpleNamespace as NS
|
|
|
|
from steward.core.watch import GAP_S, Watch, watch
|
|
|
|
T0 = datetime(2026, 10, 9, 2, 0, tzinfo=timezone.utc)
|
|
|
|
|
|
def test_the_first_run_starts_the_watch_and_steady_runs_keep_it():
|
|
w = Watch()
|
|
assert w.tick(T0) == T0
|
|
for minute in range(1, 30):
|
|
assert w.tick(T0 + timedelta(minutes=minute)) == T0
|
|
|
|
|
|
def test_a_gap_longer_than_two_missed_runs_starts_it_again():
|
|
w = Watch()
|
|
w.tick(T0)
|
|
assert w.tick(T0 + timedelta(seconds=GAP_S)) == T0 # just within
|
|
back = T0 + timedelta(seconds=2 * GAP_S + 1)
|
|
assert w.tick(back) == back # Steward was away
|
|
assert w.tick(back + timedelta(minutes=1)) == back
|
|
|
|
|
|
def test_each_job_has_its_own_watch_on_its_own_app():
|
|
a, b = NS(), NS()
|
|
assert watch(a, "x") is watch(a, "x")
|
|
assert watch(a, "x") is not watch(a, "y")
|
|
assert watch(a, "x") is not watch(b, "x")
|