fix(maintenance): a host that came back late stops reading as failed once it is back (#6105)
Docker02's Sunday job updated and rebooted, then failed only because portal_app was short of its replica past the 30-minute settle. Its maintenance_failed figure stayed 1, and the alert kept firing, until a later job succeeds. A job failed for not coming back into service is checked again each minute and reads 0 once the host is back; a failed or timed-out update still reads 1 until a job succeeds. Also: compose.deploy.yml's syslog comment still said devices are told apart by address under Swarm (stale since #6096). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -356,7 +356,7 @@ METRICS: dict[str, tuple[str, str]] = {
|
||||
"updates_pending": ("Updates waiting", "packages the host's package manager would upgrade, checked hourly"),
|
||||
"updates_security": ("Security updates waiting", "of the updates waiting, those from a security source"),
|
||||
"reboot_required": ("Reboot required", "1 when the host needs a reboot to finish an update, 0 when not"),
|
||||
"maintenance_failed": ("Maintenance failed", "1 when the host's latest scheduled or manual update failed, timed out or didn't come back into service"),
|
||||
"maintenance_failed": ("Maintenance failed", "1 when the host's latest scheduled or manual update failed or timed out, or the host didn't come back into service and still isn't"),
|
||||
"temp_c_max": ("Hottest sensor", "the host's hottest real thermometer; inputs with nothing wired to them are left out"),
|
||||
"temp_c": ("Sensor temperature", "one temperature sensor (resource host:temp:label)"),
|
||||
"net_rx_bps": ("Network in", "bytes per second received, by the whole host or one interface"),
|
||||
|
||||
Reference in New Issue
Block a user