fix(maintenance): a late return, once back, stays back, and every reader agrees (#6175)
18032a4 re-checked a "not back in service" job every minute with no end,
so any later swarm wobble on the host (radarr's failed task during the
downloads move, say) flipped maintenance_failed back to 1: five false
firings and ten emails for Docker02 overnight on 10-19. And the overview's
finding never learned the rule at all, so it still said Docker02's
maintenance failed three days on.
The job now keeps came_back_at, set the first time its host is found back.
maintenance.still_failed(job) is the one answer: the figure, the overview,
the cadence page's run rows and the host panel all ask it. A failed update
or a timed-out job still flags its host until a job succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# plugins/host_agent/migrations/versions/host_agent_011_came_back.py
|
||||
"""host_agent: when a job failed for not coming back into service in time
|
||||
first found its host back (#6175)
|
||||
|
||||
Revision ID: host_agent_011_came_back
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision: str = "host_agent_011_came_back"
|
||||
down_revision: Union[str, None] = "host_agent_010_maint_check"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("host_maintenance_jobs",
|
||||
sa.Column("came_back_at", sa.DateTime(timezone=True), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("host_maintenance_jobs", "came_back_at")
|
||||
Reference in New Issue
Block a user