Commit Graph
2 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5.5 715b58a09f feat(maintenance): a run spreads swarm services back out after its last host, moving only what the emptier node can start (#6398)
CI / lint (push) Successful in 2s
CI / unit (push) Failing after 1m36s
CI / integration (push) Successful in 1m47s
CI / publish (push) Skipped
Each weekly run drains Docker02 then Docker03, and the swarm never moves a
task back by itself, so every run ended with nearly every service on one
node. On 2026-10-25 the operator had that spread out by hand.

- Agent 1.26.0 reports each swarm service's pinned digest, and every
  repo@sha256 its host holds (from /system/df, every 10 min).
- Once a run's last host is back in service, Steward picks services on the
  fullest node to move to the emptiest until they are within one task. It
  takes services with several replicas on one node first. A service moves only
  when the emptier node already holds its exact image. One the plan keeps in
  place, one below its replicas, and one with no pinned digest stay put, and
  the run says why.
- A swarm manager's helper carries it out through a new action, rebalance,
  with the service names (validated, at most 48). It force-updates each in
  turn with the spec unchanged and no registry lookup, skips any a
  constraint places, and gives each 5 min to settle.
- The run ends done with what moved, or failed if a move didn't settle or
  the manager didn't answer in 60 min. A docker-scoped window covers the
  restarts, and the step goes on the timeline.
- Plans gain "Spread swarm services back out" (on by default) and "Keep in
  place", on the create form and in an "After the run" form on each plan.
  list_maintenance_plans shows both and the latest run's spread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 10:42:26 -04:00
bvandeusenandClaude Opus 5.5 3ad8eed83d feat: the host agent is always on; the MCP sees and starts host maintenance (#5834)
CI / lint (push) Successful in 2s
CI / integration (push) Successful in 1m32s
CI / unit (push) Successful in 1m39s
CI / publish (push) Successful in 1m14s
#5834 "The host agent is always on, and the MCP sees a host's
maintenance", step 1 of milestone 520 "The MCP sees what Steward has
grown".

The host agent is a core system (operator, 2026-10-15: "it's very core
to the functionality of Steward as a whole and shouldn't be considered
something that can be toggled off"). ALWAYS_ON_PLUGINS in core settings
makes to_plugins_cfg report it enabled whatever its stored row says, so
it always loads and its MCP tools are always listed. Settings -> Plugins
shows it as "Built in: always on" with no switch, and a save keeps it
enabled. Its code and URLs stay under plugins/host_agent: every
installed agent posts to /plugins/host_agent/ingest.

Maintenance through the MCP:
- list_maintenance_plans: each plan's group, weekly slot, reboot policy,
  next slot, and its latest run (hosts, how far it got, who it skipped
  and why, how it ended).
- get_maintenance_job(host | job): a host's jobs, newest and running
  first, or one job: state, packages updated, reboot, drained by, why it
  failed, when its current state times out.
- run_maintenance(host, reboot="if_required" | "never"): admin, as the
  host page's Update now; refuses a busy host or one with no helper;
  audited "<user> via MCP".
- The overview names a host whose latest job in the last week failed or
  timed out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 12:59:42 -04:00