Two operator-reported issues with the GPU agent:
1. Worker count flopped almost every cycle, spiking the GPU. The hill-climb
probed +1, judged it over a too-short noisy throughput window, saw no clear
gain and reverted -1 — every tick. Replace it with a GPU-utilization-band
controller: HOLD while smoothed util sits in a healthy band, grow only on
clear spare capacity (util below the low mark + VRAM headroom), shrink under
saturation or memory pressure. Util is EWMA-smoothed and decisions are spaced
(DECIDE_EVERY samples), so a noisy nvidia-smi reading can't move the pool.
Load stays consistent instead of probe/reverting.
2. GPU util/VRAM bars only updated on manual refresh. They rode the /status
poll, which blocks on the curator queue call (slow when curator is busy), so
the meters froze between refreshes. Give them a dedicated /gpu endpoint
(local nvidia-smi only, no curator round-trip) polled every 1.5s, and drop
the curator queue-status timeout 15s -> 5s so /status itself stays snappy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
bvandeusen
3b34230fbd