The SigLIP embedder + YOLO proposers load lazily then stay resident for the
container's whole lifetime — a 24/7 agent with an empty queue squats on ~5GB of
VRAM doing nothing (operator-observed: 4900MiB held at GPU-util 8% / P8). Sleep
mode only sheds downloaders + poll cadence; even a UI Stop left the models loaded.
Add a monitor thread that unloads the torch-owned models after
cfg.idle_unload_seconds (env IDLE_UNLOAD_SECONDS, default 300; 0 disables) with
the GPU genuinely idle (active==0, buffer drained, no job completed in the
window), then torch.cuda.empty_cache() to hand the blocks back to the driver.
They reload lazily on the next job via the existing _ensure_embedder /
_proposers_for. Covers both sleep-mode idle and a full Stop. Surfaced in
/status (models_loaded) and the agent UI pipe line; the VRAM meter drops too.
Residual: imgutils CCIP/person ONNX sessions + the CUDA context stay resident
(no clean unload API) — idle VRAM drops substantially, not to zero.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TbrA36zNczjVhrM6cWThQa
bvandeusen
57e52433d0