feat(agent): idle-unload GPU models to free VRAM when the queue is idle
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 4m3s

The SigLIP embedder + YOLO proposers load lazily then stay resident for the
container's whole lifetime — a 24/7 agent with an empty queue squats on ~5GB of
VRAM doing nothing (operator-observed: 4900MiB held at GPU-util 8% / P8). Sleep
mode only sheds downloaders + poll cadence; even a UI Stop left the models loaded.

Add a monitor thread that unloads the torch-owned models after
cfg.idle_unload_seconds (env IDLE_UNLOAD_SECONDS, default 300; 0 disables) with
the GPU genuinely idle (active==0, buffer drained, no job completed in the
window), then torch.cuda.empty_cache() to hand the blocks back to the driver.
They reload lazily on the next job via the existing _ensure_embedder /
_proposers_for. Covers both sleep-mode idle and a full Stop. Surfaced in
/status (models_loaded) and the agent UI pipe line; the VRAM meter drops too.

Residual: imgutils CCIP/person ONNX sessions + the CUDA context stay resident
(no clean unload API) — idle VRAM drops substantially, not to zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TbrA36zNczjVhrM6cWThQa
This commit is contained in:
2026-07-17 12:57:31 -04:00
parent ec66ea5f83
commit 57e52433d0
5 changed files with 119 additions and 1 deletions
+15
View File
@@ -170,6 +170,13 @@ class YoloProposer:
))
return out
def unload(self) -> None:
"""Drop the loaded YOLO so its VRAM can be reclaimed; detect() reloads it
lazily on the next job. Leaves _ok untouched — a healthy proposer comes
back, but one that self-disabled on a fault stays off."""
with self._lock:
self._model = None
class Proposers:
"""The agent's proposer set, built from config. Each detector is optional —
@@ -216,3 +223,11 @@ class Proposers:
def panels(self, image):
return self._top(self._panel, image, self.cfg.max_panels)
def unload(self) -> None:
"""Release every loaded proposer's YOLO (idle VRAM reclaim). The worker
also drops its reference to this Proposers and rebuilds a fresh one via
_proposers_for on the next job, so this is belt-and-braces."""
for p in (self._person, self._anatomy, self._panel):
if p is not None:
p.unload()