bvandeusen
4a1a9ec5a7
Control UI gains what the operator asked for: - GPU load (nvidia-smi): util %, VRAM used/total + bar, temp — so you can see how hard the card is working while you're at the desktop. - Worker count is now a live − / + control (POST /concurrency), not just an env: the worker is a pool of independent slots (shared model, so slots add concurrent inference, not N× VRAM). Dial up for speed, down to free the card. Replaces pause/resume with Start/Stop + the worker dial. - Graceful release on stop / pool-shrink: a slot hands its still-leased jobs back via client.release() so they're re-picked immediately (pairs with the server recovery sweep). Not CI-tested (agent/ outside CI) — verified by running. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
31 lines
1.0 KiB
Python
31 lines
1.0 KiB
Python
"""GPU load readout via nvidia-smi (present in the container thanks to the
|
|
NVIDIA Container Toolkit's `utility` capability). Returns None if unavailable —
|
|
the UI just shows n/a (e.g. CPU-fallback run)."""
|
|
import subprocess
|
|
|
|
|
|
def read_gpu() -> dict | None:
|
|
try:
|
|
out = subprocess.run(
|
|
[
|
|
"nvidia-smi",
|
|
"--query-gpu=utilization.gpu,memory.used,memory.total,temperature.gpu",
|
|
"--format=csv,noheader,nounits",
|
|
],
|
|
capture_output=True, text=True, timeout=5, check=True,
|
|
).stdout.strip().splitlines()
|
|
except (OSError, subprocess.SubprocessError):
|
|
return None
|
|
if not out:
|
|
return None
|
|
parts = [p.strip() for p in out[0].split(",")]
|
|
try:
|
|
return {
|
|
"util_pct": int(float(parts[0])),
|
|
"mem_used_mb": int(float(parts[1])),
|
|
"mem_total_mb": int(float(parts[2])),
|
|
"temp_c": int(float(parts[3])),
|
|
}
|
|
except (ValueError, IndexError):
|
|
return None
|