feat: a GPU agent on the CPU shows as degraded, in the System view and on its own page (4410)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 27s
CI and images / backend-lint-and-test (push) Successful in 34s
CI and images / integration (push) Successful in 2m22s
CI and images / sign-extension (push) Successful in 4s
CI and images / build-web (push) Successful in 2m46s
CI and images / smoke-web (push) Successful in 50s
CI and images / build-agent (push) Successful in 6m35s
CI and images / promote (push) Successful in 2s
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 27s
CI and images / backend-lint-and-test (push) Successful in 34s
CI and images / integration (push) Successful in 2m22s
CI and images / sign-extension (push) Successful in 4s
CI and images / build-web (push) Successful in 2m46s
CI and images / smoke-web (push) Successful in 50s
CI and images / build-agent (push) Successful in 6m35s
CI and images / promote (push) Successful in 2s
torch and onnxruntime both fall back to the CPU without raising, so the agent that ran CPU-bound for weeks after a driver update leased and checked in like a healthy one. - The agent sends its startup accel report on every lease and heartbeat. - The server keeps a bounded copy on the roster row. A running agent with a runtime off the GPU becomes `degraded`, with a sentence naming the runtime and the reason. - The top nav shows it amber. - The agent page carries a banner, and its pill reads "CPU only". Also: the bandwidth field gets the page's − / + stepper, and both number fields drop the browser's spin arrows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
+31
-2
@@ -245,6 +245,29 @@ async def errors_recover(image_id: int):
|
||||
|
||||
# --- Agent (bearer token): lease / submit / heartbeat / fail ------------
|
||||
|
||||
|
||||
def _accel_detail(body: dict) -> dict:
|
||||
"""The agent's own report of which runtime got the GPU, kept on its roster
|
||||
row so the System view can call a CPU-bound agent degraded (#4410).
|
||||
|
||||
Only a dict of {runtime: {device, error?}} is kept, and each value is
|
||||
reduced to those two short strings: this is written on every lease, by a
|
||||
client the server does not control. An agent that sends nothing (an
|
||||
older build) simply has no `accel`, which reads as not-yet-reported.
|
||||
"""
|
||||
raw = body.get("accel")
|
||||
if not isinstance(raw, dict):
|
||||
return {}
|
||||
accel = {}
|
||||
for name, entry in list(raw.items())[:4]:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
clean = {"device": str(entry.get("device") or "")[:16]}
|
||||
if entry.get("error"):
|
||||
clean["error"] = str(entry["error"])[:200]
|
||||
accel[str(name)[:16]] = clean
|
||||
return {"accel": accel} if accel else {}
|
||||
|
||||
@gpu_bp.route("/jobs/lease", methods=["POST"])
|
||||
async def lease():
|
||||
body = await request.get_json(silent=True) or {}
|
||||
@@ -267,7 +290,10 @@ async def lease():
|
||||
key=f"agent:{agent_id}",
|
||||
kind="agent",
|
||||
display_name="GPU agent" if agent_id == "agent" else f"GPU agent ({agent_id})",
|
||||
details={"agent_id": agent_id, "last_call": "lease", "leased": len(jobs)},
|
||||
details={
|
||||
"agent_id": agent_id, "last_call": "lease", "leased": len(jobs),
|
||||
**_accel_detail(body),
|
||||
},
|
||||
)
|
||||
ml = await MLSettings.load(session)
|
||||
# image rows for url/mime in one shot
|
||||
@@ -347,7 +373,10 @@ async def heartbeat():
|
||||
key=f"agent:{agent_id}",
|
||||
kind="agent",
|
||||
display_name="GPU agent" if agent_id == "agent" else f"GPU agent ({agent_id})",
|
||||
details={"agent_id": agent_id, "last_call": "heartbeat", "extended": n},
|
||||
details={
|
||||
"agent_id": agent_id, "last_call": "heartbeat", "extended": n,
|
||||
**_accel_detail(body),
|
||||
},
|
||||
)
|
||||
await session.commit()
|
||||
return jsonify({"extended": n})
|
||||
|
||||
Reference in New Issue
Block a user