feat(gpu): HTTP job API + token auth + backfill — the agent's server side (#114 slice 3b)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Failing after 3m31s

The thin HTTP surface over the queue so the desktop agent stays HTTP-only:
- Agent endpoints (Authorization: Bearer <token>): POST /api/gpu/jobs/lease
  (returns jobs + image_url + mime + video frame cadence), /submit (stores
  regions via RegionService + closes the job; 409 on a stale lease), /heartbeat,
  /fail. Token validated against AppSetting (mirrors the extension-key pattern,
  constant-time compare).
- Admin (browser): GET/POST /api/gpu/token[/rotate] (generate + show the agent
  token), GET /api/gpu/status (queue counts), POST /api/gpu/backfill → dispatches
  enqueue_gpu_backfill.
- enqueue_gpu_backfill(task): one INSERT…SELECT enqueues a job per image lacking
  one for the task (scales to the full library; idempotent).

Agent flow: lease over HTTP → fetch pixels via the normal FC image URL → compute
on the GPU → submit. Redis/Postgres never exposed.

Tests: bearer required (+ wrong-token 401), lease→submit round-trip (region+CCIP
vector stored, job done via /status), stale-lease 409, backfill enqueue +
idempotency.

NEXT: the agent container + control UI, then the CCIP detector/embedder + matcher.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
2026-06-29 11:33:05 -04:00
parent b735432d02
commit 6cabef07a4
4 changed files with 326 additions and 0 deletions
+28
View File
@@ -738,3 +738,31 @@ def scheduled_apply_head_tags() -> str:
run_id = run.id
apply_head_tags.delay(run_id)
return "dispatched"
@celery.task(name="backend.app.tasks.ml.enqueue_gpu_backfill")
def enqueue_gpu_backfill(task_name: str) -> int:
"""Enqueue a gpu_job for every image that doesn't already have one for
`task_name` (one INSERT…SELECT, so it scales to a full library). The desktop
agent drains the queue over HTTP. Returns the number enqueued."""
from sqlalchemy import exists, insert, literal, select as sa_select
from ..models import GpuJob, ImageRecord
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
already = exists().where(
GpuJob.image_record_id == ImageRecord.id,
GpuJob.task == task_name,
GpuJob.status.in_(["pending", "leased", "done"]),
)
sel = sa_select(
ImageRecord.id, literal(task_name), literal("pending")
).where(~already)
result = session.execute(
insert(GpuJob).from_select(
["image_record_id", "task", "status"], sel
)
)
session.commit()
return result.rowcount or 0