bvandeusen
b735432d02
Answers "how are videos/all media handled by the GPU worker": a job is per ITEM, but the agent fans a VIDEO into per-frame instances (ffmpeg in the agent, the existing cadence), each stored with a timestamp — so a video becomes a BAG of frame embeddings (fixes the mean-embedding muddle) instead of one washed-out vector. Stills → frame_time NULL; animated GIF/WebP treated like short video. - image_region.frame_time (migration 0061, not yet deployed so folded in): the source frame's seconds for video/animated media; NULL for stills. RegionService passes it through. A whole frame is just kind='frame'. - gpu_job + GpuJobService (migration 0062): the durable work list that keeps the desktop agent HTTP-only — enqueue (dedupes (image,task)) / lease (FOR UPDATE SKIP LOCKED, re-claims expired leases so the queue self-heals) / heartbeat / complete / fail (re-queues until MAX_ATTEMPTS then 'error'). The server enqueues; the agent leases+submits over the web API; Redis/Postgres stay private. Tests: enqueue dedupe, lease-then-skip-when-held, expired-lease reclaim, scoped heartbeat, complete, fail-requeue-then-error. region test now covers frame_time. NEXT: the thin HTTP API (lease/submit/heartbeat) + bearer-token auth, then the agent container + control UI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
51 lines
2.1 KiB
Python
51 lines
2.1 KiB
Python
"""GpuJob — a unit of GPU work the desktop agent pulls over HTTP (#114).
|
|
|
|
The durable work list that lets the agent stay HTTP-only: the server enqueues a
|
|
job per (image, task) — e.g. detect figures + CCIP-embed — and the agent LEASES a
|
|
batch, computes on its GPU, then SUBMITS results, all over the already-exposed web
|
|
API. Redis/Postgres stay private. A lease has an expiry; the lease query itself
|
|
re-claims expired leases (agent died / stopped mid-batch), so the queue is
|
|
self-healing without a separate sweep. One job is per ITEM; the agent fans a
|
|
VIDEO out into per-frame instances internally (see image_region.frame_time).
|
|
|
|
State: pending → leased → done | error (a failure under the attempt cap returns to
|
|
pending for another agent).
|
|
"""
|
|
|
|
from datetime import datetime
|
|
|
|
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, func
|
|
from sqlalchemy.orm import Mapped, mapped_column
|
|
|
|
from .base import Base
|
|
|
|
|
|
class GpuJob(Base):
|
|
__tablename__ = "gpu_job"
|
|
|
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
|
image_record_id: Mapped[int] = mapped_column(
|
|
ForeignKey("image_record.id", ondelete="CASCADE"), index=True
|
|
)
|
|
# What to compute, e.g. 'ccip' (detect figures + CCIP-embed) or 'siglip_region'.
|
|
task: Mapped[str] = mapped_column(String(32), nullable=False)
|
|
status: Mapped[str] = mapped_column(
|
|
String(16), nullable=False, default="pending", index=True
|
|
)
|
|
# pending | leased | done | error
|
|
lease_token: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
|
leased_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True
|
|
)
|
|
lease_expires_at: Mapped[datetime | None] = mapped_column(
|
|
DateTime(timezone=True), nullable=True
|
|
)
|
|
attempts: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
|
error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
|
created_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
|
)
|
|
updated_at: Mapped[datetime] = mapped_column(
|
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
|
)
|