bvandeusen
b735432d02
Answers "how are videos/all media handled by the GPU worker": a job is per ITEM, but the agent fans a VIDEO into per-frame instances (ffmpeg in the agent, the existing cadence), each stored with a timestamp — so a video becomes a BAG of frame embeddings (fixes the mean-embedding muddle) instead of one washed-out vector. Stills → frame_time NULL; animated GIF/WebP treated like short video. - image_region.frame_time (migration 0061, not yet deployed so folded in): the source frame's seconds for video/animated media; NULL for stills. RegionService passes it through. A whole frame is just kind='frame'. - gpu_job + GpuJobService (migration 0062): the durable work list that keeps the desktop agent HTTP-only — enqueue (dedupes (image,task)) / lease (FOR UPDATE SKIP LOCKED, re-claims expired leases so the queue self-heals) / heartbeat / complete / fail (re-queues until MAX_ATTEMPTS then 'error'). The server enqueues; the agent leases+submits over the web API; Redis/Postgres stay private. Tests: enqueue dedupe, lease-then-skip-when-held, expired-lease reclaim, scoped heartbeat, complete, fail-requeue-then-error. region test now covers frame_time. NEXT: the thin HTTP API (lease/submit/heartbeat) + bearer-token auth, then the agent container + control UI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
60 lines
2.2 KiB
Python
60 lines
2.2 KiB
Python
"""Region read/write for the crop pipeline (#114).
|
|
|
|
The GPU agent's results endpoint calls replace_regions() to store a freshly
|
|
detected/embedded set; the character matcher + concept-bag scorer read via
|
|
get_regions(). Replacement is scoped BY KIND so the figure pipeline and the
|
|
concept pipeline don't clobber each other.
|
|
"""
|
|
|
|
from typing import Any
|
|
|
|
from sqlalchemy import delete, select
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
|
|
from ...models import ImageRegion
|
|
|
|
|
|
class RegionService:
|
|
def __init__(self, session: AsyncSession):
|
|
self.session = session
|
|
|
|
async def get_regions(
|
|
self, image_id: int, kinds: list[str] | None = None
|
|
) -> list[ImageRegion]:
|
|
stmt = select(ImageRegion).where(ImageRegion.image_record_id == image_id)
|
|
if kinds:
|
|
stmt = stmt.where(ImageRegion.kind.in_(kinds))
|
|
return list(
|
|
(await self.session.execute(stmt.order_by(ImageRegion.id))).scalars()
|
|
)
|
|
|
|
async def replace_regions(
|
|
self, image_id: int, kinds: list[str], regions: list[dict[str, Any]]
|
|
) -> int:
|
|
"""Replace this image's regions OF THE GIVEN KINDS with `regions` (a
|
|
re-detect/re-propose supersedes the prior set without touching other
|
|
kinds). Each region dict: {kind, bbox:(x,y,w,h), score?, detector_version?,
|
|
crop_version?, embedding_version?, ccip_embedding?, siglip_embedding?}.
|
|
Returns the number inserted."""
|
|
await self.session.execute(
|
|
delete(ImageRegion)
|
|
.where(ImageRegion.image_record_id == image_id)
|
|
.where(ImageRegion.kind.in_(kinds))
|
|
)
|
|
n = 0
|
|
for r in regions:
|
|
rx, ry, rw, rh = r["bbox"]
|
|
self.session.add(ImageRegion(
|
|
image_record_id=image_id, kind=r["kind"],
|
|
frame_time=r.get("frame_time"),
|
|
rx=rx, ry=ry, rw=rw, rh=rh,
|
|
score=r.get("score"),
|
|
detector_version=r.get("detector_version"),
|
|
crop_version=r.get("crop_version"),
|
|
embedding_version=r.get("embedding_version"),
|
|
ccip_embedding=r.get("ccip_embedding"),
|
|
siglip_embedding=r.get("siglip_embedding"),
|
|
))
|
|
n += 1
|
|
return n
|