bvandeusen
19b962f1a7
The ml-worker's ONLY processing role is now the CPU whole-image embed fallback (tag_and_embed renamed embed_image — Camie tagging was retired #1189 and the name kept implying otherwise; videos were already handled agent-style: frame sampling + mean-pool). Detection/cropping/CCIP stay GPU-agent-only, and their completion is judged per-pipeline: ccip by gpu_job rows, siglip by concept regions at the current model version — never by image_record.siglip_embedding. A CPU embed therefore can NEVER close crop work for the agent (regression test pins this; only the whole-image 'embed' job, the same artifact, is satisfied). Making removal actually safe (operator will drop the container): - GPU-queue coordination (enqueue_gpu_backfill, recover_orphaned_gpu_jobs, reprocess_gpu_jobs) moved verbatim to tasks/gpu_queue.py on the maintenance quick lane — it lived on the 'ml' queue only by module colocation, which made the ml-worker a hard dependency of the whole agent pipeline. - New ml_settings.cpu_embed_enabled (migration 0074, default ON so agent-less installs keep working): OFF stops the four import hooks queueing embed work nothing will consume and no-ops the manual backfill; switch lives on the renamed 'CPU embedding backfill' card. - NB heads training / auto-apply still run on the ml image (sklearn) — a stack that removes the container gives those up too. Deploy note: in-flight messages under the old task names are dropped by the new workers; the 60s orphan sweep + hourly backfill re-fire under the new names immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
36 lines
1.0 KiB
Python
36 lines
1.0 KiB
Python
"""ml_settings.cpu_embed_enabled — the CPU embed fallback becomes a switch
|
|
|
|
B3 (operator 2026-07-02): the ml-worker's only processing role is the CPU
|
|
whole-image embed for stacks without a GPU agent. ON by default (a fresh
|
|
install works agent-less); agent-equipped stacks that drop the ml-worker
|
|
container turn it off so import hooks stop queueing embed work into a queue
|
|
nothing consumes — the daily GPU 'embed' backfill covers those images.
|
|
|
|
Revision ID: 0074
|
|
Revises: 0073
|
|
Create Date: 2026-07-02
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision: str = "0074"
|
|
down_revision: Union[str, None] = "0073"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column(
|
|
"ml_settings",
|
|
sa.Column(
|
|
"cpu_embed_enabled", sa.Boolean(), nullable=False,
|
|
server_default=sa.true(),
|
|
),
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("ml_settings", "cpu_embed_enabled")
|