Works through the optional CCIP ideas + the "keep moving even if I forget" ask:
AUTOMATION (no button needed):
- Hourly beat auto-enqueues CCIP backfill — new images get embedded (and errored
ones retried) on their own; the queue never goes idle waiting for a click.
- CCIP auto-apply: a daily sweep tags confident matches (source='ccip_auto') so
identity tags keep flowing. ON by default (opt-out, like head auto-apply);
ml_settings.ccip_auto_apply_enabled + _threshold (0.92, above the suggest cut),
migration 0064. Vectorized (one matmul + reduceat per image), reversible, skips
already-applied/rejected. Switch + threshold in the GPU agent card; GET/PATCH
/api/ml/settings; auto_applied count in /api/ccip/overview.
REFERENCE QUALITY (the over-fire root cause):
- character_references now draws ONLY from single-character images — on a
multi-character image the tag is image-level, so every figure would otherwise
pollute each character's prototypes (a 2-char image tagged 'Velma' made
Daphne's figure a Velma reference). This is the contamination behind residual
over-firing.
- Cached on a cheap signature (char-tag count + ccip-region count/max-id) so the
reference load isn't redone on every modal open.
Tests: multi-character image not used as a reference; auto-apply tags a confident
match as ccip_auto.
NEXT (not done, confirmed): comic-panel cropping + SigLIP concept crops ("spot
interesting content").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Accept already re-focused the tag input (so you keep typing without re-clicking);
reject (✗) and un-reject (↶) went straight to the store and skipped it. Route
them through onDismiss/onUndismiss which emit 'dismissed', and wire that to
focusTagInput in TagPanel — same return-to-input behaviour as accept. TagPanel is
shared, so this covers both the image modal and the Explore workspace. The
field's mobile-focus guard is preserved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Two reports: → sometimes "loops back", and the walk gets stuck on near-identical
images. Cause: forwardTarget picked a uniformly-random neighbour from the 24
NEAREST, so it (a) often landed on an image already in the trail — which snaps
the cursor back into history and makes → bounce between visited nodes — and (b)
only ever offered near-duplicates.
forwardTarget now: excludes already-visited neighbours (→ opens something new,
no snap-back), and skips the closest third of the (similarity-sorted) pool so the
jump favours the more-varied remainder instead of lookalikes. Neighbour pool
widened 24→40 for more variety to browse + jump into. The post-← browser-forward
walk through visited crumbs is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Live data showed the v1 flat 0.75 cosine over-fired — ~64% of matched images got
3-10 character guesses dominated by the most-referenced characters (a 27-ref
character clears a low bar on many images). A sweep showed 0.85 collapses the
noise (noisy multi-matches 47→3) while keeping the confident single-character
matches.
- ml_settings.ccip_match_threshold (migration 0063, default 0.85); match_image
reads it (override still accepted). DEFAULT_SIM_THRESHOLD fallback 0.75→0.85.
- Exposed in GET/PATCH /api/ml/settings (validated 0.5–0.999).
- Slider in the GPU agent card ("Character-match strictness") — tune live, no
redeploy, same observe-and-tune loop as auto-apply.
Test: a ~0.9-cosine figure matches at 0.85, dropped at 0.95.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
At 8 workers the GPU sat at ~5% util / <5GB VRAM — the pipeline is I/O-bound
(downloading + decoding images over HTTP), so the GPU starves until many workers
overlap that I/O. Raise MAX_CONCURRENCY 8→32 and make the UI worker control a
number input (reaching 32 by ±1 was tedious); the cap is reported via /status so
the UI clamps to it. Also size the shared requests pool (pool_maxsize=64) — the
default 10 would have throttled 32 workers + spammed "connection pool is full".
Verified by running; watch GPU util/VRAM climb as you dial up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
A naive convert('RGB') on a palette-with-transparency image (common: character
PNGs on a clear background) lets PIL guess the transparent pixels — black-ish
artifacts that bleed into the crop + the CCIP embedding (and the "should be
converted to RGBA" warning). to_rgb() composites over white first for a clean,
consistent background; used by both stills and video frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Control UI gains what the operator asked for:
- GPU load (nvidia-smi): util %, VRAM used/total + bar, temp — so you can see how
hard the card is working while you're at the desktop.
- Worker count is now a live − / + control (POST /concurrency), not just an env:
the worker is a pool of independent slots (shared model, so slots add concurrent
inference, not N× VRAM). Dial up for speed, down to free the card. Replaces
pause/resume with Start/Stop + the worker dial.
- Graceful release on stop / pool-shrink: a slot hands its still-leased jobs back
via client.release() so they're re-picked immediately (pairs with the server
recovery sweep).
Not CI-tested (agent/ outside CI) — verified by running.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
So work an agent orphaned gets picked back up quickly, three layers:
- GpuJobService.release(): a graceful agent stop hands its still-leased jobs back
to pending instantly (POST /api/gpu/jobs/release), no waiting out the lease.
- GpuJobService.recover_orphaned() + recover_orphaned_gpu_jobs Celery task on a
60s beat: resets expired leases (a hard-crashed agent) to pending and keeps the
queue counts honest even when nothing is leasing.
- Lease TTL 300→180s: still well above any single job (a capped-frame video embed
is tens of seconds, and a live worker heartbeats), but a hard crash recovers
faster once the sweep fires.
Tests: release returns-to-pending (token-scoped), recover_orphaned resets only
expired leases, release API round-trip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
onnxruntime-gpu needs cuDNN 9; the plain cuda:12.4.1-runtime image lacks it
(libcudnn.so.9 missing → CUDAExecutionProvider falls back to CPU). Switch to
the -cudnn-runtime variant which bundles cuDNN 9.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
compose file (pull the published image, GPU reservation, model-cache volume,
.env for the token) so the agent runs with `docker compose up -d` instead of a
long docker run. A copy + .env template also placed in ~/Documents/fc-gpu-agent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Build + push fabledcurator-agent alongside web/ml (own CUDA + onnxruntime-gpu
image, context=agent/, same tag cadence: main → :main/:latest/:c-<sha>, tag →
:<version>). So the operator PULLS + runs it on the GPU machine instead of
building locally. README switched to docker pull.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The last piece: a Dockerised desktop-GPU worker that talks to FC ONLY over HTTP
(lease → fetch pixels → detect figures + CCIP-embed → submit), so Redis/Postgres
stay private. New top-level agent/ (outside CI scope — verified by running it):
- fc_agent/worker.py: the lease/compute/submit loop, concurrency 1, start/pause/
stop (stop frees the card; unprocessed leases expire + re-queue).
- fc_agent/models.py: imgutils wrappers — detect_person (figures) + CCIP embed.
The two API seams to verify against the installed dghs-imgutils (flagged).
- fc_agent/media.py: stills + video frame sampling (ffmpeg) at FC's cadence →
per-frame instances (the bag).
- fc_agent/crops.py: vendored crop primitive. client.py: the FC HTTP client.
- fc_agent/app.py: FastAPI localhost control UI (start/pause/stop + progress +
queue depth). Dockerfile (CUDA + onnxruntime-gpu + ffmpeg) + requirements +
README (token → build → run --gpus all → Start; CPU-fallback path).
This completes the CCIP pipeline end to end: agent produces region CCIP vectors →
RegionService stores → matcher suggests characters → rail. Verified by running on
the desktop (not CI). README calls out the imgutils API + model-string checks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
So the work can be checked through an API as the agent fills in vectors (same
pattern as /api/heads/metrics):
- GET /api/ccip/overview: regions by kind, images with figure CCIP vectors, the
per-character reference counts (which characters have enough examples to match
on), and the embedding versions present.
- GET /api/ccip/images/<id>: that image's stored regions (bbox, frame_time,
has_ccip/has_siglip, versions) + the CCIP character matches it would get — for
spot-checking detector + matcher output.
Read-only, no GPU. (Queue depth is already at /api/gpu/status.)
Tests: overview coverage counts + per-character refs; per-image regions + matches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
SuggestionService.for_image now merges CCIP character matches with the SigLIP
head suggestions — they're complementary, not exclusive: CCIP is the identity-
specialized signal but needs a detected figure; the heads work whole-image but
conflate identity with style. Merged by tag: 'both' when they corroborate
(higher score wins), 'ccip' / 'head' otherwise. Cheap when no CCIP vectors exist
yet (match_image returns early without a figure vector), so it's a no-op until
the agent runs. Suggestion.source is now 'head' | 'ccip' | 'both'.
Test: a character with a CCIP reference figure surfaces (source='ccip') on a new
image whose figure matches.
NEXT: the agent container (real CCIP/detector models, hands-on) that produces the
vectors this consumes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The server-side brain that turns stored CCIP vectors into character suggestions
— no GPU. character_references() gathers each character tag's prototype vectors
(figure/face-region CCIP embeddings on images carrying that tag); match_image()
cosine-matches an image's figure vectors against every character (multi-
prototype: best over a character's examples), surfacing those above a tunable
threshold as {tag_id, name, category:'character', score, source:'ccip'},
excluding already-applied characters. v1 = cosine on raw CCIP vectors; the exact
CCIP metric/threshold gets validated against the model in the hands-on eval.
Tests (synthetic vectors): same-character match across images, no-match for an
orthogonal figure, already-applied exclusion, no-figure-vectors empty.
NEXT: merge CCIP character suggestions into the rail; the agent container that
actually produces the vectors (hands-on, GPU — not CI-verifiable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The FC-side control surface the operator asked for: Settings → Tagging → "GPU
agent". Generate/reveal/copy/rotate the agent bearer token (with the FC URL to
point the agent at), see the live job-queue depth (pending/in-flight/done/
errored, polled), and a "Queue character embedding (CCIP)" button that triggers
the library backfill. Plain-HTTP-safe copy (copyText resolves on success,
throws on fail). Closes the "how do I get the token in the UI" gap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The thin HTTP surface over the queue so the desktop agent stays HTTP-only:
- Agent endpoints (Authorization: Bearer <token>): POST /api/gpu/jobs/lease
(returns jobs + image_url + mime + video frame cadence), /submit (stores
regions via RegionService + closes the job; 409 on a stale lease), /heartbeat,
/fail. Token validated against AppSetting (mirrors the extension-key pattern,
constant-time compare).
- Admin (browser): GET/POST /api/gpu/token[/rotate] (generate + show the agent
token), GET /api/gpu/status (queue counts), POST /api/gpu/backfill → dispatches
enqueue_gpu_backfill.
- enqueue_gpu_backfill(task): one INSERT…SELECT enqueues a job per image lacking
one for the task (scales to the full library; idempotent).
Agent flow: lease over HTTP → fetch pixels via the normal FC image URL → compute
on the GPU → submit. Redis/Postgres never exposed.
Tests: bearer required (+ wrong-token 401), lease→submit round-trip (region+CCIP
vector stored, job done via /status), stale-lease 409, backfill enqueue +
idempotency.
NEXT: the agent container + control UI, then the CCIP detector/embedder + matcher.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Answers "how are videos/all media handled by the GPU worker": a job is per ITEM,
but the agent fans a VIDEO into per-frame instances (ffmpeg in the agent, the
existing cadence), each stored with a timestamp — so a video becomes a BAG of
frame embeddings (fixes the mean-embedding muddle) instead of one washed-out
vector. Stills → frame_time NULL; animated GIF/WebP treated like short video.
- image_region.frame_time (migration 0061, not yet deployed so folded in): the
source frame's seconds for video/animated media; NULL for stills. RegionService
passes it through. A whole frame is just kind='frame'.
- gpu_job + GpuJobService (migration 0062): the durable work list that keeps the
desktop agent HTTP-only — enqueue (dedupes (image,task)) / lease (FOR UPDATE
SKIP LOCKED, re-claims expired leases so the queue self-heals) / heartbeat /
complete / fail (re-queues until MAX_ATTEMPTS then 'error'). The server enqueues;
the agent leases+submits over the web API; Redis/Postgres stay private.
Tests: enqueue dedupe, lease-then-skip-when-held, expired-lease reclaim, scoped
heartbeat, complete, fail-requeue-then-error. region test now covers frame_time.
NEXT: the thin HTTP API (lease/submit/heartbeat) + bearer-token auth, then the
agent container + control UI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The storage backbone both crop jobs write to and read from. image_region =
normalized bbox (rx/ry/rw/rh) + kind ('face'/'figure' → CCIP character id;
'concept' → SigLIP head bag) + the crop's embedding (nullable Vector(768) CCIP /
Vector(1152) SigLIP, one per kind) + version stamps for compute-once gating. The
bbox doubles as grounded-tag provenance. Migration 0061.
RegionService.replace_regions (scoped BY KIND so the figure + concept pipelines
don't clobber each other) + get_regions — the GPU agent's results endpoint will
call the writer; the character matcher + bag scorer read. Server-side, no GPU.
Tests: replace/get round-trip, kind-scoped replacement, CCIP vector round-trip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The trunk of both crop jobs — CCIP figure-crops and SigLIP concept-crops call
the SAME crop_region(): normalized-bbox crop with optional context padding,
edge-clamping, and the lower-bound size floor (max of a fraction-of-short-side
and an absolute pixel floor) below which a region is too small to embed and
returns None. Only the proposer (where) and embedder (what) differ; the crop is
shared. Pure Pillow — importable + testable anywhere (the GPU agent imports it
for the crop step). Unit-lane tests (no DB): region pixels, floor rejection,
edge clamp, pad expansion, out-size resize.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The Explore center pane hardcoded ImageCanvas, so a video anchor (e.g. a 169 MB
MP4) tried to load the MP4 into an <img> and showed only the alt text — the
thumbnail worked but the "main image" never rendered. Branch on
mime.startsWith('video/') to VideoCanvas (with mime), exactly like the image
modal. The anchor payload already carries mime.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The autocomplete suppressed the Create row whenever any existing tag matched
name+kind — but characters are unique by (name, kind, fandom), so a same-named
character in a different fandom (e.g. another "Raven") is a valid distinct tag.
allowCreate now always offers Create for the character kind; the fandom picker
disambiguates and find_or_create is idempotent if the same fandom is re-picked.
The Create row reads "Create another \"Raven\" character (different fandom)" when
a same-name character already exists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Surfaces earned auto-apply + its observability in Settings → Tagging → Concept
heads:
- Auto-apply section: an on/off switch (writes head_auto_apply_enabled), the
precision-target + min-examples-to-fire tuning inputs, a Preview (dry-run →
"would apply N", per-concept chips) and Apply-now button, with live run state.
- "How auto-apply is landing": per-concept table from /api/heads/metrics —
applied volume, misfires, realized misfire rate (green/amber/red), and missed
(under-fires) — the signal to tune the precision target from.
store: autoApply(dryRun) / autoApplyStatus() / metrics(). Card polls the sweep
to completion, then refreshes counts + metrics. Completes the auto-apply task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
test_auto_apply_disabled_blocks_real_run assumed head_auto_apply_enabled
defaulted False; it now defaults True (opt-out), so a real sweep is accepted
(202). Set the switch off in the test to exercise the disabled→400 path.
(run 1629)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
dict(session.execute(...)) on a bare Result invokes the mapping protocol (a
Result has .keys() = column names) and subscripts it → "CursorResult is not
subscriptable". Materialize with .all() so dict() consumes rows as key-value
pairs. The API path already did this; the snapshot task missed it. Caught by
test_snapshot_records_timeseries_point (run 1628).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Auto-apply is now ON by default (operator-asked: opt-OUT, not opt-in) — migration
0059 + model default flipped. The support (>=30) + measured-precision gates keep
it safe and every auto-tag is reversible.
Observability so the operator can tune from real data:
- MISFIRE = an auto-applied (source='head_auto') tag the operator later removes.
UNDER-FIRE = a tag with a head the operator adds by hand (the head missed it).
Both captured at correction time in TagService.add_to_image/remove_from_image
(source is lost on delete) into durable per-tag counters (head_metric), keyed
by tag so they survive head retrain/prune.
- Daily snapshot_head_metrics writes a per-concept time-series point
(head_metrics_snapshot): auto-applied volume + cumulative misfires/under-fires
+ head quality; 180-day retention; daily beat.
- GET /api/heads/metrics: per-concept current counts + realized misfire rate +
head quality, plus the snapshot time-series — the report to tune the precision
target + support floor.
Migration 0060. Tests: misfire/under-fire counting (and the negatives — manual
removal isn't a misfire, headless manual add isn't an under-fire), snapshot
time-series, metrics API.
What's the autofire threshold? There's no single number — each graduated head
derives its OWN probability cutoff from its PR curve: the operating point that
holds precision >= head_auto_apply_precision (0.97) at max recall. The global
knobs are that target + the >=30 support floor.
NEXT (slice 3): UI — enable toggle, dry-run preview, per-concept trends.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Graduated heads can now apply their tag without a human — gated so it's safe:
- FIRING GATE: a head fires only when the master switch (head_auto_apply_enabled,
default OFF) is on AND it has >= head_auto_apply_min_positives (default 30)
clean labels. A precise-looking but under-supported low-N head can't spray tags.
- auto_apply_sweep (heads.py): streams every embedded image in chunks, scores
against the eligible heads (numpy, no sklearn), applies each head's tag where
score >= its auto_apply_threshold and the tag isn't already applied/rejected,
with source='head_auto' (distinguishable + reversible). dry_run counts only.
- HeadAutoApplyRun (migration 0059) tracks each sweep / preview; apply_head_tags
task (ml queue) + scheduled_apply_head_tags daily beat (no-op unless enabled)
+ recovery sweep + retention(20).
- API: POST /api/heads/auto-apply {dry_run} (202 / 409 running / 400 disabled),
GET /api/heads/auto-apply (recent runs + per-concept report). Settings
head_auto_apply_enabled + min_positives via /api/ml/settings.
Tests: sweep applies above threshold, dry-run writes nothing, skips under-
supported + ungraduated heads; API disabled/dry-run/conflict guards.
NEXT (slice 2): the observability the operator asked for — per-concept misfire
(auto-applied-then-removed) + under-fire tracking, time-series snapshots, and a
reporting API to tune. Slice 3: the UI (enable, preview, trends).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Two cadences for keeping heads in sync with your tagging:
- PASSIVE: a nightly `scheduled_train_heads` beat (skips if a run is already
in flight; creates+commits the run row before dispatching train_heads so the
ml worker always finds it). Folds the day's accepts/rejects + newly-eligible
concepts into the heads without anyone clicking.
- ACTIVE: a "Retrain heads" button in the Explore trail bar — bank the +/-
feedback you just gave while walking content, without a trip to Settings.
Shared logic in a new useHeadTraining composable (trigger + poll + start/finish
toasts), used by the Explore button; reflects an already-running run (incl. the
nightly one) on mount.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Arrow keys walk the Explore breadcrumb trail: ← steps back, → goes forward to
an already-visited item or — with no forward history — jumps to a RANDOM
neighbour to keep the rabbit-hole going (operator-asked).
The trail gains a cursor (browser back/forward semantics): stepping back no
longer trims the forward branch, so → can return to it; a genuinely new walk
off a back-step truncates the stale branch then appends. The crumb-bar "current"
highlight follows the cursor, not the tip.
Arrows are ignored while typing a tag, but still navigate when the tag input is
focused-but-empty (it auto-focuses after every walk, so otherwise arrow-nav
would dead-end after one step). Modifier-key combos pass through untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The rail's Suggestions now come from the trained per-concept heads. SuggestionService.for_image scores the image's frozen SigLIP embedding against
every head (heads.score_image) and surfaces concepts above each head's own
suggest threshold; the typed-dropdown's min=0 "show everything" mode maps to a
flat floor so any head-scored concept can still be picked. Already-applied tags
drop; rejected tags stay flagged + reversible (unchanged).
REMOVED from the suggestion path (rule 22, no fallback): the Camie
ImagePrediction candidate/alias/merge pipeline and the per-tag centroid
augmentation, plus the now-dead SuggestionService internals (_load_predictions,
_threshold_for, _settings, self.aliases, self.centroids). Head suggestions are
always canonical tags, so raw_name/via_alias are null/false and the rail's
alias kebab is inert by data (its removal + the Camie ingest-tagger rip are the
flagged follow-up). for_selection (bulk consensus) now aggregates head
suggestions unchanged.
Tests rewritten to the head path: test_ml_suggestions (surfaces/applied/
rejected-reversible/override/no-embedding/no-heads), test_suggestions_bulk
(consensus), test_api_suggestions (get + dropped the Camie-alias roundtrip),
and test_ml_artist_retired (artist not head-eligible via _HEAD_KINDS).
DEPLOY NOTE: after this lands, the rail is empty until you run Train heads
(Settings → Tagging → Concept heads) — deploy, train, then the rail populates.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The UI for the heads subsystem: Settings → Tagging → "Concept heads". Shows
head count, auto-apply-ready count, and last-trained; a Train/Retrain button
(one run at a time, polls while running, surfaces a failed run's error); an
empty state guiding the operator to tag first; and a per-concept table (name,
category, +tags, AP, P, R, auto-apply ⚡) sorted strongest-first so weak/under-
tagged concepts are obvious. Rehydrates status from GET /api/heads on mount so
it survives navigation. Pulls head_min_positives from ML settings for copy.
Slice C (swap the rail's suggestions to heads, remove Camie + centroid) is next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
Alphabetize HeadTrainingRun in models/__init__ + maintenance imports (H before
I), and drop the inline comment that split heads.py's import block. Pure import
ordering — no behavior change. (run 1601 lint)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
test_rejected_tag_surfaced_flagged_then_reversible asserted "Rejectme" but an
existing tag keeps its stored name ("rejectme"), so the suggestion's
display_name is lowercase. Match by canonical_tag_id instead (casing-robust).
The feature was correct — only the assertion was wrong (run 1595 integration).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
The eval (#1130) proved the frozen-embedding + trained-head spine; this lands
its production form (the first of three slices that make heads the suggestion
source, replacing Camie + centroid).
- tag_head: one logistic-regression head per general/character concept with
enough labelled positives. Weights (pgvector), honest CV-derived suggest
threshold + earned-auto-apply point, and per-concept quality metrics.
- head_training_run: persisted batch lifecycle (mirrors tag_eval_run) so the
admin card shows live + historical status across navigation.
- services/ml/heads.py: TRAIN (sync, ml worker, reuses tag_eval's proven data
loaders + metric math so production heads match measured eval numbers) and
SCORE (async, API worker — numpy via pgvector, no scikit-learn): score one
image's embedding against all heads → the rail's suggestions, cached on
(count, max trained_at) so a retrain invalidates without per-request loads.
- tasks.ml.train_heads (ml queue, commits per head so a kill leaves progress)
+ recover_stalled_head_training_runs sweep + retention(20) + 5-min beat
(rule 89).
- api/heads.py: POST /api/heads/train (one run at a time, 409 guard) + GET
/api/heads (count, graduated, last-trained, running, per-concept table,
recent runs).
- ml_settings: head_min_positives + head_auto_apply_precision, tunable via
/api/ml/settings.
Scoring isn't wired into the rail yet (slice C) and the admin UI is slice B —
this slice makes training + scoring exist and CI-verifiable. 'precision' column
stored as precision_cv (SQL reserved word). Migration 0058.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
bvandeusen