7939dba9ed88d9d35b4a8fdd7fd76bd033256ca8
98 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 bvandeusen
|
ab362bc79c |
feat(ml): Settings → Tagging 'Crop proposers' card (#134 step 3)
Exposes the detector config (per-proposer enable + weights + confidence, caps, dedupe IoU) in Settings → Tagging, backed by MLSettings via /api/ml/settings. ml_admin adds the detector fields to _EDITABLE + GET payload + validation (conf 0..1, caps >=1, IoU 0..1). New CropProposersCard.vue (mirrors HeadsCard) with working defaults pre-filled, per-field live-save (no restart — the agent picks changes up on its next lease), weights-format help, switch-revert on error. Closes milestone #134: all three proposers are on out-of-the-box and tunable in the UI. Test: detector defaults GET + patch round-trip + range validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
e039689eff |
feat(ia): wave 2 — Activity becomes the whole-app pulse; Overview gets the health strip
The Activity tab only knew Celery — the GPU agent (the majority of processing) and the download pipeline were invisible there. Two new self-polling panels: - GpuActivityPanel: queue depths + triage verdicts (defects / file-ok / unprobed, top reason buckets) with a jump to Maintenance -> Failed processing. The triage detail refetches only when the error count moves. - DownloadsActivityPanel: 24h stat chips + failing-source names with a jump into Subscriptions. Both panels join the Activity tab under Queues+workers AND double as the Overview health strip (side-by-side grid under the Celery summary) — one component set, so Overview answers 'is everything healthy?' across all systems. SystemStatsCards reviewed: content still accurate, left as-is. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
5b34c9221c |
feat(ia): wave 1 — Import tab dissolves, Maintenance regroups by system, one extension home
Settings IA per the approved A3 design (the old layout was the two-app merge fossilized): - Import tab retired: ImportTriggerPanel + ImportTaskList deleted (manual /import scans stay API-level; imports arrive via downloads/extension, heal via the Layer-2 auto-refetch sweep, and show in Activity). ImportFiltersForm moves to Maintenance → 'Ingestion & filters' and loads its own settings; the import store shrinks to settings-only (no remaining consumers of the scan/task-list machinery). Overview's pending banner now points at Activity. - Maintenance regrouped: Ingestion & filters / GPU agent & embeddings (GpuAgent, Failed processing, CPU embedding backfill) / Tagging (sliders, Heads, Aliases) / Library health (MissingFiles, Thumbnails, DB, Archive re-extract demoted last) / Storage. - One extension home: BrowserExtensionCard moves from Settings → Overview to Subscriptions → Settings, above the API key bar it authenticates. - Single-color import filter WIRED: skip_single_color/threshold existed since FC-2 but nothing read them (the audit module's docstring said as much) — now enforced on both import paths via the audit's canonical predicate (tolerance 30, matching the Cleanup card default; animated images exempt like the transparency check). Default stays off; test added. - Dead weight: PlaceholderView (zero refs) and the permanently-disabled 'Export failed logs (CSV — v2)' menu stub deleted; stale docs fixed (celery queue docstring, threshold comment citing retired tasks, ml package docstring, HeadsCard 'replaces Camie' blurb). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
19b962f1a7 |
feat(b3): ml-worker becomes optional — embed-only role, decoupled GPU coordination, cpu-embed switch
The ml-worker's ONLY processing role is now the CPU whole-image embed fallback (tag_and_embed renamed embed_image — Camie tagging was retired #1189 and the name kept implying otherwise; videos were already handled agent-style: frame sampling + mean-pool). Detection/cropping/CCIP stay GPU-agent-only, and their completion is judged per-pipeline: ccip by gpu_job rows, siglip by concept regions at the current model version — never by image_record.siglip_embedding. A CPU embed therefore can NEVER close crop work for the agent (regression test pins this; only the whole-image 'embed' job, the same artifact, is satisfied). Making removal actually safe (operator will drop the container): - GPU-queue coordination (enqueue_gpu_backfill, recover_orphaned_gpu_jobs, reprocess_gpu_jobs) moved verbatim to tasks/gpu_queue.py on the maintenance quick lane — it lived on the 'ml' queue only by module colocation, which made the ml-worker a hard dependency of the whole agent pipeline. - New ml_settings.cpu_embed_enabled (migration 0074, default ON so agent-less installs keep working): OFF stops the four import hooks queueing embed work nothing will consume and no-ops the manual backfill; switch lives on the renamed 'CPU embedding backfill' card. - NB heads training / auto-apply still run on the ml image (sklearn) — a stack that removes the container gives those up too. Deploy note: in-flight messages under the old task names are dropped by the new workers; the 60s orphan sweep + hourly backfill re-fire under the new names immediately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
7c19ad91ed |
feat: cap-aware autoscaler + token-gated whole-instance tag reset (operator feedback)
Autoscaler (agent 2026-07-02.5): the buffer-occupancy signal alone would peg downloaders at DL_MAX while the bandwidth CAP — not concurrency — is the real constraint (8 streams sharing 8 MB/s move no more data than 4). Growth is now gated on the pipe having headroom (net < 85% of cap) and a pipe pinned at the cap (>= 95%) sheds streams down to 3; dead band prevents flapping. The UI hint says 'holding at the bandwidth cap' and /status reports bw_capped, so the behavior is legible without tests that need the ML stack. Reset content tagging: stays a FULL-instance reset (operator's call), but now lives in a fenced 'Danger zone' section on Cleanup and the apply is gated by a preview-derived confirm token (mirrors the Tier-C bulk-delete pattern — stale counts are rejected server-side). Copy no longer claims suggestions repopulate: it says plainly the heads' training examples are deleted and re-tagging starts fresh. Moved out of TagMaintenanceCard into DangerZoneCard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
eaea4308fc |
chore: retire the tag-eval harness — it proved the heads system, job done (operator-approved)
The head-vs-centroid eval (#1130) existed to prove the 'frozen embedding + trained head' spine; the operator accepted the tagging system and dropped the harness. Removed per rule 22: TagEvalCard + store, /api/tag_eval blueprint, tag_eval_run ml task, recover-stalled-tag-eval-runs sweep + beat entry, TagEvalRun model + table (migration 0073), and its tests. The eval's data loaders + metric helpers were NOT eval-specific — the nightly heads trainer runs on them — so they moved verbatim to services/ml/training_data.py (heads.py import updated; behavior unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
a7abcc41ca |
feat(triage): failed-processing triage — probe errored files, flag defects, recover (#125 C1-C3)
An errored GPU job's stored reason is a suspicion; the file probe is the
verdict. A 15-min beat sweep (triage_gpu_errors) runs verify_integrity's own
probe (sha256 + decode) on each errored image ONCE and writes both verdicts:
ImageRecord.integrity_status and the new GpuJob.triage_status ('defect' |
'file_ok', migration 0072). Every classification logs at WARNING so it
surfaces in Logs/System Activity.
- 'defect' rows are excluded from /retry_errors (re-running a known-bad file
burns agent time re-minting the tombstone); response now reports
defects_kept and the GpuAgentCard toast says so.
- GET /api/gpu/errors: triage view — reason buckets (classify_reason),
probe verdicts, per-job detail. POST /errors/triage runs the sweep now.
- POST /api/gpu/errors/<id>/recover: reuses the Layer-2 refetch pattern —
delete the defective copy + record (full cascade takes the tombstones too)
and re-poll its subscription Source so a fresh copy re-imports and re-enters
the pipeline; 'no_source' when nothing pollable resolves.
- New 'Failed processing' card (GpuTriageCard) in Maintenance: verdict counts,
reason summary, probe-now, defect list with thumbnails + per-image Recover.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
|
||
|
bvandeusen
|
1f27189b8f |
chore: retire ml-backfill-daily beat + the spent purge-legacy action (operator-approved)
- ml-backfill-daily: the CPU tag_and_embed backfill raced the GPU agent's daily embed backfill for the same NULL-embedding images at ~100x the cost (B1 audit verdict, milestone #124). The backfill TASK stays — the manual /api/ml/backfill button remains the deliberate CPU fallback pending B3. - purge-legacy: one-time IR-migration cleanup, dry-run verified 0 targets on the live library before removal (A2 audit, milestone #123). Fully retired per rule 22: tile, store action, route, service fn, tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
09e2772628 |
fix(gpu-jobs): end the error-tombstone loop — deliberate retry semantics + poison-job guards
The hourly ccip backfill's skip-list lacked 'error' (and the daily
siglip/embed variants re-gated failures on their missing results), so every
permanently-bad file got a fresh doomed job each run — ~24 duplicate error
rows/day per file, the perpetual 'unprocessable' flood. An errored job is now
a TOMBSTONE: no backfill re-enqueues it; retry is deliberate-only via
/retry_errors (an errored back-catalogue needs one button press after a
model swap).
One shared set of dedupe DELETEs (services/ml/gpu_jobs.error_dedupe_statements)
runs before every backfill and inside /retry_errors: error rows made moot by a
later pending/leased/done row go first, then older duplicates (newest reason
survives) — so the error count reads as distinct failing files and a retry
can't fan one file out into duplicate pending jobs. /retry_errors now returns
{requeued, pruned} and the toast shows both.
Poison-loop guards (release and lease-expiry burn no attempts, so a job that
stalls its transfer or crashes the agent every time cycled forever —
operator-observed jobs 99044/125288/131594/143131):
- agent: 3 in-session transient bounces (fetch or submit) → fail with the real
reason instead of another release; strikes never count while stopping, and
clear on submit success. Agent build 2026-07-02.3.
- server: the 60s orphan sweep (statements shared between the beat task and
GpuJobService so they can't drift) converts expired leases with >=5 lease
grants and pending jobs with >=10 to 'error', preserving the last stored
failure reason. Backstops old agent builds.
Tests: tombstone rule across all three backfill variants, moot-row pruning,
poison conversions, and the extended /retry_errors dedupe contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
|
||
|
bvandeusen
|
3d6201734c |
fix(settings): maintenance tiles start collapsed; remember manual open state
GpuAgentCard was hardcoded :open=true, HeadsCard opened whenever any head existed, TagEvalCard whenever a persisted run existed — so a fresh Settings load greeted the operator with several tiles already expanded. All three now force-open only while their task is actually running (the #877 resurface behavior on the busy-driven tiles is untouched). MaintenanceTile additionally persists MANUAL expand/collapse per tile in localStorage, so the section reloads the way the operator left it; a forced open while a task runs stays transient and is never saved as a preference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM |
||
|
bvandeusen
|
686808d3f3 |
feat(gpu): "Retry errored jobs" — scoped requeue of errors only
After an agent-side fix (e.g. the short-video sampler), the errored jobs
(~2.8k) have exhausted their 3 attempts and stay parked: backfill skips
images that already have a job, and /reprocess is the nuclear option (it
resets the 179k DONE jobs too). There was no way to re-run just the errors.
POST /api/gpu/retry_errors resets every status='error' job (all task types)
to pending with attempts=0 and the stored error cleared — a small inline
UPDATE that returns {requeued: n} so the UI toast can show the count.
UI: a "Retry errored jobs" button on the GPU-agent card, right under the
queue tiles; disabled when errored==0. With the agent now logging ffmpeg's
stderr on failure, retrying also reveals which errors were real vs victims
of the fps-filter bug.
Test: retry_errors requeues the errored job (fresh attempts, error cleared)
and leaves done work untouched; asserts via column selects (Core-DML
gotcha), not ORM refresh.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
bvandeusen
|
359bc5a283 |
feat(ml): default to SigLIP 2 (new installs) + model dropdown, no free-text (#1203)
- Migration 0069: new installs default to SigLIP 2 (so400m, 512px, 1152-d drop-in) — UPDATE applies ONLY where no image is embedded yet (fresh install), so an existing library is NOT silently invalidated; it switches deliberately via the dropdown → Re-embed → Retrain. Column server_defaults moved to SigLIP 2. - GET /api/ml/embedder-models: server-authoritative supported list (SigLIP 2 512 recommended / 384 faster / SigLIP 1 384 original) so the UI never free-types. - GpuAgentCard: the two name/version text fields → a single model dropdown; Save sets name+version from the picked option (the current model is always selectable even if off-list). - embedder.py DEFAULT_MODEL_NAME unchanged (stays the baked local-dir SigLIP 1) to avoid a local-dir/weights mismatch; SigLIP 2 loads by HF name, cached on the ml-worker's persistent HF_HOME. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
80f8eb4756 |
feat(gpu): re-process trigger to apply new crop detectors to the existing library (#1202)
The siglip/ccip backfills skip images that already have current-version regions, so adding crop detectors only affected NEW images — the back-catalogue would never be re-cropped. Add a reprocess trigger that resets every done/error job of a task back to pending, so the agent re-runs the FULL pipeline (figure detection + CCIP + concept/panel crops) over the whole library under the current detectors. - reprocess_gpu_jobs(task='ccip') task + POST /api/gpu/reprocess. - gpu store reprocess() + GpuAgentCard "Re-process library (re-detect + re-crop)" button with a confirm (it's heavy). - Test: a done job resets to pending (attempts cleared). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
bc6d43d3f2 |
refactor(ml): drop dead tagger/suggestion settings + columns (#1199)
Hygiene follow-up to the Camie retirement (#1189) — these were left inert to bound that change; nothing reads them now. Migration 0068 drops: - ml_settings: tagger_store_floor, tagger_model_version, suggestion_threshold_ character/general (already dead pre-retirement — scoring uses per-head thresholds), video_min_tag_frames (only the deleted video-prediction aggregator used it). - image_record: tagger_model_version (no writer), centroid_scores (dead JSON cache, no reader). Also: ml_admin _EDITABLE/GET/_validate pruned (dropped the store-floor invariant + video_min_tag_frames check); MLThresholdSliders trimmed to a video-embedding card (interval + max frames only); importer no longer resets the dropped cols; download_models drops the Camie fetch; stale CASCADE comments in cleanup_service no longer name the removed tables. Tests updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
485387ff0b |
refactor(ml): retire the Camie tagger + allowlist bulk-apply (#1189)
Heads + CCIP are the tag source and head auto-apply is the earned propagation.
The Camie tagger ran only to feed the allowlist bulk-apply (its ImagePrediction
rows had no other consumer), and the allowlist was a SECOND, un-earned auto-apply
path firing in parallel with heads on every accept — exactly the un-earned spray
the v2 pivot replaced. Retire both.
Behavior change: accepting a suggestion now applies the tag to THAT image only
(source='ml_accepted', a head-training positive) — it no longer allowlists +
fans the tag across the library via Camie. Propagation is heads' earned
auto-apply. (Loses instant cold-start propagation for booru-vocab tags; that was
un-earned and bypassed the precision gate.)
- tag_and_embed is now EMBED-ONLY (no Camie load/infer, no ImagePrediction
writes); backfill enqueues it for images with no embedding.
- Removed: services/ml/tagger.py, apply_allowlist_tags + helpers + daily beat +
every enqueue caller (accept/alias/merge/per-image), api/allowlist.py +
blueprint, ImagePrediction + TagAllowlist models/tables (migration 0067),
AllowlistTable.vue + allowlist store, the accept coverage-projection payload.
- AllowlistService gutted to accept/dismiss/undismiss/reject (the rejection store
the rail still needs); accept returns nothing, API returns {accepted, tag_id}.
- tag merge no longer repoints/triggers the allowlist; _keep_as_alias now keys on
ML-applied image_tag sources (incl. head_auto) instead of the allowlist.
- UI: MLBackfillCard relabelled to embedding-only; accept toast simplified;
MaintenancePanel drops the allowlist tile.
Left for a follow-up hygiene pass (now-inert, harmless): the dead settings
columns (tagger_store_floor, tagger_model_version, suggestion_threshold_*,
video_min_tag_frames), image_record.tagger_model_version, MLThresholdSliders
trim, and the Camie model download in download_models.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
|
||
|
bvandeusen
|
3d77a38a25 |
refactor(ml): remove the dead per-tag centroid subsystem (#1189)
The v2 pivot replaced per-tag SigLIP centroids with learned heads + CCIP. Centroids were still recomputed (on every tag merge + a daily beat) but NOTHING read them — suggestions come from heads+CCIP and apply_allowlist_tags applies via Camie predictions, not centroids. Pure dead wiring; remove it. Removed: CentroidService, recompute_centroid/recompute_centroids tasks, the daily beat, POST /api/ml/recompute-centroids, the recompute-on-merge trigger, the tag_reference_embedding table + model, the centroid_similarity_threshold + min_reference_images settings (migration 0066), the CentroidRecomputeCard + its store action + MaintenancePanel tile, and the centroid slider in MLThresholdSliders. _keep_as_alias drops its vestigial has-centroid branch (the allowlist branch already covers "could re-emit"); tag merge no longer clears a table that no longer exists. NOT touched (still live, parallel to heads): the Camie tagger, ImagePrediction, and the allowlist bulk-apply — accepting a suggestion still allowlists + applies it across the library. The tag-eval "centroid" baseline metric is unrelated (in-memory) and stays. (image_record.centroid_scores JSON column also remains — separate legacy field, its own micro-cleanup.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
4daa3f2790 |
feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.
- settings: embedder_model_name is now a setting (migration 0065) alongside the
existing embedder_model_version; both editable + validated (non-empty) in the
ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
model-agnostic), preferring the pre-downloaded local dir for the default so
existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
under the lease-announced model → POST /jobs/submit_embedding writes
image_record.siglip_embedding + siglip_model_version. The lease now announces
the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
version images; 'siglip' now re-embeds concept crops whose version != current
(so a swap re-triggers crops, not just the never-embedded back-catalogue). The
CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
churn the library at 512px) — the GPU agent owns version re-embeds. Daily
'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
image gated by siglip_model_version, concept regions by embedding_version) so a
mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
"Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.
Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
|
||
|
bvandeusen
|
c6f38b0dac |
feat(tagging): SigLIP concept crops + max-over-bag scoring (#114)
Lift recall on small/local concepts (glasses, cum, stomach-bulge, xray, lactation) that the whole-image SigLIP vector washes out: the GPU agent now embeds figure crops with SigLIP too, stored as kind='concept' regions, and the suggestion rail scores each image as a BAG (whole-image + every concept crop), taking each head's MAX over the bag. The whole-image vector is always in the bag, so this can never score lower than before. Model-agnostic by construction: the server ANNOUNCES the embedding model (HF name + version) in the lease, so the agent loads whatever the heads were trained in and stays in lock-step — a model swap is a server setting + a re-embed migration, never an agent change. - agent: model-agnostic CropEmbedder (torch/transformers get_image_features, fp16 on CUDA, inference-locked); worker branches on job.task — 'ccip' emits figure(CCIP)+concept(SigLIP) in one pass, 'siglip' emits concept-only so the back-catalogue backfill never churns figure/CCIP regions; torch cu124 + transformers in the image. - server: lease announces embed_model_name/embed_version; score_image is max-over-bag (version-filtered region embeddings); enqueue_gpu_backfill 'siglip' gates on a missing concept region (drains the back-catalogue, retries failures, no double-enqueue); daily siglip-backfill beat; UI button; /api/ccip/overview reports images_with_concept_siglip. - v1 scope: suggestion rail only — auto-apply stays whole-image (conservative; heads' thresholds were calibrated on whole-image). Bulk-apply bag = follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
b91a230f12 |
feat(ccip): automation + reference quality — keep identity flowing hands-free (#114)
Works through the optional CCIP ideas + the "keep moving even if I forget" ask:
AUTOMATION (no button needed):
- Hourly beat auto-enqueues CCIP backfill — new images get embedded (and errored
ones retried) on their own; the queue never goes idle waiting for a click.
- CCIP auto-apply: a daily sweep tags confident matches (source='ccip_auto') so
identity tags keep flowing. ON by default (opt-out, like head auto-apply);
ml_settings.ccip_auto_apply_enabled + _threshold (0.92, above the suggest cut),
migration 0064. Vectorized (one matmul + reduceat per image), reversible, skips
already-applied/rejected. Switch + threshold in the GPU agent card; GET/PATCH
/api/ml/settings; auto_applied count in /api/ccip/overview.
REFERENCE QUALITY (the over-fire root cause):
- character_references now draws ONLY from single-character images — on a
multi-character image the tag is image-level, so every figure would otherwise
pollute each character's prototypes (a 2-char image tagged 'Velma' made
Daphne's figure a Velma reference). This is the contamination behind residual
over-firing.
- Cached on a cheap signature (char-tag count + ccip-region count/max-id) so the
reference load isn't redone on every modal open.
Tests: multi-character image not used as a reference; auto-apply tags a confident
match as ccip_auto.
NEXT (not done, confirmed): comic-panel cropping + SigLIP concept crops ("spot
interesting content").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
|
||
|
bvandeusen
|
625336b6b4 |
feat(ccip): tunable match threshold, default 0.85 (#114)
Live data showed the v1 flat 0.75 cosine over-fired — ~64% of matched images got
3-10 character guesses dominated by the most-referenced characters (a 27-ref
character clears a low bar on many images). A sweep showed 0.85 collapses the
noise (noisy multi-matches 47→3) while keeping the confident single-character
matches.
- ml_settings.ccip_match_threshold (migration 0063, default 0.85); match_image
reads it (override still accepted). DEFAULT_SIM_THRESHOLD fallback 0.75→0.85.
- Exposed in GET/PATCH /api/ml/settings (validated 0.5–0.999).
- Slider in the GPU agent card ("Character-match strictness") — tune live, no
redeploy, same observe-and-tune loop as auto-apply.
Test: a ~0.9-cosine figure matches at 0.85, dropped at 0.95.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
|
||
|
bvandeusen
|
d91eef7a4b |
feat(gpu): GPU agent admin card — token, queue, backfill (#114)
The FC-side control surface the operator asked for: Settings → Tagging → "GPU agent". Generate/reveal/copy/rotate the agent bearer token (with the FC URL to point the agent at), see the live job-queue depth (pending/in-flight/done/ errored, polled), and a "Queue character embedding (CCIP)" button that triggers the library backfill. Plain-HTTP-safe copy (copyText resolves on success, throws on fail). Closes the "how do I get the token in the UI" gap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
1463794778 |
feat(heads): auto-apply UI on the Concept-heads card (#114 auto-apply C)
Surfaces earned auto-apply + its observability in Settings → Tagging → Concept heads: - Auto-apply section: an on/off switch (writes head_auto_apply_enabled), the precision-target + min-examples-to-fire tuning inputs, a Preview (dry-run → "would apply N", per-concept chips) and Apply-now button, with live run state. - "How auto-apply is landing": per-concept table from /api/heads/metrics — applied volume, misfires, realized misfire rate (green/amber/red), and missed (under-fires) — the signal to tune the precision target from. store: autoApply(dryRun) / autoApplyStatus() / metrics(). Card polls the sweep to completion, then refreshes counts + metrics. Completes the auto-apply task. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
06d5e83da4 |
feat(heads): admin card to train + inspect concept heads (#114 B)
The UI for the heads subsystem: Settings → Tagging → "Concept heads". Shows head count, auto-apply-ready count, and last-trained; a Train/Retrain button (one run at a time, polls while running, surfaces a failed run's error); an empty state guiding the operator to tag first; and a per-concept table (name, category, +tags, AP, P, R, auto-apply ⚡) sorted strongest-first so weak/under- tagged concepts are obvious. Rehydrates status from GET /api/heads on mount so it survives navigation. Pulls head_min_positives from ML settings for copy. Slice C (swap the rail's suggestions to heads, remove Camie + centroid) is next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa |
||
|
bvandeusen
|
b69c70ab2b |
feat(tag-eval): "keep" records a confirmation so doubts stop resurfacing
"Keep" on a doubted positive was a no-op, so the same confirmed-correct images came back in "head doubts" every run (operator-flagged: reinforcement keeps surfacing the same images). Add tag_positive_confirmation (mirror of tag_suggestion_rejection): keep → POST /images/<id>/tags/<tag_id>/confirm, and the eval excludes confirmed positives from the doubts list — exactly as rejected items already drop out of the suggest list. The tag stays a positive either way (confirmation is a "reviewed" marker, not a training change). - model TagPositiveConfirmation + migration 0057; confirm endpoint (idempotent). - tag_eval: _confirmed_ids + exclude from head_doubts_positive examples. - store.confirmTag + card "keep" calls it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
5143f4c34f |
feat(tag-eval): auto-apply operating point + server-side top-N concept discovery
Two additions driven by "what's the commit threshold?" + "find more tags":
1. High-precision operating point (Bar 4). Per concept, report the threshold that
maximizes recall while holding precision >= a target (default 0.97, configurable
via `precision_target`) — i.e. "could this fire without a human, and how much
would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
null if the target is unreachable. Surfaced on the card.
2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
concept list with the N most-tagged general tags (one fast DB query) so the
eval can broaden itself without hand-listing — replaces the slow HTTP directory
paging. Card gains "+ auto-add top-N" and precision-target inputs.
No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
bvandeusen
|
fc64f130b8 |
fix(tag-eval): thumbnail click opens the view modal, not Explore
Clicking an example in the maintenance card navigated to /explore/<id> — heavier than wanted (operator: just want a bigger look). Open the existing app-wide ImageViewer modal via modal.open(id) instead: bigger image + tags in place, no navigation away from Settings. The ✓/✗ actions are unaffected (separate overlay buttons). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
13d297b881 |
feat(tag-eval): inline confirm/reject actions on example thumbnails
Closes the learn-from-tags loop directly on the eval lists (operator-flagged: no surface to confirm/refine the head's suggestions). Each thumbnail gets a green ✓ / red ✗ that writes the SAME tables the head trains on: - suggest + ✓ → apply tag (new positive, POST /images/<id>/tags) - suggest + ✗ → record rejection (hard negative, suggestions/dismiss) - doubt + ✗ → remove tag + record rejection (kill bad positive, add negative) - doubt + ✓ → keep (stays a positive, no write) Acted thumbs grey out with a badge; re-run to see the head sharpen. Thumb still links to /explore/<id>. All endpoints already existed — no backend change. Inline is the starting point; longer-term the modal Suggestions rail gets the red "No" (negative) so per-image rejection is native there too (next slice). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4974b7cf77 |
feat(tag-eval): bigger, clickable example thumbnails (label-review queue)
The 56px example thumbs were too small to judge a label (operator-flagged). Bump to 120px and wrap each in a link to /explore/<id> (new tab) so the "head doubts / would suggest" galleries double as a review-and-fix queue — click a doubted positive, land on it in Explore, correct the tag, re-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
6cd7281af5 |
feat(settings): tag-eval admin card — trigger + persisted report (survives nav)
Frontend for #1130. A maintenance tile in Settings → Tagging: - Editable concept list + "Run eval" → POST /api/tag-eval (one running at a time). - Rehydrates on mount via the persisted run (getRun by latest id) and polls while running — so the report SURVIVES navigation (operator-flagged); the task runs backend-side regardless and the card reconnects to its row. - Renders the saved report: per-concept head-vs-centroid metrics table (AP/F1/ precision/recall) with Δ AP, the learning curve (AP @ N positives), and thumbnail galleries (head-would-suggest / head-doubts-positive) for eyeballing. Backend: _examples now stores thumbnail_urls (not just ids) so the report is a self-contained artifact that renders without per-id lookups on reload. No new top-level surface — slots into the existing maintenance area. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
958378312c |
fix(settings): sticky headers on the virtual data tables
Allowlist / Alias / ImportTask tables scroll their bodies (height=360/480) but the column headers scrolled away with the rows, so you lost the column labels (operator-flagged 2026-06-27). Add Vuetify `fixed-header` so the header row stays pinned while the body scrolls. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
e49cea3eba |
feat(tagging): allowlist tuning dashboard + post-accept toast + merge preview UI (#7c/#7d/#8b)
Cluster B frontend, milestone #99. #7c: AllowlistTable gains Applied + Covers columns and a live 'covers ~N at T' projection as the operator drags a row's threshold (debounced coverage call, then commits the threshold). allowlist store gains coverage(tagId, threshold) and refreshes coverage_count after a save. #7d: suggestions store surfaces a non-blocking toast when accept/alias newly allowlists a tag — '<verb>: <tag> — allowlisted, auto-applying to ~N images' (N is the projection; apply runs async). Falls back to the plain toast when the tag was already allowlisted. #8b: TagsView merge picker now previews the merge via usePreviewCommit before committing — shows images moving / already-on-target / series pages / alias-or- delete / a thumbnail sample, blocks the Merge button on an incompatible kind/fandom. adminStore.mergeTags gains a dryRun option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX |
||
|
bvandeusen
|
7c94d99b9f |
refactor(settings): canonical usePreviewCommit for maintenance preview→commit tiles (#753)
Frontend pattern-consistency sweep (note #1026, the last DRY-thread item). TagMaintenanceCard (4 flows) + PostMaintenanceCard (2 flows) each hand-rolled the same sync preview→commit state machine: a previewData/previewing/committing triple + onPreview/onCommit that dry-run-previews, then applies and collapses the projection (the apply shares the backend predicate, so afterward it's empty). Extract usePreviewCommit({preview, commit, emptyPreview}) owning that lifecycle. The 6 flows become declarative: supply the two thunks + the collapse shape. The normalize flow (commit dispatches a self-resuming background task, not a sync apply) omits emptyPreview so the projection stays and a truthy result = queued. Composable returns are aliased to the cards' existing local names, so the templates only change where they read the apply result (the success badges). Long-Celery-task cards (GatedPurge/VideoDedup) keep useMaintenanceTask — a different pattern (navigable-away task lifecycle), deliberately not merged. Exhaustiveness: no card hand-rolls the refs anymore; the only dryRun:false callers are these two cards, both via the composable. Added a vitest spec for the primitive (collapse static + fn, dispatch-variant, re-preview clears result). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
311fe0ee9c |
feat(settings): tidy Maintenance tab into compact tiles + center the views (pass 2)
Goal (operator 2026-06-18): the overview of a Settings tab fits one unscrolled viewport; expanding a tile to read into it is the only reason to scroll. - Every Maintenance card converted to the collapsible MaintenanceTile (collapsed by default = icon + short title + one-line blurb). Task cards (ML backfill, centroids, thumbnails, archive re-extract, missing-file repair, DB maintenance) sit in a responsive grid; running tasks auto-expand. Tagging config (suggestion thresholds, allowlist, aliases) grouped in one Tagging section as collapsible tiles; Backup is its own collapsible tile. - Three labeled sections mirror the Cleanup tab: Backfills and reprocessing / Tagging / Storage. - Center the whole Settings surface: SettingsView is now a centered, width-capped (1140px) column so the tab strip and every panel sit in a tidy centered measure (was full-width). CleanupView drops its own left-aligned max-width to fill it. All card logic unchanged - only the chrome. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
3b435dc0ba |
feat(settings): tidy Cleanup tab into sectioned compact tiles (pass 1)
The Cleanup + Maintenance sections had ~17 full-width stacked cards with long descriptions — a hunt to scan. Operator wants compact, sectioned, scannable tiles (2026-06-18: keep both tabs, group inside, compact tiles in a grid). New common/MaintenanceTile.vue: a compact expandable tile (icon + short title + one-line blurb collapsed; click the header to expand the full controls/preview/ result inline; keyboard-accessible button + focus ring; tints the icon, keeps a running task expanded). Cleanup tab (this pass) restructured into 3 sections — Import-filter audits (Min dimensions, Transparency, Single-color) / Duplicates & posts (Bare posts, Duplicate posts, Deduplicate videos, Gated-post previews) / Tags (Unused, Legacy, Reset content tagging, Standardize casing) — each a responsive grid of tiles. PostMaintenanceCard split into 2 tiles, TagMaintenanceCard into 4. Moved VideoDedupCard + GatedPurgeCard from the Maintenance tab here (both are destructive content cleanup). All card logic unchanged — only the chrome. Maintenance tab tiling is pass 2 (TODO noted in MaintenancePanel). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
eff64275fc |
feat(maintenance): reconcile duplicate posts (gallery-dl→native unify)
An artist first downloaded by gallery-dl gets Post rows keyed by the per-
attachment id; a later native walk keys the SAME real post by the post id. They
never dedup (uq_post_source_external_id is on external_post_id) → duplicate post
rows (cheunart: 943→1109). The real post id is recoverable in-DB from
raw_metadata['post_id'] (both eras store the sidecar there).
reconcile_duplicate_posts (cleanup_service): group posts by (source_id, canonical
post_id = raw_metadata.post_id else external_post_id); for each group >1, keep the
row already keyed by the post id (the format the CURRENT native downloader
produces, so future walks dedup and this can't recur), re-point
ImageRecord.primary_post_id / ImageProvenance / PostAttachment / ExternalLink onto
it conflict-safe (drop the loser's row where the keeper already has the equivalent,
per each table's uniqueness), backfill the keeper's empty date/title/body/raw_meta
from a loser, set external_post_id=post_id + derive post_url, delete losers.
IMAGES ARE NOT TOUCHED (content-addressed/deduped already; operator-confirmed).
Preview/apply share find_duplicate_post_groups (rule 93). API
/api/admin/posts/reconcile-duplicates (dry_run→{groups,posts_to_merge,sample};
apply→{groups,merged,sample}; optional source_id). UI: a second section on
PostMaintenanceCard (preview groups+sample → confirm merge). Tests: merge +
metadata backfill + image move, no-op when unique, provenance-collision dedup.
Design: milestone #73. Forensics: note #917. Out of scope (flagged): cheunart vs
Cheunart case-variant artist dirs/rows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
e287802ecb |
fix(maint): resurface dedup/gated-purge results after navigate-away (#877)
Long-running maintenance tasks must survive navigating away or reloading
the page. VideoDedupCard + GatedPurgeCard held the in-flight Celery task id
only in component refs and polled task-result inline, so leaving the page
mid-run lost the id and the result was never shown — even though the task
finished on the worker.
New shared composable useMaintenanceTask: persists {taskId, mode, startedAt}
to localStorage on dispatch, re-attaches on mount, and re-shows the result
when the task finishes (the celery result backend retains the summary well
under result_expires). Stale-guard skips resume past 3h. Both cards refactored
onto it; card-specific computeds + confirm dialog kept.
Also fixed the QueueStatusBar lane: both cards watched queue="maintenance"
but tasks.admin.* routes to maintenance_long, so the bar never reflected
their own task — now queue="maintenance_long".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
bvandeusen
|
540151290b |
feat(cleanup): purge misgrabbed gated-post blurred previews (#874 follow-up)
A one-shot Maintenance action to remove the blurred locked-preview images the ingester downloaded from tier-gated Patreon posts before #874. current_user_can_view was never persisted, so the cleanup re-walks each enabled Patreon source (read-only) to re-derive which posts are gated now and the blurred filehashes Patreon serves for them, then matches by CONTENT HASH against stored source_filehash. Because the hash is content-addressed, a real file downloaded when access existed has a different hash and can never match — regained-then-lost-access content is provably spared (operator's hard requirement). NULL source_filehash => unverifiable, kept + reported. On apply: delete matched ImageRecords + files (provenance cascades), clear seen/dead-letter ledger rows for those hashes so the real media re-ingests if access returns, and delete gated posts left bare. Shares one match predicate between preview and apply (rule 93). - cleanup_service: collect_gated_previews + purge_gated_previews - tasks.admin: purge_gated_previews_task (async re-walk bridge, timeboxed) - api.admin: POST /maintenance/purge-gated-previews - GatedPurgeCard.vue in Settings > Maintenance (preview -> confirm -> apply) - tests: collect predicate, hash-match delete/spare/unverifiable, ledger clear, bare-post removal, no-op Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
369e3de684 |
feat(ml): cadence-based video frame sampling + min-frame tag aggregation (#747)
Video tag noise root cause: frames were a FIXED count (6) max-pooled — a tag firing on one frame survived at peak confidence, and a fixed count under-samples long multi-scene videos so real scene-local tags looked like noise. Redesign (operator-steered): - Sample at a fixed CADENCE — one frame every `video_frame_interval_seconds` (default 4) across the 5–95% window — so a tag's frame-presence reflects real screen time independent of video length. Capped at `video_max_frames` (default 64): a long video stretches the spacing instead of exploding into hundreds of inferences, bounding per-video cost on the single ml-worker (per-frame ffmpeg timeout also cut 60s→30s). - Aggregate with `_aggregate_video_predictions`: keep a tag only if it appears in >= `video_min_tag_frames` sampled frames (≈ that many × interval seconds on screen — duration-independent noise rejection), with confidence = MEAN over the frames it appears in (not max). Clamps the threshold to the sample count so a 1–2-frame short video still tags. - All three knobs are DB-backed ml_settings (migration 0053), patchable via /api/ml/settings + sliders in the ML settings card — replaces the VIDEO_ML_FRAMES env var (product-not-project). Tests: aggregation drops one-frame noise + means corroborated tags + clamps on short videos; settings round-trip + min>max validation. Replaced the _maxpool_predictions unit test. NOTE: this is the QUALITY half of #747. The perf half — the ml-worker runs CPU-only — is GPU enablement, tracked separately in #872. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
41652db20f |
feat(maintenance): retroactive video-dedup action — preview + apply (#871)
Phase 2 of #871: clean up the duplicate videos already in the library (the #859 "same video from multiple sources" clutter). Import-time dedup (Phase 1) only prevents NEW dups; this is the operator-triggered cleanup of existing ones. cleanup_service.dedup_videos(dry_run): - backfill_video_durations: re-probe NULL-duration videos (pre-#871 rows) so the existing library participates; idempotent (only NULL rows), writes a negative sentinel for un-probeable files so they're neither re-probed forever nor matched. - find_video_dup_groups: cluster same-artist videos by duration (±tol) + aspect, anchored per cluster to bound the span (no chain drift); keeper = highest pixel area then bytes. Reuses the importer's _VIDEO_DUP_* tolerances. - apply: re-point each loser's post links to the keeper (so no post loses the video) THEN delete the redundant records + files via delete_images (cascade). dry_run shares the same discovery predicate and returns the projection only (rule 93). Tags on a loser are NOT merged (noted; videos rarely hand-curated). - dedup_videos_task (maintenance queue; summary → task_run.metadata). - POST /maintenance/dedup-videos {dry_run} + GET /maintenance/task-result/<id> so the card shows the dry-run projection before the destructive apply. - VideoDedupCard: Preview → shows groups/redundant/reclaimable, then Apply behind a confirm dialog. Mounted in the Maintenance panel. Tests: dedup collapses + re-links the loser's post to the keeper + removes the file; dry-run deletes nothing; distinct durations aren't grouped; task registered. (Migration 0052 for duration_seconds already shipped with Phase 1.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
949c9abcc6 |
fix(external): path-safe unlink + per-link staging + orphan repair (#859)
External downloads import IN PLACE, so the post-attach dedup-skip unlink could delete a file that IS an ImageRecord's backing file — orphaning the record and 404-ing on playback. Two sources of that: - Two links on the same post (same film from mega + gdrive) emitted the same filename into one external/<post_id>/ dir; the second overwrote the first. Stage per-LINK now (external/<post_id>/<link_id>/) so each file keeps its path. - The duplicate_hash/duplicate_phash branch unlinked `f` unconditionally. Make it path-safe: only unlink when `f` is NOT the existing record's canonical file. Plus an operator-triggered orphan-repair maintenance task (prune_missing_file_records_task) to clean up records already orphaned by the bug: scans ImageRecords, deletes those whose file is gone (cascade), with an NFS-stall guard that aborts without deleting if a large sample is mostly missing. Wired through POST /api/admin/maintenance/prune-missing-files and a MissingFileRepairCard in the Maintenance panel. Tests: refetch-same-link keeps the canonical file; orphan repair deletes only real orphans and aborts on the mostly-missing guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
bvandeusen
|
3610ba495f |
feat(ml): drop image_record.tagger_predictions — image_prediction is sole store (#768 step 3)
Read cutover verified in prod (suggestions + allowlist read image_prediction; backfill complete at 908k rows / 51k images). Removes the old JSON column and everything that fed it: - ImageRecord.tagger_predictions column removed; migration 0046 DROPs it. tagger_model_version kept as the "tagged / current?" signal the backfill sweep reads (needs-tagging check switched to tagger_model_version IS NULL). - tag_and_embed no longer dual-writes the JSON — image_prediction is the only write path. - importer re-import reset drops the JSON line (image_prediction rows are already deleted on re-import). - Retired the one-time #768 backfill task + the #764 prune task, their admin endpoints, and their Maintenance cards (Backfill/PrunePredictionsCard). - Tests seed/assert via image_prediction; stale column refs removed. Disk reclaim is NOT automatic: DROP COLUMN is a catalog change. Run `VACUUM FULL image_record` off-hours afterward to return the ~100 GB to the OS so DB backups go small (#739). image_prediction (~90 MB) stays in pg_dump — it's the source of truth now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
65211a3f2f |
fix(migration): make 0045 DDL-only; backfill image_prediction via batched task (#768)
The inline INSERT…SELECT backfill in migration 0045 wrapped the table creation and a ~100 GB pass over image_record.tagger_predictions in one transaction: nothing committed until the end, it was unmonitorable, and an earlier MATERIALIZED-CTE form spilled the full 100 GB to temp on NFS. A deploy got stuck on it for ~2h with image_prediction never appearing. Split the concerns: - 0045 now creates ONLY the table + indexes (instant DDL → web boots). - New backend.app.tasks.admin.backfill_image_predictions_task copies the >= store-floor predictions from the JSON into image_prediction, batched by id window and committed per chunk: live progress, resumable (re-enqueues from the last committed id), idempotent (ON CONFLICT DO NOTHING). json_each stays in the DB executor streaming each window — no Python-side 100 GB load, no materialization. - POST /api/admin/maintenance/backfill-predictions + a Maintenance-tab card to trigger the one-time run after upgrading. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
d55e52ae9b |
feat(admin): prune_low_confidence_predictions backfill task + UI (#764)
The one-time backfill that actually shrinks the DB: drops stored tagger_predictions entries below ml_settings.tagger_store_floor from every image_record row, and clamps any allowlist min_confidence below the floor up to it. Keep predicate (confidence >= floor) mirrors Tagger.infer's store gate so backfilled rows match new imports. Keyset by id ASC, idempotent, self-resumes on the soft time limit; runs on the maintenance_long lane. pg_dump copies live data only, so this alone fixes the #739 backup timeout — the reclaim (VACUUM FULL / pg_repack on image_record) is a separate, optional disk-return step, brief because post-prune the live data is tiny. - admin.prune_low_confidence_predictions_task + POST /api/admin/maintenance/prune-predictions - PrunePredictionsCard in the Maintenance panel (shows the current floor) - tests: registration + prune-keeps->=floor/drops-<floor + allowlist clamp Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c8b815afe6 |
feat(ml): clamp allowlist min_confidence to the tagger store floor
Consumer #4 of the store-floor change (#764). An allowlist tag can't auto-apply more permissively than the ingest floor — predictions below tagger_store_floor aren't stored, so a lower min_confidence behaves identically to the floor. update_threshold now clamps to max(value, floor); the AllowlistTable confidence input min-binds to the live floor and clamps on edit. Keeps the stored threshold honest about actual apply behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
3f92669f12 |
feat(ml): DB-backed tagger_store_floor (default 0.70), the ingest confidence floor
Promotes the prediction store-floor from the TAGGER_STORE_FLOOR env (default 0.05) to a DB-backed, Settings-UI-tunable ml_settings column (default 0.70). Storing every tag down to 0.05 from a ~10k-tag tagger is what grew image_record's TOAST to ~100 GB; the suggestion path already filters at 0.70 and the centroid/learned path covers lower-confidence preferred tags, so the sub-0.70 tail is redundant. Foundation for plan-task #764 (backfill + reclaim land next; this only changes the write gate for NEW imports). - ml_settings.tagger_store_floor (migration 0044, default 0.70) - tagger.Tagger.infer(store_floor=...); ml task passes settings.tagger_store_floor - ML admin GET/PATCH expose it; PATCH rejects a category suggestion threshold below the floor (nothing below the floor is stored, so the gap surfaces nothing) — server backstop for the UI slider clamp - Settings → ML: store-floor slider + caption; category sliders min-bound to it Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
70d4017cf6 |
feat(activity): search/filter on both Activity-tab panes
Recent failures gains a client-side search over the already-loaded 24h rows (task/queue/target/error), shown as a filtered/total count alongside the existing error-type chips. All recent activity gains a debounced server-side task-name search (new `task` ILIKE param on /runs) so it spans the full history, not just the loaded page. LIKE wildcards are escaped so task names' literal underscores match literally. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
9deebfa133 |
refactor(ui): CardHeading primitive for icon+title card/dialog headings (DRY pattern sweep)
The icon+title v-card-title heading (d-flex align-center + gap + <v-icon size=small> + <span>) was hand-rolled identically in 13 cards/dialogs (15 heading instances). Consolidate to <CardHeading icon title> (components/common) with an iconColor prop (error headings) and a default slot for trailing content (spacer+actions, inline status chip). Adopted everywhere the pattern appears — all-or-nothing per the hardened DRY process. Over-DRY guard: plain text-only <v-card-title> one-liners are NOT this pattern and stay; DownloadDetailModal leads with a status CHIP (not an icon), a different concept, left alone. §8b: the only remaining d-flex align-center v-card-title is that intentional variant. Catalog updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4854d74c5a |
refactor(ui): SampleNameGrid primitive for maintenance-card previews (DRY pattern sweep)
The preview sample-name grid (scrollable monospace chip grid) was hand-rolled 5 times with verbatim-duplicated markup + CSS — TagMaintenanceCard (×4) and PostMaintenanceCard. Consolidate to <SampleNameGrid> (components/common): pass :names for the plain case, default slot for the normalize from→to chips (styled via :slotted .fc-name). Removed the duplicated .fc-name-grid/.fc-name CSS from both cards. Over-DRY guard: only the verbatim-duplicated grid is merged — each card's preview/commit logic and result-count lines genuinely differ and stay put; MinDimensionCard's typed-token confirm is a separate variant, untouched. §8b: fc-name-grid now lives only in SampleNameGrid. Catalog updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
4e83b4225a |
refactor(ui): single global .fc-muted token (DRY pattern sweep)
The muted-text token was redefined identically in 12 component <style scoped> blocks. Consolidate to one global utility in styles/app.css; remove the 12 copies. Keeps the explicit on-surface-variant (vellum) token, NOT Vuetify's opacity-based text-medium-emphasis (per the muted-text-token rule). Behavior- preserving: every class=fc-muted usage now resolves to the single source. §8b exhaustiveness caught (and I fixed) my own sed clobbering the new app.css rule — now exactly one .fc-muted definition exists, zero component-local. Catalog updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
bvandeusen
|
c774042a85 |
refactor(ui): consolidate 7 hand-rolled kebabs into one KebabMenu (DRY pattern sweep)
First pattern-consistency DRY pass (process #594). The overflow kebab was hand-rolled 7 ways in two divergent activator strategies — Pattern A (#activator + v-bind) which silently breaks inside the teleported image modal (#711), and Pattern B (manual v-model + activator=parent + open-on-click=false + z-index 2400) the modal kebabs needed as a workaround. New <KebabMenu> (components/common) bakes in the modal-safe strategy UNIVERSALLY, so every kebab works in modal and non-modal contexts — folding the latent #711-class bug fix into all five Pattern-A sites. Menu items go in the default slot; variations (size/variant/location/label/min-width) are props. Adopted across all 7: TagChip, SuggestionItem, TagCard, SeriesView card, SeriesManageView, BackupRunsTable, SourceActions. Exhaustiveness (§8b): mdi-dots-vertical now lives only in KebabMenu. Labeled dropdowns / nav menus / filter popovers are a different concept and left alone. Seeded the pattern catalog so new code reuses the primitive. Test: KebabMenu renders slot items + trigger label/glyph + presentational props. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |