Commit Graph
115 Commits
Author SHA1 Message Date
bvandeusen a7e626a67a feat(extension): report the channel beside the version (step 7)
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 5s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 32s
extension / lint (push) Successful in 28s
CI / integration (push) Successful in 3m52s
Build images / sign-extension (push) Successful in 4s
Build images / build-ml (push) Failing after 5s
Build images / build-agent (push) Successful in 13s
Build images / build-web (push) Successful in 2m4s
Closes the half of the ask the signing work didn't: a way to tell a dev
build from a main one. FC_CHANNEL is baked into the web image at build
time and /api/extension/manifest reports it as its own key, next to
version — the popup banner, the toolbar tooltip and the Settings card all
name it.

Beside the version, never inside it. A `1.0.3499884-dev` suffix is the
obvious shortcut and it is the exact failure this design comes from:
versionIsNewer parses each dotted segment with parseInt, so a suffixed
segment reads as 0, every dev build compares equal to every other, and
"no update available" stops being distinguishable from "I cannot read this
version". The comparator already degrades rather than discarding (rule
150), which is a reason not to NEED the suffix, not a licence to add one.
Two tests hold the line — one backend, asserting version and channel are
separate keys; one frontend, asserting the rendered version text stays the
bare derived number.

Optional on the read side, and absent rather than defaulted. An image
built before this field says nothing by not having the key; an image built
without a channel now says nothing the same way, so there is one absence
to handle instead of a second spelling of "unknown". Every reader drops
the label entirely when it is missing and reads exactly as it did before.
Reported verbatim rather than validated against {dev, main}: if an image
declares something else, showing what it claims helps whoever is debugging
more than dropping it would.

FC_CHANNEL is declared LAST in the Dockerfile. An ARG invalidates every
layer below it, and this is the one value that differs between the dev and
main builds of identical source — earlier, and the two channels could
never share a cached pip install. A tag push counts as main: a vYY.MM.DD
tag is cut from main, so that image is a main-channel artifact wearing an
immutable name.

No channel switcher, deliberately. background.js:34 already records that
Firefox's static update_url cannot apply, because every FC instance is a
different host — so the extension asks its configured backend, and the
channel IS the instance it points at. Switching is repointing apiUrl and
reinstalling from that host. A separate setting would contradict each
server build shipping its own extension.

This commit touches packaged extension files, so it moves the derived
version and will sign a new one via AMO — the first push to exercise the
extension-changed path from dev end to end.
2026-08-27 11:47:30 -04:00
bvandeusen 1ac448d881 refactor: four small cleanups from the review pass (#3072)
CI / extension-version (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 4m57s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
Items 2-5 of #3072. Item 1 (the per-row sweep inserts) is separate.

2. .fc-bad was not merely duplicated — it is .fc-weak under a second
   name. Both local definitions were `color: rgb(var(--v-theme-error))`,
   identical to the global .fc-weak, and GpuAgentCard was already using
   .fc-weak to colour exactly what GpuActivityPanel coloured .fc-bad (an
   errored count, red when non-zero). So rather than promoting a synonym
   to app.css, both call sites now use .fc-weak and the local defs are
   gone. app.css's status-colour comment records why there is no .fc-bad,
   next to the existing note on why .fc-ok is deliberately NOT global.

3. GalleryItem.vue's obsidian literals now use --v-theme-background,
   which IS obsidian (vuetify-theme.js maps background -> surfaces.
   obsidian). Preferred over --fc-chrome-rgb: same value, but that
   variable is named for the nav fade, not for the palette entry.

   The ticket said these were the only three real uses in the tree. They
   are not — GalleryItem itself had two more in the artist-label
   gradient (fixed here, so the file is now consistent), and ~13 more
   live in SeriesView, SeriesReaderView, ImageViewer, ArtistHeader,
   ExploreView and GalleryFilterBar. Those are a separate sweep, filed
   rather than folded in here.

4. The attachment download path had two hand-formatted copies. One
   definition now, `attachment_download_url`, next to the model both
   serializers already import. The test pins it by MATCHING the built
   path against the app's real URL map rather than comparing to a
   literal — a string-equality test would still pass after someone
   renamed the route, which is the drift the helper exists to prevent.

5. Extension API key now compares with hmac.compare_digest. Compared as
   BYTES, not str: compare_digest's str form raises TypeError on
   non-ASCII, and this value comes straight from an attacker-controlled
   header, so the str form would turn a junk key into a 500 instead of a
   403. Low stakes either way — the API is unauthenticated-by-design on
   a LAN — but it costs nothing.

Refs #3072
2026-08-27 07:48:18 -04:00
bvandeusenandClaude Opus 5 ddf896078c refactor(platforms): retire deviantart end-to-end (#3069)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 23s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m43s
Executes the 2026-07-05 product decision (FC downloaders = art-dedicated
services only), which removed Twitter/X and Bluesky but left deviantart
fully wired for seven weeks — the half-retired state rule 22 exists to
prevent.

Removed: the PlatformInfo module and its registry entry, the gallery-dl
extractor block, extension_service's artist-page pattern, the extension's
PLATFORMS + PLATFORM_ARTIST_PATTERNS entries, its manifest host permission
and content-script match, the frontend icon/colour/label, and the operator-
facing "supported platforms" list that still advertised it.

Two judgment calls, both recorded in migration 0088:

  * existing `source` rows are DISABLED, not deleted. The row is the only
    record of the artist's DeviantArt URL. Disabling is also required for
    correctness rather than tidiness: with the platform unregistered the
    download path falls through to gallery-dl, which carries its OWN
    deviantart extractor, so an enabled row would have kept downloading
    from a dropped platform.
  * the `credential` row IS deleted — a live session cookie for a site FC
    will never call again.

Adds the invariant whose absence is why manifest.json drifted in the first
place: nothing tied its domain lists back to the platform table. The
extension suite now asserts both directions, plus that no host permission
belongs to an unclaimed domain (`*://*/*` exempted — FC is self-hosted at
an operator-chosen URL the extension cannot enumerate).

Extension version 1.0.10 -> 1.0.11: ci.yml's guard hard-fails a packaged
extension change without a bump. No release is cut — build.yml's
sign-extension job only runs on main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 07:24:08 -04:00
bvandeusenandClaude Opus 5 2e0f8f8c61 feat(cleanup): reclaim orphaned attachments — rows and store blobs (#3068)
CI / extension-version (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 28s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 24s
CI / integration (push) Successful in 3m47s
PostAttachment's two FKs are both ON DELETE SET NULL, so a deleted post or
artist left the row behind rather than taking it. Nothing ever pruned those
rows, and nothing in the repo had ever unlinked a file under the attachment
store — so both rows and bytes accumulated permanently, invisible to every
existing diagnostic.

Why a disk->DB reconciliation rather than a row sweep: the store is
sha-addressed and idempotent, so ONE blob backs MANY rows. Deleting a row
does not free its blob, and since the artist cascade (#3066) now deletes
its attachment rows outright, a freed blob has no DB pointer left to find
it by. Walking the store and asking "does any row still reference this
sha?" catches orphans from every cause, including ones no future delete
path will think to report.

Preview and apply share `_orphan_attachment_conditions` (rule 93). The
dry-run derives its surviving-sha set by NEGATING that same predicate, so
it is honest about blobs the delete would free rather than counting them as
still-referenced — the one place this was easy to get backwards, so it has
its own parity test.

Guards, each with a reason:
- A blob is written before its row commits, so a just-stored file legitimately
  has no referencing row. Files under 6h are never judged — same guard and
  reasoning as ORPHAN_TEMP_MIN_AGE_HOURS.
- `.partial` staging files belong to cleanup_orphaned_temp_files; skipped
  rather than raced.
- The sha is parsed as the first 64 chars, not via Path.stem: store() takes
  the extension from the source filename, and a URL-encoded basename yields a
  multi-dot suffix that would make stem eat part of the sha.
- A 900s walk budget reports partial=True instead of running to the task's
  hard limit (rule 89).
- TASK_STUCK_THRESHOLD_MINUTES override at 30 (= time_limit 25 + 5). Without
  it a healthy 20-minute walk is phantom-flagged 'RecoverySweep' at the bare
  5-min default — the #883 failure class; its invariant test is mirrored here.

Defaults to the safe preview at both the task and the route, unlike the other
maintenance triggers: this apply unlinks files. Operator-triggered only,
never on a beat.

Ships with its UI (rule 27): AttachmentReclaimCard in Cleanup → Duplicates &
leftovers, built on the existing useMaintenanceTask/MaintenanceTile shapes, so
a run survives navigating away. Surfaces files_failed and partial explicitly,
since both change what the numbers mean.

Also promotes humanBytes to utils/bytes.js — it was byte-identical in
VideoDedupCard and GatedPurgeCard and this card would have been the third
copy. The three divergent `formatBytes` helpers are deliberately left alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:37:24 -04:00
bvandeusenandClaude Opus 4.8 ec66ea5f83 refactor(ui): settings-card primitives + fix threshold clamp / card misgroup (#161)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m52s
extension / lint (pull_request) Successful in 10s
Tier-3 frontend DRY for the ML settings cards, plus the F-D2 clamp bug and the
F-D3 card misgrouping.

New primitives (components/common + composables):
- <SettingToggleRow> — the accent-icon + .fc-section-h label + right-aligned
  switch row (HeadsCard x3, CropProposersCard). iconColor prop absorbs the
  on/off dim.
- <SettingNumberField> — compact numeric field that CLAMPS to [min,max] on
  commit. This fixes F-D2: HeadsCard/CropProposersCard previously sent
  Number(raw) straight to the API, so an out-of-range threshold bounced off the
  400 validator (only TranslationCard clamped). density prop for the grid cards.
- useSettingSave(patchFn) — the busy + patch + toast + revert-on-failure flow
  each card hand-rolled (HeadsCard x6 handlers, CropProposersCard, MLBackfillCard,
  VideoEmbeddingCard). Returns ok/false for the optimistic-switch revert.

Adopted in HeadsCard, CropProposersCard, MLBackfillCard (handler only — its
plain labelled switch is a different affordance), VideoEmbeddingCard.

F-D3: MLThresholdSliders.vue actually rendered a "Video embedding" (frame-
sampling) card but sat under "Tagging → Suggestion thresholds". Renamed it
VideoEmbeddingCard.vue and moved it to the "GPU agent & embeddings" section.

Left deliberately (over-DRY guard): TranslationCard uses an inline error ALERT
(not a toast), already clamps its confidence with a NaN fallback, and lives on
the ImportStore — a genuinely different save pattern, so forcing it onto
useSettingSave would change its UX.

Behaviour-preserving refactor; CI has no Vue type-check so this needs a live
UI pass (toggles persist + revert on failure, thresholds clamp on blur, video
card now under Embeddings).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi
2026-07-13 22:28:47 -04:00
bvandeusenandClaude Opus 4.8 a2d1ed935d refactor(ui): DRY the settings-card CSS tokens + fix unstyled headers (#161)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m57s
- Promote .fc-section-h to a global token (app.css). It was copied identically
  into 4 cards, and TranslationCard used the class with NO local def — so its
  section headers rendered unstyled. Now fixed everywhere.
- Promote .fc-good / .fc-weak status colours to globals; delete the local copies
  in the GPU/heads cards. (.fc-ok stays local — divergent: on-surface in
  HeadsCard vs success in QueuesTable. .fc-bad stays — different name.)
- Delete 10 identical local .fc-muted redefinitions that crept back after the
  2026-06-09 sweep; the global utility already covers them.
- DbMaintenanceCard: opacity:0.6 muted text → the .fc-muted token (the exact
  anti-pattern that token's comment forbids).
- HeadsCard: collapse byte-identical ratePct() into pct().

CSS-only + one template class swap; no logic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NsmJSQxnNxGgtM5Yz4GAqi
2026-07-13 21:51:23 -04:00
bvandeusenandClaude Opus 4.8 6104452d2e feat(deps): vuetify 3→4 (batch 4 phase B, #1449)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m49s
Vuetify 4 is a Material-Design-3 styling refresh with revert snippets, not a
component-API overhaul. FC's exposure was small:

- Bump vuetify ^4, vite-plugin-vuetify ^2.1.0, vue ^3.5, engines node>=24.
- Restore the dropped global CSS reset (minimal reset from the upgrade guide) in
  Vuetify's own low-precedence reset layer, so FC's margin-zeroing assumptions hold.
- v-row prop→utility: 'dense' → density=compact (×3), align=center → class=align-center.
- v-snackbar: multi-line removed → min-height=68.
- v-autocomplete #item slot: item→internalItem (item now aliases raw) in TagPicker
  + GalleryFilterBar (item.raw.* → internalItem.raw.*).

ACCEPTED (cosmetic, operator reviews live per plan #158): MD3 typography
(text-body-2 ×73), non-uppercase buttons (v4 dropped the uppercase default),
MD3 elevation. CI verifies BUILD only — the LOOK is the live-review pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 13:35:01 -04:00
bvandeusenandClaude Opus 4.8 af0d39ed52 feat(wip): soft title tier — sketch/doodle vocab + ring-loud audit (#1474)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m53s
Extends WIP title-tagging to lower-precision cues (sketch/doodle/scribble) safely.

- wip_title.py: soft matcher (word-anchored; sketchbook/kadoodle don't trip it);
  WIP_TITLE_SOFT_SOURCE + soft SQL prefilter; apply_wip_image_tags takes a source arg.
- training_data._AUTO_SOURCES += 'wip_title_soft' → the soft tier is PROVISIONAL and
  never trains the wip head (a finished "sketch" can't pollute it). Only the hard
  tier (wip_title) + manual train.
- ImportSettings.wip_soft_title_tagging_enabled (OFF by default, opt-in). Migration 0087.
- importer: hard tier wins, soft is the fallback (source wip_title_soft).
- backfill: refactored into a shared _backfill_wip_tier; hard always, soft when enabled.
- heads.soft_wip_conflict_audit + daily beat: score soft-tagged images against content
  heads, flag ring-loud ones (PresentationReview mode=process) for the review strip —
  the operator's "measure if they got falsely tagged" safety.
- api settings toggle; ImportFiltersForm soft toggle.
- tests: soft matcher pos/neg; soft source not a training positive; audit flags
  ring-loud + spares quiet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 10:14:38 -04:00
bvandeusenandClaude Opus 4.8 d9a14e890d feat(system-tags): process auto-tag settings UI + mode-aware review strip (#1464)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m44s
Frontend for the system-tag refactor (milestone #157 step 6).

- HeadsCard: new 'Auto-tag work-in-progress' section (enable + tag-confidence +
  conflict knobs) for wip/editor process auto-apply, mirroring the chrome card;
  copy notes they stay VISIBLE and the head only learns from titles/manual (no
  runaway). Presentation copy narrowed to banner-only.
- HiddenReviewStrip: mode-aware — chrome flags read 'hidden as X / Keep hidden /
  Un-hide'; process flags read 'auto-tagged X / Keep tag / Remove tag'. Same
  endpoints (the backend returns mode), different words.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 23:22:13 -04:00
bvandeusenandClaude Opus 4.8 571938781a feat(tagging): title-based WIP auto-tagging (#1458)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Failing after 3m50s
Auto-apply the `wip` system tag to posts whose TITLE explicitly declares
work-in-progress ("WIP" / "work in progress") — a deterministic, high-precision
complement to the image-based ML `wip` head. WIP images are excluded from the
Explore/gallery browse, so honouring the artist's own label keeps unfinished
pieces out of the main browse.

- services/wip_title.py: precision-first token-anchored matcher (swipe/wiped
  never trip it) + sync apply helpers (source='wip_title', ON CONFLICT DO
  NOTHING, chunked under the psycopg param ceiling).
- importer: live hook on FRESH import only (never on deep-scan/supersede), so a
  manually-removed WIP tag is never re-applied by a routine re-scan.
- maintenance.backfill_wip_title_tags: operator-triggered back-catalogue sweep
  (coarse SQL prefilter + regex confirm, keyset-paginated). Deliberately NOT a
  beat — a periodic re-run would silently undo manual removals.
- ImportSettings.wip_title_tagging_enabled (default ON, migration 0085) gating
  the live hook; GET/PATCH + POST /settings/wip-title/scan.
- Settings UI: toggle + "Scan existing posts" button.
- Tests: pure matcher unit tests + integration (apply idempotency, backfill
  precision).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:12:19 -04:00
bvandeusenandClaude Opus 4.8 aea2701c28 feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m52s
The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh ==
cached, verified by probe) confidently mis-detects short ASCII English like
"... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a
re-translate reproduced it. Confidence alone can't separate the 0.86 collision
(genuine German lands there too), and single-word mis-flags sit at a confident
1.0 no floor catches.

Two operator-approved levers:

- Acceptance floor is now a live Settings value (ImportSettings.
  translation_min_confidence, default 0.90; surfaced in the Translation card), so
  it's tunable without a redeploy. _accept takes the threshold as a parameter.

- Per-post sticky override (Post.translation_override: auto/force/original).
  'force' stores a translation even below the floor (rescue a skipped
  legit-foreign title); 'original' keeps the original and clears any stored
  translation (kill a confident mis-flag no floor catches). The sweep honors it
  on every run and _reset_translations skips 'original', so the choice survives a
  Re-translate-all. POST /api/posts/<id>/translation-override applies it
  immediately (translate now when the service is up, else queue for the sweep).
  UI: PostTranslationControl on the posts-feed card.

Migration 0084 (both columns + a CHECK on the override). The feed + provenance
serializers expose translation_override.

With a stricter floor the rollback finally works: raise it -> Re-translate all ->
the 0.86 mis-flags are rejected and restored to the original; force /
keep-original handle the residual either way.

Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor
cases); sweep force/original + re-translate-skips-original; override endpoint
(validation, original clears, force queues when disabled, feed exposes it);
settings min_confidence default/save/validate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 22:38:25 -04:00
bvandeusenandClaude Opus 4.8 ffdbdbaf07 feat(translation): 8h sweep + drain button + detection probe (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 46s
CI / integration (push) Successful in 3m55s
Throughput: translate_posts now runs every 8h (was daily) as the
steady-state cadence for newly-imported posts, and the Settings
"Translate now" button runs it in drain mode (run-until-done, no reset)
so one press clears the whole untranslated backlog instead of a single
300-post chunk. The interrupt/backoff re-enqueue now preserves the drain
flag so a bulk drain resumes cleanly after an Interpreter restart.

Misdetection groundwork: surface the detector's confidence from the
Interpreter client (it was in the detectedLanguage payload but discarded)
and add a read-only "Test translation" box — POST /settings/translation/
probe + TranslationCard UI — that shows detected language + confidence +
engine + result for pasted text, without saving. Lets the operator see
why a short/abbreviation-heavy English title gets mis-detected so the
detection guard (min-length + confidence floor) can be tuned from real
numbers. The guard itself follows once the mis-detected cases are probed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-08 20:13:49 -04:00
bvandeusenandClaude Opus 4.8 4371ddb7e7 feat(translation): live re-translate progress in the Settings card
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m45s
CI / frontend-build (push) Successful in 20s
translation/status now reports `active` (a translate/retranslate sweep is
running, from the TaskRun table) and `last_run` (the most recent finished run's
task + status). The Settings card polls live while a sweep runs, showing a
spinner + "Translating… N remaining" that ticks down, and flags a last run that
ended in error/timeout. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:20:32 -04:00
bvandeusenandClaude Opus 4.8 1f6d94f51d feat(translation): re-translate on model change — artist-scoped + global re-run (#146)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
retranslate_posts resets the 5 translation columns to NULL for a scoped set of posts (all, or WHERE artist_id IN ids) then reuses the untranslated sweep to re-run them, chasing the tail until drained (run-until-done). Interpreter cache keys on engine_version so a changed model re-translates, an unchanged one is cache-fast. Reset only happens when the service is configured+healthy so translations are never wiped when they can't be rebuilt. New POST /settings/translation/retranslate (artist_id | all=true). UI: per-artist 'Re-translate posts' on the Artist Management tab + 'Re-translate all' in the Settings Translation card, both with confirm dialogs. No migration (reuses m143 columns).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:03:19 -04:00
bvandeusenandClaude Opus 4.8 6a255482ea feat(translation): "Test connection" button — on-demand Interpreter health check (#143)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m43s
New POST /api/settings/translation/test pings /v1/health for a GIVEN base URL (not
the saved one), so the operator can verify a URL before enabling it. TranslationCard
gains a Test-connection button that reports reachable/unreachable inline and
updates the status dot. Endpoint test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 13:00:37 -04:00
bvandeusenandClaude Opus 4.8 83c1745fd0 feat(ui): translation-forward post text + Settings card (#143 step 5)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 3m45s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
PostCard + modal ProvenancePanel show the English title/description by default when
a translation exists, with a per-card "show original (<lang>)" toggle — translated
bodies render as plain text, originals keep their sanitized HTML. New
TranslationCard in Settings → Ingestion & filters: enable switch, Interpreter base
URL (generic placeholder, no default host), target language, a reachability
indicator + untranslated-posts count + "Translate now".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:38:41 -04:00
bvandeusenandClaude Opus 4.8 1f548d8a7b feat(ui): presentation-chrome auto-hide Settings controls (#141 step 4)
CI / backend-lint-and-test (push) Successful in 32s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 3m41s
HeadsCard gains a "Hide presentation chrome" section: on/off switch + "Hide
confidence" (presentation_auto_apply_threshold) + "Flag if content ≥"
(presentation_conflict_threshold), wired to MLSettings via patchSettings and
loaded on mount. Makes the step-4 sweep's thresholds operator-tunable
(config-in-UI). wip is called out as never auto-hidden.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:16:41 -04:00
bvandeusenandClaude Opus 4.8 ab362bc79c feat(ml): Settings → Tagging 'Crop proposers' card (#134 step 3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m39s
Exposes the detector config (per-proposer enable + weights + confidence, caps,
dedupe IoU) in Settings → Tagging, backed by MLSettings via /api/ml/settings.
ml_admin adds the detector fields to _EDITABLE + GET payload + validation (conf
0..1, caps >=1, IoU 0..1). New CropProposersCard.vue (mirrors HeadsCard) with
working defaults pre-filled, per-field live-save (no restart — the agent picks
changes up on its next lease), weights-format help, switch-revert on error.
Closes milestone #134: all three proposers are on out-of-the-box and tunable in
the UI. Test: detector defaults GET + patch round-trip + range validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-05 19:51:55 -04:00
bvandeusenandClaude Fable 5 e039689eff feat(ia): wave 2 — Activity becomes the whole-app pulse; Overview gets the health strip
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m35s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
The Activity tab only knew Celery — the GPU agent (the majority of processing)
and the download pipeline were invisible there. Two new self-polling panels:

- GpuActivityPanel: queue depths + triage verdicts (defects / file-ok /
  unprobed, top reason buckets) with a jump to Maintenance -> Failed
  processing. The triage detail refetches only when the error count moves.
- DownloadsActivityPanel: 24h stat chips + failing-source names with a jump
  into Subscriptions.

Both panels join the Activity tab under Queues+workers AND double as the
Overview health strip (side-by-side grid under the Celery summary) — one
component set, so Overview answers 'is everything healthy?' across all
systems. SystemStatsCards reviewed: content still accurate, left as-is.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 17:45:45 -04:00
bvandeusenandClaude Fable 5 5b34c9221c feat(ia): wave 1 — Import tab dissolves, Maintenance regroups by system, one extension home
CI / frontend-build (push) Successful in 19s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m32s
Settings IA per the approved A3 design (the old layout was the two-app merge
fossilized):
- Import tab retired: ImportTriggerPanel + ImportTaskList deleted (manual
  /import scans stay API-level; imports arrive via downloads/extension, heal
  via the Layer-2 auto-refetch sweep, and show in Activity). ImportFiltersForm
  moves to Maintenance → 'Ingestion & filters' and loads its own settings; the
  import store shrinks to settings-only (no remaining consumers of the
  scan/task-list machinery). Overview's pending banner now points at Activity.
- Maintenance regrouped: Ingestion & filters / GPU agent & embeddings
  (GpuAgent, Failed processing, CPU embedding backfill) / Tagging (sliders,
  Heads, Aliases) / Library health (MissingFiles, Thumbnails, DB, Archive
  re-extract demoted last) / Storage.
- One extension home: BrowserExtensionCard moves from Settings → Overview to
  Subscriptions → Settings, above the API key bar it authenticates.
- Single-color import filter WIRED: skip_single_color/threshold existed since
  FC-2 but nothing read them (the audit module's docstring said as much) —
  now enforced on both import paths via the audit's canonical predicate
  (tolerance 30, matching the Cleanup card default; animated images exempt
  like the transparency check). Default stays off; test added.
- Dead weight: PlaceholderView (zero refs) and the permanently-disabled
  'Export failed logs (CSV — v2)' menu stub deleted; stale docs fixed
  (celery queue docstring, threshold comment citing retired tasks, ml
  package docstring, HeadsCard 'replaces Camie' blurb).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 17:37:21 -04:00
bvandeusenandClaude Fable 5 19b962f1a7 feat(b3): ml-worker becomes optional — embed-only role, decoupled GPU coordination, cpu-embed switch
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m31s
The ml-worker's ONLY processing role is now the CPU whole-image embed fallback
(tag_and_embed renamed embed_image — Camie tagging was retired #1189 and the
name kept implying otherwise; videos were already handled agent-style: frame
sampling + mean-pool). Detection/cropping/CCIP stay GPU-agent-only, and their
completion is judged per-pipeline: ccip by gpu_job rows, siglip by concept
regions at the current model version — never by image_record.siglip_embedding.
A CPU embed therefore can NEVER close crop work for the agent (regression test
pins this; only the whole-image 'embed' job, the same artifact, is satisfied).

Making removal actually safe (operator will drop the container):
- GPU-queue coordination (enqueue_gpu_backfill, recover_orphaned_gpu_jobs,
  reprocess_gpu_jobs) moved verbatim to tasks/gpu_queue.py on the maintenance
  quick lane — it lived on the 'ml' queue only by module colocation, which made
  the ml-worker a hard dependency of the whole agent pipeline.
- New ml_settings.cpu_embed_enabled (migration 0074, default ON so agent-less
  installs keep working): OFF stops the four import hooks queueing embed work
  nothing will consume and no-ops the manual backfill; switch lives on the
  renamed 'CPU embedding backfill' card.
- NB heads training / auto-apply still run on the ml image (sklearn) — a stack
  that removes the container gives those up too.

Deploy note: in-flight messages under the old task names are dropped by the
new workers; the 60s orphan sweep + hourly backfill re-fire under the new
names immediately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 16:53:08 -04:00
bvandeusenandClaude Fable 5 7c19ad91ed feat: cap-aware autoscaler + token-gated whole-instance tag reset (operator feedback)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m28s
Autoscaler (agent 2026-07-02.5): the buffer-occupancy signal alone would peg
downloaders at DL_MAX while the bandwidth CAP — not concurrency — is the real
constraint (8 streams sharing 8 MB/s move no more data than 4). Growth is now
gated on the pipe having headroom (net < 85% of cap) and a pipe pinned at the
cap (>= 95%) sheds streams down to 3; dead band prevents flapping. The UI hint
says 'holding at the bandwidth cap' and /status reports bw_capped, so the
behavior is legible without tests that need the ML stack.

Reset content tagging: stays a FULL-instance reset (operator's call), but now
lives in a fenced 'Danger zone' section on Cleanup and the apply is gated by a
preview-derived confirm token (mirrors the Tier-C bulk-delete pattern — stale
counts are rejected server-side). Copy no longer claims suggestions repopulate:
it says plainly the heads' training examples are deleted and re-tagging starts
fresh. Moved out of TagMaintenanceCard into DangerZoneCard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 16:14:48 -04:00
bvandeusenandClaude Fable 5 eaea4308fc chore: retire the tag-eval harness — it proved the heads system, job done (operator-approved)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m24s
The head-vs-centroid eval (#1130) existed to prove the 'frozen embedding +
trained head' spine; the operator accepted the tagging system and dropped the
harness. Removed per rule 22: TagEvalCard + store, /api/tag_eval blueprint,
tag_eval_run ml task, recover-stalled-tag-eval-runs sweep + beat entry,
TagEvalRun model + table (migration 0073), and its tests.

The eval's data loaders + metric helpers were NOT eval-specific — the nightly
heads trainer runs on them — so they moved verbatim to
services/ml/training_data.py (heads.py import updated; behavior unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 12:41:24 -04:00
bvandeusenandClaude Fable 5 a7abcc41ca feat(triage): failed-processing triage — probe errored files, flag defects, recover (#125 C1-C3)
An errored GPU job's stored reason is a suspicion; the file probe is the
verdict. A 15-min beat sweep (triage_gpu_errors) runs verify_integrity's own
probe (sha256 + decode) on each errored image ONCE and writes both verdicts:
ImageRecord.integrity_status and the new GpuJob.triage_status ('defect' |
'file_ok', migration 0072). Every classification logs at WARNING so it
surfaces in Logs/System Activity.

- 'defect' rows are excluded from /retry_errors (re-running a known-bad file
  burns agent time re-minting the tombstone); response now reports
  defects_kept and the GpuAgentCard toast says so.
- GET /api/gpu/errors: triage view — reason buckets (classify_reason),
  probe verdicts, per-job detail. POST /errors/triage runs the sweep now.
- POST /api/gpu/errors/<id>/recover: reuses the Layer-2 refetch pattern —
  delete the defective copy + record (full cascade takes the tombstones too)
  and re-poll its subscription Source so a fresh copy re-imports and re-enters
  the pipeline; 'no_source' when nothing pollable resolves.
- New 'Failed processing' card (GpuTriageCard) in Maintenance: verdict counts,
  reason summary, probe-now, defect list with thumbnails + per-image Recover.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 12:36:02 -04:00
bvandeusenandClaude Fable 5 1f27189b8f chore: retire ml-backfill-daily beat + the spent purge-legacy action (operator-approved)
- ml-backfill-daily: the CPU tag_and_embed backfill raced the GPU agent's
  daily embed backfill for the same NULL-embedding images at ~100x the cost
  (B1 audit verdict, milestone #124). The backfill TASK stays — the manual
  /api/ml/backfill button remains the deliberate CPU fallback pending B3.
- purge-legacy: one-time IR-migration cleanup, dry-run verified 0 targets on
  the live library before removal (A2 audit, milestone #123). Fully retired
  per rule 22: tile, store action, route, service fn, tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-02 11:24:08 -04:00
bvandeusenandClaude Fable 5 09e2772628 fix(gpu-jobs): end the error-tombstone loop — deliberate retry semantics + poison-job guards
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m27s
CI / frontend-build (push) Successful in 18s
The hourly ccip backfill's skip-list lacked 'error' (and the daily
siglip/embed variants re-gated failures on their missing results), so every
permanently-bad file got a fresh doomed job each run — ~24 duplicate error
rows/day per file, the perpetual 'unprocessable' flood. An errored job is now
a TOMBSTONE: no backfill re-enqueues it; retry is deliberate-only via
/retry_errors (an errored back-catalogue needs one button press after a
model swap).

One shared set of dedupe DELETEs (services/ml/gpu_jobs.error_dedupe_statements)
runs before every backfill and inside /retry_errors: error rows made moot by a
later pending/leased/done row go first, then older duplicates (newest reason
survives) — so the error count reads as distinct failing files and a retry
can't fan one file out into duplicate pending jobs. /retry_errors now returns
{requeued, pruned} and the toast shows both.

Poison-loop guards (release and lease-expiry burn no attempts, so a job that
stalls its transfer or crashes the agent every time cycled forever —
operator-observed jobs 99044/125288/131594/143131):
- agent: 3 in-session transient bounces (fetch or submit) → fail with the real
  reason instead of another release; strikes never count while stopping, and
  clear on submit success. Agent build 2026-07-02.3.
- server: the 60s orphan sweep (statements shared between the beat task and
  GpuJobService so they can't drift) converts expired leases with >=5 lease
  grants and pending jobs with >=10 to 'error', preserving the last stored
  failure reason. Backstops old agent builds.

Tests: tombstone rule across all three backfill variants, moot-row pruning,
poison conversions, and the extended /retry_errors dedupe contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-01 22:52:38 -04:00
bvandeusenandClaude Fable 5 3d6201734c fix(settings): maintenance tiles start collapsed; remember manual open state
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
GpuAgentCard was hardcoded :open=true, HeadsCard opened whenever any head
existed, TagEvalCard whenever a persisted run existed — so a fresh Settings
load greeted the operator with several tiles already expanded. All three now
force-open only while their task is actually running (the #877 resurface
behavior on the busy-driven tiles is untouched).

MaintenanceTile additionally persists MANUAL expand/collapse per tile in
localStorage, so the section reloads the way the operator left it; a forced
open while a task runs stays transient and is never saved as a preference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-01 22:28:52 -04:00
bvandeusenandClaude Fable 5 686808d3f3 feat(gpu): "Retry errored jobs" — scoped requeue of errors only
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m26s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
After an agent-side fix (e.g. the short-video sampler), the errored jobs
(~2.8k) have exhausted their 3 attempts and stay parked: backfill skips
images that already have a job, and /reprocess is the nuclear option (it
resets the 179k DONE jobs too). There was no way to re-run just the errors.

POST /api/gpu/retry_errors resets every status='error' job (all task types)
to pending with attempts=0 and the stored error cleared — a small inline
UPDATE that returns {requeued: n} so the UI toast can show the count.

UI: a "Retry errored jobs" button on the GPU-agent card, right under the
queue tiles; disabled when errored==0. With the agent now logging ffmpeg's
stderr on failure, retrying also reveals which errors were real vs victims
of the fps-filter bug.

Test: retry_errors requeues the errored job (fresh attempts, error cleared)
and leaves done work untouched; asserts via column selects (Core-DML
gotcha), not ORM refresh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-01 21:09:07 -04:00
bvandeusenandClaude Opus 4.8 359bc5a283 feat(ml): default to SigLIP 2 (new installs) + model dropdown, no free-text (#1203)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
- Migration 0069: new installs default to SigLIP 2 (so400m, 512px, 1152-d drop-in)
  — UPDATE applies ONLY where no image is embedded yet (fresh install), so an
  existing library is NOT silently invalidated; it switches deliberately via the
  dropdown → Re-embed → Retrain. Column server_defaults moved to SigLIP 2.
- GET /api/ml/embedder-models: server-authoritative supported list (SigLIP 2 512
  recommended / 384 faster / SigLIP 1 384 original) so the UI never free-types.
- GpuAgentCard: the two name/version text fields → a single model dropdown;
  Save sets name+version from the picked option (the current model is always
  selectable even if off-list).
- embedder.py DEFAULT_MODEL_NAME unchanged (stays the baked local-dir SigLIP 1)
  to avoid a local-dir/weights mismatch; SigLIP 2 loads by HF name, cached on the
  ml-worker's persistent HF_HOME.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 16:29:27 -04:00
bvandeusenandClaude Opus 4.8 80f8eb4756 feat(gpu): re-process trigger to apply new crop detectors to the existing library (#1202)
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m25s
CI / lint (push) Successful in 3s
The siglip/ccip backfills skip images that already have current-version regions,
so adding crop detectors only affected NEW images — the back-catalogue would
never be re-cropped. Add a reprocess trigger that resets every done/error job of
a task back to pending, so the agent re-runs the FULL pipeline (figure detection
+ CCIP + concept/panel crops) over the whole library under the current detectors.

- reprocess_gpu_jobs(task='ccip') task + POST /api/gpu/reprocess.
- gpu store reprocess() + GpuAgentCard "Re-process library (re-detect + re-crop)"
  button with a confirm (it's heavy).
- Test: a done job resets to pending (attempts cleared).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 16:09:37 -04:00
bvandeusenandClaude Opus 4.8 bc6d43d3f2 refactor(ml): drop dead tagger/suggestion settings + columns (#1199)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m31s
Hygiene follow-up to the Camie retirement (#1189) — these were left inert to
bound that change; nothing reads them now. Migration 0068 drops:
- ml_settings: tagger_store_floor, tagger_model_version, suggestion_threshold_
  character/general (already dead pre-retirement — scoring uses per-head
  thresholds), video_min_tag_frames (only the deleted video-prediction
  aggregator used it).
- image_record: tagger_model_version (no writer), centroid_scores (dead JSON
  cache, no reader).

Also: ml_admin _EDITABLE/GET/_validate pruned (dropped the store-floor invariant
+ video_min_tag_frames check); MLThresholdSliders trimmed to a video-embedding
card (interval + max frames only); importer no longer resets the dropped cols;
download_models drops the Camie fetch; stale CASCADE comments in cleanup_service
no longer name the removed tables. Tests updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 13:41:25 -04:00
bvandeusenandClaude Opus 4.8 485387ff0b refactor(ml): retire the Camie tagger + allowlist bulk-apply (#1189)
CI / lint (push) Failing after 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m27s
Heads + CCIP are the tag source and head auto-apply is the earned propagation.
The Camie tagger ran only to feed the allowlist bulk-apply (its ImagePrediction
rows had no other consumer), and the allowlist was a SECOND, un-earned auto-apply
path firing in parallel with heads on every accept — exactly the un-earned spray
the v2 pivot replaced. Retire both.

Behavior change: accepting a suggestion now applies the tag to THAT image only
(source='ml_accepted', a head-training positive) — it no longer allowlists +
fans the tag across the library via Camie. Propagation is heads' earned
auto-apply. (Loses instant cold-start propagation for booru-vocab tags; that was
un-earned and bypassed the precision gate.)

- tag_and_embed is now EMBED-ONLY (no Camie load/infer, no ImagePrediction
  writes); backfill enqueues it for images with no embedding.
- Removed: services/ml/tagger.py, apply_allowlist_tags + helpers + daily beat +
  every enqueue caller (accept/alias/merge/per-image), api/allowlist.py +
  blueprint, ImagePrediction + TagAllowlist models/tables (migration 0067),
  AllowlistTable.vue + allowlist store, the accept coverage-projection payload.
- AllowlistService gutted to accept/dismiss/undismiss/reject (the rejection store
  the rail still needs); accept returns nothing, API returns {accepted, tag_id}.
- tag merge no longer repoints/triggers the allowlist; _keep_as_alias now keys on
  ML-applied image_tag sources (incl. head_auto) instead of the allowlist.
- UI: MLBackfillCard relabelled to embedding-only; accept toast simplified;
  MaintenancePanel drops the allowlist tile.

Left for a follow-up hygiene pass (now-inert, harmless): the dead settings
columns (tagger_store_floor, tagger_model_version, suggestion_threshold_*,
video_min_tag_frames), image_record.tagger_model_version, MLThresholdSliders
trim, and the Camie model download in download_models.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 13:04:31 -04:00
bvandeusenandClaude Opus 4.8 3d77a38a25 refactor(ml): remove the dead per-tag centroid subsystem (#1189)
CI / backend-lint-and-test (push) Successful in 26s
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / integration (push) Successful in 3m32s
The v2 pivot replaced per-tag SigLIP centroids with learned heads + CCIP.
Centroids were still recomputed (on every tag merge + a daily beat) but NOTHING
read them — suggestions come from heads+CCIP and apply_allowlist_tags applies
via Camie predictions, not centroids. Pure dead wiring; remove it.

Removed: CentroidService, recompute_centroid/recompute_centroids tasks, the
daily beat, POST /api/ml/recompute-centroids, the recompute-on-merge trigger,
the tag_reference_embedding table + model, the centroid_similarity_threshold +
min_reference_images settings (migration 0066), the CentroidRecomputeCard +
its store action + MaintenancePanel tile, and the centroid slider in
MLThresholdSliders. _keep_as_alias drops its vestigial has-centroid branch (the
allowlist branch already covers "could re-emit"); tag merge no longer clears a
table that no longer exists.

NOT touched (still live, parallel to heads): the Camie tagger, ImagePrediction,
and the allowlist bulk-apply — accepting a suggestion still allowlists + applies
it across the library. The tag-eval "centroid" baseline metric is unrelated
(in-memory) and stays. (image_record.centroid_scores JSON column also remains —
separate legacy field, its own micro-cleanup.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 11:48:09 -04:00
bvandeusenandClaude Opus 4.8 4daa3f2790 feat(ml): operator model swap — GPU re-embed + embedder as a setting (#1190)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m33s
Make the SigLIP embedder an operator choice (drop-in to SigLIP 2:
google/siglip2-so400m-patch16-512 is a verified 1152-d model at 512px → no
schema change, better small-cue fidelity). A swap = set model + re-embed +
retrain, all operator-driven; the GPU agent does the re-embed so it's fast.

- settings: embedder_model_name is now a setting (migration 0065) alongside the
  existing embedder_model_version; both editable + validated (non-empty) in the
  ml admin API. The server embedder loads by HF name (AutoImageProcessor/Model,
  model-agnostic), preferring the pre-downloaded local dir for the default so
  existing deploys don't re-download; rebuilds on a name change.
- agent: new 'embed' job = whole-image SigLIP embedding (mean-pool video frames)
  under the lease-announced model → POST /jobs/submit_embedding writes
  image_record.siglip_embedding + siglip_model_version. The lease now announces
  the model FROM THE SETTING (not a constant).
- re-embed routing: enqueue_gpu_backfill('embed') selects unembedded + stale-
  version images; 'siglip' now re-embeds concept crops whose version != current
  (so a swap re-triggers crops, not just the never-embedded back-catalogue). The
  CPU ml-worker backfill no longer re-embeds on a version mismatch (it can't
  churn the library at 512px) — the GPU agent owns version re-embeds. Daily
  'embed' + 'siglip' beats self-heal.
- scoring: score_image only bags embeddings in the CURRENT model's space (whole-
  image gated by siglip_model_version, concept regions by embedding_version) so a
  mid-swap stale vector isn't scored by new-space heads; legacy NULL = current.
- UI: GpuAgentCard "Embedding model (advanced)" — edit name/version, Save, and
  "Re-embed library (GPU)" (queues embed + siglip); points at SigLIP 2.

Tests: lease announces model + submit_embedding round-trip; enqueue 'embed'
selects stale/unembedded; stale-version excluded from scoring; embedder model
settable + empty rejected; siglip gate updated to current-version concept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 10:24:30 -04:00
bvandeusenandClaude Opus 4.8 c6f38b0dac feat(tagging): SigLIP concept crops + max-over-bag scoring (#114)
Lift recall on small/local concepts (glasses, cum, stomach-bulge, xray,
lactation) that the whole-image SigLIP vector washes out: the GPU agent now
embeds figure crops with SigLIP too, stored as kind='concept' regions, and the
suggestion rail scores each image as a BAG (whole-image + every concept crop),
taking each head's MAX over the bag. The whole-image vector is always in the
bag, so this can never score lower than before.

Model-agnostic by construction: the server ANNOUNCES the embedding model
(HF name + version) in the lease, so the agent loads whatever the heads were
trained in and stays in lock-step — a model swap is a server setting + a
re-embed migration, never an agent change.

- agent: model-agnostic CropEmbedder (torch/transformers get_image_features,
  fp16 on CUDA, inference-locked); worker branches on job.task — 'ccip' emits
  figure(CCIP)+concept(SigLIP) in one pass, 'siglip' emits concept-only so the
  back-catalogue backfill never churns figure/CCIP regions; torch cu124 +
  transformers in the image.
- server: lease announces embed_model_name/embed_version; score_image is
  max-over-bag (version-filtered region embeddings); enqueue_gpu_backfill
  'siglip' gates on a missing concept region (drains the back-catalogue,
  retries failures, no double-enqueue); daily siglip-backfill beat; UI button;
  /api/ccip/overview reports images_with_concept_siglip.
- v1 scope: suggestion rail only — auto-apply stays whole-image (conservative;
  heads' thresholds were calibrated on whole-image). Bulk-apply bag = follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-30 08:17:47 -04:00
bvandeusenandClaude Opus 4.8 b91a230f12 feat(ccip): automation + reference quality — keep identity flowing hands-free (#114)
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m32s
CI / frontend-build (push) Successful in 19s
Works through the optional CCIP ideas + the "keep moving even if I forget" ask:

AUTOMATION (no button needed):
- Hourly beat auto-enqueues CCIP backfill — new images get embedded (and errored
  ones retried) on their own; the queue never goes idle waiting for a click.
- CCIP auto-apply: a daily sweep tags confident matches (source='ccip_auto') so
  identity tags keep flowing. ON by default (opt-out, like head auto-apply);
  ml_settings.ccip_auto_apply_enabled + _threshold (0.92, above the suggest cut),
  migration 0064. Vectorized (one matmul + reduceat per image), reversible, skips
  already-applied/rejected. Switch + threshold in the GPU agent card; GET/PATCH
  /api/ml/settings; auto_applied count in /api/ccip/overview.

REFERENCE QUALITY (the over-fire root cause):
- character_references now draws ONLY from single-character images — on a
  multi-character image the tag is image-level, so every figure would otherwise
  pollute each character's prototypes (a 2-char image tagged 'Velma' made
  Daphne's figure a Velma reference). This is the contamination behind residual
  over-firing.
- Cached on a cheap signature (char-tag count + ccip-region count/max-id) so the
  reference load isn't redone on every modal open.

Tests: multi-character image not used as a reference; auto-apply tags a confident
match as ccip_auto.

NEXT (not done, confirmed): comic-panel cropping + SigLIP concept crops ("spot
interesting content").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-29 22:25:40 -04:00
bvandeusenandClaude Opus 4.8 625336b6b4 feat(ccip): tunable match threshold, default 0.85 (#114)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m28s
Live data showed the v1 flat 0.75 cosine over-fired — ~64% of matched images got
3-10 character guesses dominated by the most-referenced characters (a 27-ref
character clears a low bar on many images). A sweep showed 0.85 collapses the
noise (noisy multi-matches 47→3) while keeping the confident single-character
matches.

- ml_settings.ccip_match_threshold (migration 0063, default 0.85); match_image
  reads it (override still accepted). DEFAULT_SIM_THRESHOLD fallback 0.75→0.85.
- Exposed in GET/PATCH /api/ml/settings (validated 0.5–0.999).
- Slider in the GPU agent card ("Character-match strictness") — tune live, no
  redeploy, same observe-and-tune loop as auto-apply.

Test: a ~0.9-cosine figure matches at 0.85, dropped at 0.95.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-29 20:41:09 -04:00
bvandeusenandClaude Opus 4.8 d91eef7a4b feat(gpu): GPU agent admin card — token, queue, backfill (#114)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m25s
The FC-side control surface the operator asked for: Settings → Tagging → "GPU
agent". Generate/reveal/copy/rotate the agent bearer token (with the FC URL to
point the agent at), see the live job-queue depth (pending/in-flight/done/
errored, polled), and a "Queue character embedding (CCIP)" button that triggers
the library backfill. Plain-HTTP-safe copy (copyText resolves on success,
throws on fail). Closes the "how do I get the token in the UI" gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-29 11:53:46 -04:00
bvandeusenandClaude Opus 4.8 1463794778 feat(heads): auto-apply UI on the Concept-heads card (#114 auto-apply C)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m23s
Surfaces earned auto-apply + its observability in Settings → Tagging → Concept
heads:
- Auto-apply section: an on/off switch (writes head_auto_apply_enabled), the
  precision-target + min-examples-to-fire tuning inputs, a Preview (dry-run →
  "would apply N", per-concept chips) and Apply-now button, with live run state.
- "How auto-apply is landing": per-concept table from /api/heads/metrics —
  applied volume, misfires, realized misfire rate (green/amber/red), and missed
  (under-fires) — the signal to tune the precision target from.

store: autoApply(dryRun) / autoApplyStatus() / metrics(). Card polls the sweep
to completion, then refreshes counts + metrics. Completes the auto-apply task.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-29 00:51:59 -04:00
bvandeusenandClaude Opus 4.8 06d5e83da4 feat(heads): admin card to train + inspect concept heads (#114 B)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m20s
The UI for the heads subsystem: Settings → Tagging → "Concept heads". Shows
head count, auto-apply-ready count, and last-trained; a Train/Retrain button
(one run at a time, polls while running, surfaces a failed run's error); an
empty state guiding the operator to tag first; and a per-concept table (name,
category, +tags, AP, P, R, auto-apply ) sorted strongest-first so weak/under-
tagged concepts are obvious. Rehydrates status from GET /api/heads on mount so
it survives navigation. Pulls head_min_positives from ML settings for copy.

Slice C (swap the rail's suggestions to heads, remove Camie + centroid) is next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
2026-06-28 10:44:35 -04:00
bvandeusenandClaude Opus 4.8 b69c70ab2b feat(tag-eval): "keep" records a confirmation so doubts stop resurfacing
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m23s
"Keep" on a doubted positive was a no-op, so the same confirmed-correct images
came back in "head doubts" every run (operator-flagged: reinforcement keeps
surfacing the same images). Add tag_positive_confirmation (mirror of
tag_suggestion_rejection): keep → POST /images/<id>/tags/<tag_id>/confirm, and
the eval excludes confirmed positives from the doubts list — exactly as rejected
items already drop out of the suggest list. The tag stays a positive either way
(confirmation is a "reviewed" marker, not a training change).

- model TagPositiveConfirmation + migration 0057; confirm endpoint (idempotent).
- tag_eval: _confirmed_ids + exclude from head_doubts_positive examples.
- store.confirmTag + card "keep" calls it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 01:32:20 -04:00
bvandeusenandClaude Opus 4.8 5143f4c34f feat(tag-eval): auto-apply operating point + server-side top-N concept discovery
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m24s
Two additions driven by "what's the commit threshold?" + "find more tags":

1. High-precision operating point (Bar 4). Per concept, report the threshold that
   maximizes recall while holding precision >= a target (default 0.97, configurable
   via `precision_target`) — i.e. "could this fire without a human, and how much
   would it catch?" `head.auto_apply` = {target, threshold, precision, recall} or
   null if the target is unreachable. Surfaced on the card.

2. Server-side concept auto-discovery. `auto_top_n` param unions the explicit
   concept list with the N most-tagged general tags (one fast DB query) so the
   eval can broaden itself without hand-listing — replaces the slow HTTP directory
   paging. Card gains "+ auto-add top-N" and precision-target inputs.

No migration; numpy/sklearn stay lazy. Existing _normalize_params test still
holds (new keys additive; None still falls back to DEFAULT_CONCEPTS).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 00:50:28 -04:00
bvandeusenandClaude Opus 4.8 fc64f130b8 fix(tag-eval): thumbnail click opens the view modal, not Explore
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Successful in 3m17s
Clicking an example in the maintenance card navigated to /explore/<id> —
heavier than wanted (operator: just want a bigger look). Open the existing
app-wide ImageViewer modal via modal.open(id) instead: bigger image + tags
in place, no navigation away from Settings. The ✓/✗ actions are unaffected
(separate overlay buttons).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 00:07:38 -04:00
bvandeusenandClaude Opus 4.8 13d297b881 feat(tag-eval): inline confirm/reject actions on example thumbnails
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m22s
Closes the learn-from-tags loop directly on the eval lists (operator-flagged:
no surface to confirm/refine the head's suggestions). Each thumbnail gets a
green ✓ / red ✗ that writes the SAME tables the head trains on:
- suggest + ✓  → apply tag (new positive, POST /images/<id>/tags)
- suggest + ✗  → record rejection (hard negative, suggestions/dismiss)
- doubt   + ✗  → remove tag + record rejection (kill bad positive, add negative)
- doubt   + ✓  → keep (stays a positive, no write)
Acted thumbs grey out with a badge; re-run to see the head sharpen. Thumb still
links to /explore/<id>. All endpoints already existed — no backend change.

Inline is the starting point; longer-term the modal Suggestions rail gets the
red "No" (negative) so per-image rejection is native there too (next slice).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:37:21 -04:00
bvandeusenandClaude Opus 4.8 4974b7cf77 feat(tag-eval): bigger, clickable example thumbnails (label-review queue)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m24s
The 56px example thumbs were too small to judge a label (operator-flagged).
Bump to 120px and wrap each in a link to /explore/<id> (new tab) so the
"head doubts / would suggest" galleries double as a review-and-fix queue —
click a doubted positive, land on it in Explore, correct the tag, re-run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:19:58 -04:00
bvandeusenandClaude Opus 4.8 6cd7281af5 feat(settings): tag-eval admin card — trigger + persisted report (survives nav)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m19s
Frontend for #1130. A maintenance tile in Settings → Tagging:
- Editable concept list + "Run eval" → POST /api/tag-eval (one running at a time).
- Rehydrates on mount via the persisted run (getRun by latest id) and polls while
  running — so the report SURVIVES navigation (operator-flagged); the task runs
  backend-side regardless and the card reconnects to its row.
- Renders the saved report: per-concept head-vs-centroid metrics table (AP/F1/
  precision/recall) with Δ AP, the learning curve (AP @ N positives), and
  thumbnail galleries (head-would-suggest / head-doubts-positive) for eyeballing.

Backend: _examples now stores thumbnail_urls (not just ids) so the report is a
self-contained artifact that renders without per-id lookups on reload.

No new top-level surface — slots into the existing maintenance area.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:56:41 -04:00
bvandeusenandClaude Opus 4.8 958378312c fix(settings): sticky headers on the virtual data tables
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m27s
Allowlist / Alias / ImportTask tables scroll their bodies (height=360/480) but
the column headers scrolled away with the rows, so you lost the column labels
(operator-flagged 2026-06-27). Add Vuetify `fixed-header` so the header row
stays pinned while the body scrolls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:54:03 -04:00
bvandeusenandClaude Opus 4.8 e49cea3eba feat(tagging): allowlist tuning dashboard + post-accept toast + merge preview UI (#7c/#7d/#8b)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 16s
CI / backend-lint-and-test (push) Successful in 25s
CI / integration (push) Failing after 3m17s
Cluster B frontend, milestone #99.

#7c: AllowlistTable gains Applied + Covers columns and a live 'covers ~N at T'
projection as the operator drags a row's threshold (debounced coverage call,
then commits the threshold). allowlist store gains coverage(tagId, threshold)
and refreshes coverage_count after a save.

#7d: suggestions store surfaces a non-blocking toast when accept/alias newly
allowlists a tag — '<verb>: <tag> — allowlisted, auto-applying to ~N images'
(N is the projection; apply runs async). Falls back to the plain toast when
the tag was already allowlisted.

#8b: TagsView merge picker now previews the merge via usePreviewCommit before
committing — shows images moving / already-on-target / series pages / alias-or-
delete / a thumbnail sample, blocks the Merge button on an incompatible
kind/fandom. adminStore.mergeTags gains a dryRun option.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XCUHUGQLrBrkgyk1t49kpX
2026-06-23 01:41:09 -04:00
bvandeusenandClaude Opus 4.8 7c94d99b9f refactor(settings): canonical usePreviewCommit for maintenance preview→commit tiles (#753)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m16s
Frontend pattern-consistency sweep (note #1026, the last DRY-thread item).
TagMaintenanceCard (4 flows) + PostMaintenanceCard (2 flows) each hand-rolled the
same sync preview→commit state machine: a previewData/previewing/committing
triple + onPreview/onCommit that dry-run-previews, then applies and collapses the
projection (the apply shares the backend predicate, so afterward it's empty).

Extract usePreviewCommit({preview, commit, emptyPreview}) owning that lifecycle.
The 6 flows become declarative: supply the two thunks + the collapse shape. The
normalize flow (commit dispatches a self-resuming background task, not a sync
apply) omits emptyPreview so the projection stays and a truthy result = queued.

Composable returns are aliased to the cards' existing local names, so the
templates only change where they read the apply result (the success badges).
Long-Celery-task cards (GatedPurge/VideoDedup) keep useMaintenanceTask — a
different pattern (navigable-away task lifecycle), deliberately not merged.

Exhaustiveness: no card hand-rolls the refs anymore; the only dryRun:false
callers are these two cards, both via the composable. Added a vitest spec for
the primitive (collapse static + fn, dispatch-variant, re-preview clears result).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 19:39:56 -04:00
bvandeusenandClaude Opus 4.8 311fe0ee9c feat(settings): tidy Maintenance tab into compact tiles + center the views (pass 2)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 42s
CI / integration (push) Successful in 3m27s
Goal (operator 2026-06-18): the overview of a Settings tab fits one unscrolled
viewport; expanding a tile to read into it is the only reason to scroll.

- Every Maintenance card converted to the collapsible MaintenanceTile (collapsed
  by default = icon + short title + one-line blurb). Task cards (ML backfill,
  centroids, thumbnails, archive re-extract, missing-file repair, DB maintenance)
  sit in a responsive grid; running tasks auto-expand. Tagging config (suggestion
  thresholds, allowlist, aliases) grouped in one Tagging section as collapsible
  tiles; Backup is its own collapsible tile.
- Three labeled sections mirror the Cleanup tab: Backfills and reprocessing /
  Tagging / Storage.
- Center the whole Settings surface: SettingsView is now a centered, width-capped
  (1140px) column so the tab strip and every panel sit in a tidy centered measure
  (was full-width). CleanupView drops its own left-aligned max-width to fill it.

All card logic unchanged - only the chrome.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 00:10:58 -04:00