Commit Graph

1038 Commits

Author SHA1 Message Date
bvandeusen 40cc11be5b feat(deploy): container healthchecks + Swarm rolling-update auto-rollback
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m45s
web gets a /api/health liveness check; workers a lenient celery-ping check. A
shared deploy policy (update_config order=start-first, failure_action=rollback,
monitor 90s; rollback_config; restart_policy) means a bad image that never goes
healthy is rolled back automatically instead of taking the service down. Ignored
by plain `docker compose up` (deploy: is swarm-only), so the dev override is
unaffected. Assumes prod deploys from this file via docker stack deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 0b78264d62 feat(maintenance): daily janitor for orphaned .part/.partial staging files
Downloads/imports stage into <name>.part / <name>.partial then os.replace() into
place, so a kill mid-write leaves a discardable temp — never a corrupt final.
cleanup_orphaned_temp_files sweeps ones left behind under the images root, only
older than 6h so an in-flight download's staging file is never removed. Daily beat.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 9eae636047 feat(translation): pooled Interpreter session + manual sweep resumes after drain
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m45s
- interpreter_client: shared requests.Session with a connect-only retry
  (connect=2, no status retries — we map 429/5xx ourselves) so a proxy reload
  is smoothed and the keep-alive connection is pooled across the sweep.
- translate_posts: on an interrupt (drain), re-enqueue after the Retry-After
  hint / default backoff instead of waiting for the daily beat; self-terminates
  via the health gate. Steady-state one-chunk-per-run on success is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen f8105046dc feat(explore): exclude WIP-tagged work from the Explore rabbit-hole
Explore's neighbour grid (/api/gallery/similar → gallery_service.similar) now
takes an Explore-only exclude_wip flag that drops `wip` system-tagged images
from the candidates, alongside the banner/editor presentation tags. The
gallery's own "similar" button is unchanged (keeps wip, #1274) — only the
Explore store passes exclude_wip=1. The anchor itself may still be a WIP; only
neighbours are filtered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen d631ed023c style(translation): use datetime.UTC alias (ruff UP017)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:04:17 -04:00
bvandeusen c64261593d feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m42s
Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs
mid-flight. Give in-flight work room to drain and make interrupted work
resume cleanly instead of stalling.

- docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s /
  scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown
  can actually drain before SIGKILL.
- celery_app.py: task_reject_on_worker_lost=True — a task killed past the
  grace window is re-queued (safe: idempotent + chunked, recovery sweeps
  re-drive stragglers).
- interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable
  and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter
  behind a reverse proxy no longer raises an opaque HTTPError.
- translation.py: thread retry_after out of _translate_batch; retranslate_posts
  resumes after the Retry-After hint (or 60s default, capped 900s) on an
  interrupt with _reset_done=True, self-terminating via the health gate.
- tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff.

No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:01:00 -04:00
bvandeusen 1f6d94f51d feat(translation): re-translate on model change — artist-scoped + global re-run (#146)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
retranslate_posts resets the 5 translation columns to NULL for a scoped set of posts (all, or WHERE artist_id IN ids) then reuses the untranslated sweep to re-run them, chasing the tail until drained (run-until-done). Interpreter cache keys on engine_version so a changed model re-translates, an unchanged one is cache-fast. Reset only happens when the service is configured+healthy so translations are never wiped when they can't be rebuilt. New POST /settings/translation/retranslate (artist_id | all=true). UI: per-artist 'Re-translate posts' on the Artist Management tab + 'Re-translate all' in the Settings Translation card, both with confirm dialogs. No migration (reuses m143 columns).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:03:19 -04:00
bvandeusen 6a255482ea feat(translation): "Test connection" button — on-demand Interpreter health check (#143)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m43s
New POST /api/settings/translation/test pings /v1/health for a GIVEN base URL (not
the saved one), so the operator can verify a URL before enabling it. TranslationCard
gains a Test-connection button that reports reachable/unreachable inline and
updates the status dot. Endpoint test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 13:00:37 -04:00
bvandeusen af5aa21e45 test(translation): API endpoint tests for status + run (#143 step 6)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m47s
/settings/translation/status defaults (off → no health call) + /run 400-when-
unconfigured + 202-when-configured (monkeypatched .delay). requests is already a
backend dep, so no requirements/ci-requirements change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:39:43 -04:00
bvandeusen 83c1745fd0 feat(ui): translation-forward post text + Settings card (#143 step 5)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m45s
PostCard + modal ProvenancePanel show the English title/description by default when
a translation exists, with a per-card "show original (<lang>)" toggle — translated
bodies render as plain text, originals keep their sanitized HTML. New
TranslationCard in Settings → Ingestion & filters: enable switch, Interpreter base
URL (generic placeholder, no default host), target language, a reachability
indicator + untranslated-posts count + "Translate now".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:38:41 -04:00
bvandeusen ead60978e3 feat(translation): expose translated fields in post feed + provenance payloads (#143 step 4)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m44s
post_feed_service (card + detail) and provenance_service._post_dict now include
post_title_translated, description_translated (card-truncated / detail-uncapped)
and translated_source_lang, keeping the originals for the toggle. Feed
serialization test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:31:46 -04:00
bvandeusen 7a4de7278d feat(translation): backfill sweep + beat + manual trigger (#143 step 3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m49s
tasks/translation.py — translate_posts: picks untranslated posts (title OR
description non-empty), per-post [title, description] batch via the Interpreter
client, stores translations + detected lang + engine_version; passthrough /
already-target posts are marked handled with no stored translation. 503 or a
connection error interrupts (retry next cycle), 400 stops (fix config), per-post
commit keeps progress; wall-clock bounded. Wired into celery (maintenance_long
lane) + a daily beat. No-op unless enabled + base URL set + healthy. GET
/settings/translation/status + POST .../run for the Settings card. Task tests
(stubbed client, monkeypatched session).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:29:28 -04:00
bvandeusen 7f8073c4c8 feat(translation): Interpreter client — batch translate + health (#143 step 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m43s
services/interpreter_client.py: sync (requests) client for the LibreTranslate-
compatible /v1/translate — no new dep, mirrors the platform clients. translate()
maps translatedText[]↔texts (order + length), returns detected_lang +
engine_version (aggregate = first item, fine for a per-post [title, description]
batch); passthrough items come back unchanged in their slot. InterpreterUnavailable
on 503 / connection error (retry later), InterpreterBadRequest on 400. health()
checks /v1/health engines.llm. 10 unit tests with mocked HTTP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:23:49 -04:00
bvandeusen a3bc98a53c feat(translation): Post translation columns + settings + migration (#143 step 1)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m46s
Post gains post_title_translated / description_translated / translated_source_lang
/ translation_engine_version / translated_at — filled by the translate sweep so
viewing is instant. ImportSettings gains translation_enabled (OFF by default),
interpreter_base_url (EMPTY — no default host; the operator points it at their own
Interpreter proxy behind a reverse proxy) and translation_target_lang (en),
exposed + validated via /settings/import. Migration 0083. Settings defaults +
patch + validation test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:20:31 -04:00
bvandeusen e7c3f4e9c9 feat(ui): auto-tag accept/reject as an in-pill yes/no pair; confirm refocuses input
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m38s
Fold the auto-tag accept into the chip: a provisional auto-tag now shows a compact
green ✓ / red ✗ pair IN the pill (replacing the ✕), and the "auto" text label is
dropped — the yes/no is signal enough (operator-asked). ✓ confirms (trains +
shields from retraction), ✗ removes (records a negative). The name still
ellipsis-truncates so the pair stays reachable.

onConfirm now returns focus to the tag input like onRemove already does, so the
input is the cursor's resting position after any chip action in both the modal
and Explore views.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 00:56:15 -04:00
bvandeusen 29f3a485b0 fix(ui): surface auto-hide misfires proactively — review strip no longer gated on Show-hidden (#141)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m44s
The conflict-flag review strip only appeared when "Show hidden" was toggled on, so
misfires could go unnoticed — defeating the point of flagging them. Fetch pending
flags on mount and show the strip whenever there are any, independent of the
toggle (operator-flagged). Gated to the main gallery (not the post-detail view),
matching the filter bar.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 00:39:23 -04:00
bvandeusen 2bcaa20b22 feat(ml): schedule presentation auto-hide sweep + retention (#141 step 6)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m41s
scheduled_presentation_auto_apply (daily beat) runs presentation_auto_apply_sweep
— idempotent, so an interrupted run just re-runs next cycle (that's the recovery),
wall-clock bounded by soft/hard task time limits. prune_presentation_reviews
(daily beat) drops RESOLVED review flags older than 30 days (rule 89 retention).
Tests run both tasks via a monkeypatched session factory. Milestone 141 complete:
the presentation-chrome auto-hide + conflict-flagged review is now live end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:46:09 -04:00
bvandeusen 9726d6f4b5 feat(ui): hidden-view review strip — flagged auto-hides with keep / un-hide (#141 step 5)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m40s
When "Show hidden" is on, a review strip appears atop the gallery listing the
auto-hidden chrome flagged "also looks like content" (most-concerning first):
thumbnail + "also looks like <X>" + Keep hidden / Un-hide. Un-hide removes the
presentation tag (image returns to the gallery) and trains the head; Keep
resolves the flag. Self-hides when there's nothing to review; theme-token styled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:39:02 -04:00
bvandeusen 6c34f86477 feat(gallery): hidden-view review endpoints — list + keep + un-hide (#141 step 5)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m46s
GET /api/gallery/hidden-review lists unresolved presentation auto-hide flags
(image + presentation tag + conflict tag/score), most-concerning first. POST
.../keep resolves the flag (the tag stays). POST .../unhide removes the
presentation tag (image returns to the gallery), records a TagSuggestionRejection
so the head learns it misfired, and resolves the flag. Tests for list/keep/unhide.
Frontend review strip (shown when Show-hidden is on) next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:34:18 -04:00
bvandeusen 1f548d8a7b feat(ui): presentation-chrome auto-hide Settings controls (#141 step 4)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
HeadsCard gains a "Hide presentation chrome" section: on/off switch + "Hide
confidence" (presentation_auto_apply_threshold) + "Flag if content ≥"
(presentation_conflict_threshold), wired to MLSettings via patchSettings and
loaded on mount. Makes the step-4 sweep's thresholds operator-tunable
(config-in-UI). wip is called out as never auto-hidden.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:16:41 -04:00
bvandeusen eedf8d109a feat(ml): presentation-chrome auto-hide sweep + hard-skip + conflict flagging (#141 step 4)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m39s
presentation_auto_apply_sweep fires banner/editor-screenshot heads at the FLAT
presentation threshold (source=presentation_auto). Two guards: (1) hard-skip any
image already carrying a human/confirmed content tag — you valued it, so the model
can't bury it; (2) if an auto-hide ALSO scores >= presentation_conflict_threshold
on a content head, hide it but record a PresentationReview row (conflict tag +
score) for the Hidden view.

_auto_apply_heads now excludes system tags, so a graduated wip/banner can't fire
via the content path (and wip never auto-applies at all). presentation_auto added
to _AUTO_SOURCES so auto-hidden chrome never self-trains. Tests: applies,
hard-skip valued, conflict-flag, disabled no-op, ignores wip, content-path
excludes system. Settings UI + scheduling land next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:11:26 -04:00
bvandeusen ab63d94249 feat(ml): presentation auto-hide settings + review table (#141 step 3)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m47s
MLSettings gains presentation_auto_apply_enabled / _threshold (default 0.90) +
presentation_conflict_threshold (default 0.50): banner/editor auto-hide with a
FLAT threshold (decoupled from content-head graduation), plus the "also looks
like content" conflict cut. New presentation_review table (image, presentation
tag, conflict tag + score, created/resolved_at) records auto-hides flagged for
review. Migration 0082 (columns + table), ml_admin API (editable + get_settings
+ _validate bounds), settings roundtrip/bounds test. The sweep that reads these
knobs + the Settings UI land in step 4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:59:00 -04:00
bvandeusen eadaa716af feat(gallery): "Show hidden" toggle reveals presentation chrome (#141 step 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m44s
A Curation-group chip in the facet panel flips include_hidden, threaded through
the gallery filter store (default model, activeFilterParam, applyFilterFromQuery,
cloneFilter, filterToQuery) and counted in the refine badge. Off by default → the
gallery hides banner/editor-screenshot chrome; on → it's revealed. Backend
already honors include_hidden (step 1). The dedicated conflict-flagged review
surface (only the set-aside items) lands in step 5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:43:24 -04:00
bvandeusen 0efb187eb1 fix(gallery): drop include_hidden before the similar() call (#141 step 1)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m44s
The /similar route splats **filters into similar(), which doesn't take the new
include_hidden kwarg → TypeError → 500 (test_gallery_similar). Drop it like
post_id; similar() has its own presentation exclusion (#1274), so the
gallery-browse flag doesn't apply there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:32:53 -04:00
bvandeusen e86b91dfe2 feat(gallery): default-hide presentation chrome (banner/editor screenshot) (#141 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Failing after 3m41s
The default gallery + facets now implicitly exclude images carrying a
presentation system tag (banner / editor screenshot), reusing the tag-scope
EXISTS machinery. Suppressed when the operator explicitly filters FOR a
presentation tag OR passes include_hidden (the Hidden view — step 2). `wip` is
NOT hidden (real, in-progress art). include_hidden threaded through
scroll/timeline/jump_cursor/facets + the gallery API _parse_filters. Test covers
default-hide, include_hidden, explicit-filter-shows, and wip-stays-visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:27:00 -04:00
bvandeusen 4f4ddecf75 fix(ui): long tag chips no longer overflow the rail and clip their ✕
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m42s
A character+fandom(+AUTO) chip could exceed the tag-panel width, pushing the
close ✕ off the right edge so the tag couldn't be removed (operator-flagged with
a "Mirko - Rumi Usagiyama → My Hero Academia AUTO" screenshot). Cap the chip at
100% of the rail and make the NAME the elastic part (ellipsis-truncates), so the
✕, fandom, and AUTO badge stay reachable; the full name stays on the hover title.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:22:21 -04:00
bvandeusen 04d5d62cfe style(ui): auto-tag Keep button matches the suggestion accept button
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 45s
CI / integration (push) Successful in 3m48s
The ✓ Keep on a provisional auto-tag chip was a smaller tinted-outline variant;
make it the SAME filled green circle (white ✓, 26px, opacity 0.9→1 + scale on
hover, accent focus ring) as the suggestion accept button (.fc-act--yes in
SuggestionItem) so "accept this tag" reads identically across surfaces
(operator-asked).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 21:36:19 -04:00
bvandeusen 18bb25f140 fix: ruff C416 (dict() over comprehension) + frontend test playlistIds rename
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m41s
- heads.py: conf_map = dict(conf) instead of a dict comprehension (ruff C416).
- postCard.spec.js: the modal-playlist rename (postImageIds→playlistIds) missed
  this frontend test (grep was src-only); update the expected call args.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:09:29 -04:00
bvandeusen 17433c69d4 test(tag): serialize_tag unit tests include the new source/confirmed keys (m139)
CI / lint (push) Failing after 3s
CI / frontend-build (push) Failing after 17s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m46s
The confirm-UI change added source + confirmed to serialize_tag; two exact-dict
unit tests in test_tag_query.py failed on the new keys. Add them (default
None/False for rows without image scope).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:03:51 -04:00
bvandeusen 3bf41ecac3 feat(ui): modal prev/next walks the current gallery filter (#1322)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Failing after 17s
CI / backend-lint-and-test (push) Failing after 27s
CI / integration (push) Successful in 3m43s
The image modal cycled GLOBAL neighbours; now the gallery hands it a snapshot of
the currently-filtered, ordered id list so prev/next moves through exactly what
you're viewing — the filtered-playlist behaviour lost in the ImageRepo→FC move.
Generalized the modal store's post-scoped cycle into a `playlistIds` playlist
reused by both GalleryView and PostCard (falls back to global neighbours when no
playlist is passed, e.g. Explore's "open full viewer").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:02:38 -04:00
bvandeusen bae077e323 feat(ml): CCIP references exclude unconfirmed auto character tags + confirm trips detectors (m139)
CI / lint (push) Failing after 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Failing after 42s
CI / integration (push) Successful in 3m46s
Completes "no self-training": unconfirmed auto-applied character tags no longer
seed CCIP references — character_references + the prototype builder
(_current_fingerprints/_rebuild_one) gain a shared _positive_char_tag filter
(human-applied OR operator-confirmed), mirroring the head-positive exclusion.

Confirming a tag also has to move the change-detectors, or an incremental
refresh/Retrain right after a confirm wouldn't fold the tag in (only the nightly
full pass would): the CCIP global gate now counts character confirmations, and
the head training fingerprint counts confirmations. Test for the CCIP path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:59:00 -04:00
bvandeusen 775941609d feat(ui): confirm/keep auto-applied tags (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Failing after 28s
CI / integration (push) Successful in 3m39s
Auto-applied tags are provisional (they don't train the model + can be retracted
until confirmed), so surface and confirm them:
- Backend: list_for_image + get_image_with_tags now include `source` + a
  `confirmed` flag on each applied tag (via serialize_tag, image-scoped; defaulted
  for autocomplete/directory callers).
- Frontend: TagChip badges an unconfirmed auto-tag with an "auto" pill + a
  one-click Keep/confirm (✓) → POST /images/<id>/tags/<id>/confirm, which promotes
  it to a training positive and shields it from the retraction sweep; TagPanel
  reloads so the badge + button drop once confirmed.
Contract test for the source/confirmed payload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:51:39 -04:00
bvandeusen d3984ccb0d fix(ui): tag autocomplete no longer shows stale wrong-prefix results (race)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m37s
The keystroke debounce cleared the timer but not an already-fired fetch, so a
slower earlier-prefix response ("s") could land after "sex" and overwrite the
dropdown with wrong-prefix matches (operator-flagged with a "sex"→Stockings/
Super Mario screenshot). Gate each autocomplete response on a useInflightToken
(cancel on every keystroke, isCurrent() after the await) so only the latest
query's results are applied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:45:25 -04:00
bvandeusen 7d3a3b4a83 revert(ml): keep head auto-apply precision at 0.97 (operator: general tuning was fine)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
Milestone 139 raised head_auto_apply_precision 0.97→0.98; operator confirmed the
general-tag confidence was already well tuned, so revert that. The support floor
(min_positives 30→50) and CCIP match confidence (0.92→0.95) stay. Migration 0081
(not yet deployed) edited to drop the precision bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:42:08 -04:00
bvandeusen 6684907577 feat(ui): "Reject rest" per suggestion category — confirm the good, reject the rest
Each category section header gets a subtle "Reject rest" action that dismisses
every still-unhandled suggestion in it at once (store.dismissRemaining, parallel
dispatch). Canonical tags persist a rejection and stay flagged (reversible,
one-click un-reject); raw creates-new-tag rows drop client-side. Shows only when
the section has unhandled items. No confirm dialog — it's fully reversible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:42:08 -04:00
bvandeusen 2d44a26bdf feat(ml): auto-applied tags don't train a head unless confirmed (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
Makes auto-apply truly "soft" for heads: _ids_with_tag (head positives) and
_eligible_tag_ids (graduation count) now count human-applied + operator-confirmed
tags only, via a shared _AUTO_SOURCES (head_auto/ccip_auto/ml_auto) exclusion.
Unconfirmed auto-applied tags no longer train the head that judges them, so a
misfire can't reinforce itself and the retraction sweep can actually drop it.
Confirming a tag (TagPositiveConfirmation) promotes it to a positive AND protects
it from retraction. sklearn-free tests. CCIP reference exclusion is the companion
piece, next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:28:25 -04:00
bvandeusen 0de726ed48 test(ml): bump auto-apply test head n_pos default 30→60 past the new floor (m139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m42s
The stricter head_auto_apply_min_positives (30→50, migration 0081) dropped the
_head helper's default n_pos=30 below the support floor, so the "supported head"
sweep tests saw the head as ineligible (n_applied 0). Move the default to 60; the
explicit n_pos=5 under-supported test stays correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:18:46 -04:00
bvandeusen 3006e84cc0 feat(ml): soft auto-apply — retract auto-tags now below threshold (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Failing after 3m41s
Daily scheduled_retract_auto_tags re-scores standing auto-applied tags and drops
the ones the model no longer supports:
- retract_auto_applied_heads: per graduated head, re-score its source='head_auto'
  images (bounded — only the images already carrying the auto-tag, not the whole
  library) and remove ones now < auto_apply_threshold.
- retract_auto_applied_ccip: per source='ccip_auto' character tag, max-cosine the
  image's figure vectors vs that character's prototypes; remove ones now below the
  ccip auto-apply threshold.
Both SKIP operator-confirmed tags (TagPositiveConfirmation) and are SILENT — a low
score isn't proof the tag was wrong, so no hard negative is recorded (that's
reserved for an operator removal). No-op unless the relevant auto-apply switch is
on. New daily beat. sklearn-free tests for both paths + the disabled no-op.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:13:37 -04:00
bvandeusen cbc3e11a53 feat(ml): stricter auto-apply defaults to cut misfires (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Failing after 3m48s
head_auto_apply_precision 0.97→0.98, head_auto_apply_min_positives 30→50,
ccip_auto_apply_threshold 0.92→0.95 (operator-asked). Model defaults change for
fresh installs; migration 0081 bumps the existing singleton row IFF still at the
old default (won't clobber a deliberate operator change). ml_admin bounds already
permit these. Fixed a stale comment in the auto-apply test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:08:54 -04:00
bvandeusen 2cfbb284d5 feat(heads): incremental retraining — refit only changed tags (#1317 phase 2, m138)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m35s
train_all_heads is now incremental by default: a per-tag training-data
fingerprint (positive + rejection count/latest-timestamp, stored on
tag_head.train_fingerprint) means a manual Retrain refits ONLY the tags whose
data changed — O(what you touched), not O(all heads). The nightly
scheduled_train_heads passes full=True to reconcile sampled-negative + hygiene
drift across every head. First incremental run after deploy still refits
everyone (NULL fingerprints), stamping them, then it's incremental.

The refit decision + fingerprint are split into sklearn-free helpers
(_head_fingerprints, _heads_needing_retrain) so the incremental logic is
unit-tested directly (train_head itself needs scikit-learn). Migration 0080.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 16:36:30 -04:00
bvandeusen a94f6a2789 feat(ccip): matcher reads the incremental prototype store (#1317, m138 step 4)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m43s
match_image now sources character references from character_prototype via a
per-character in-process cache (_load_prototypes) that reloads ONLY the
characters whose ccip_prototype_state.updated_at advanced — no request-path
rebuild, so the per-accept ~4s stall is gone once the store is populated. Cold
start (store empty pre-first-refresh) falls back to the legacy on-the-fly
reference build, so character suggestions work immediately post-deploy and the
background refresh populates the store within ~15 min. Match math + grounding
are unchanged; existing tests exercise the legacy fallback, and a new test
covers matching from the populated prototype store.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 16:21:38 -04:00
bvandeusen a1ed53136e feat(ccip): refresh task + beat + retrain hook for character prototypes (#1317, m138 step 3)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m41s
- refresh_character_prototypes celery task wraps the incremental builder (sync
  ml worker); returns skipped / rebuilt=N removed=N.
- Beat: every ~15 min (cheap global-gate no-op when idle) + a nightly full=True
  reconcile as belt-and-suspenders.
- train_heads enqueues it on success, so the Retrain button AND the nightly head
  retrain refresh CCIP on the SAME trigger — unified lifecycle, as asked.
The initial (cold) full build loads the whole reference set once in the
background, never on a /suggestions request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 16:13:22 -04:00
bvandeusen 9504870c9a feat(ccip): incremental character-prototype builder (#1317, m138 step 2)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
refresh_character_prototypes (sync, celery ml worker):
- Cheap GLOBAL gate (a few COUNTs) → no-op when nothing that affects references
  changed since the last refresh (the operator's "only recompute if something
  was tagged" trigger).
- Else a per-character fingerprint diff (one GROUP BY: ref count + max region id)
  rebuilds ONLY the characters whose references moved — each capped to
  MLSettings.ccip_prototype_cap — and drops characters that lost all refs.
Cost scales with WHAT changed, not library size. Reuses ccip's reference
predicate (single-character, non-hygiene, figure CCIP) so prototypes match the
legacy matcher exactly. The async matcher (next step) will READ the table.

Tests: gate no-op when idle, only-changed-character rebuild, capping,
single-character exclusion, lost-reference cleanup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 16:07:07 -04:00
bvandeusen f24dc81764 feat(ccip): schema for precomputed incremental character prototypes (#1317, m138 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m42s
Foundation for making CCIP character references a precomputed, INCREMENTAL
artifact instead of a request-path rebuild (kills the per-accept ~4s suggestions
stall; cost will scale with change, not library size):

- character_prototype: a character's reference CCIP vectors, capped to
  MLSettings.ccip_prototype_cap so match cost doesn't grow with popularity.
- ccip_prototype_state: per-character fingerprint (ref count + max region id) +
  updated_at → drives per-character incremental rebuilds and the matcher cache's
  reload-only-what-advanced.
- MLSettings.ccip_ref_signature (cheap global change gate) + ccip_prototype_cap.

Migration 0079. Schema + models only — the builder service, refresh task/beat,
and matcher rewrite land in the following steps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 15:58:11 -04:00
bvandeusen 48be921b8d fix(ui): tag-rail no longer scroll-jumps on accept/reject; keep suggestions clear of the toast
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m39s
- TagAutocomplete: focus the inner input with preventScroll so handing focus
  back after an accept/reject stops yanking the rail up to the field (which sits
  above the suggestions list — you had to scroll back down every time). Applies
  to both the image modal and the Explore rail.
- ExploreView: pad the right rail's scroll bottom (88px) so the bottom-right
  snackbar floats over empty space instead of covering the last suggestions and
  their accept/reject controls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 15:25:04 -04:00
bvandeusen 7939dba9ed feat(ui): grounding overlay leads with the hovered tag, crop origin as subtext (#1206)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m34s
The overlay label showed only the crop origin (booru:head / panel / person-m).
Put the hovered TAG on top as the headline and demote the crop origin to a small,
dimmed, monospace subline — the tag is what you're evaluating; the origin is just
provenance. Threads the tag name through the fcSuggestionHover payload ({g, tag})
from both setters (SuggestionItem for suggestions, TagChip for applied chips).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 15:11:10 -04:00
bvandeusen 8f6547f8d7 refactor(subscriptions): remove the dead backfill dry-run preview (#1281)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m36s
PreviewDialog.vue was orphaned — nothing mounted it — so the dry-run preview
was unreachable. Rather than wire it up, remove the whole chain: its unique
value (a capped 3-page count of what a backfill would grab) is low, and its
adjacent needs are already covered — auth validation by verify_source_credential,
and actually fetching recent posts by "Check now". Operator decision 2026-07-06.

Removed:
- frontend: PreviewDialog.vue + sources store previewSource()
- backend: POST /api/sources/<id>/preview route, download_backends.preview_source,
  IngestCore.preview() + its now-unused NativeIngestError import
- tests: the 3 ingester preview tests

Nothing else referenced the chain (verified). Shared campaign-resolution and
ledger helpers stay — they're used by run()/verify_source_credential.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 14:15:19 -04:00
bvandeusen 2638cf1a35 style(tests): hoist test_api_tags grounding imports to module level (ruff I001)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m43s
CI ruff flagged the in-function import block; move them to the top mirroring
test_ml_suggestions.py's import line, which ruff already accepts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 13:24:55 -04:00
bvandeusen 9bb4211722 feat(ui): hover an applied tag chip → highlight its grounding crop (#133 step 4)
CI / lint (push) Failing after 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m41s
Applied tags aren't scored live, so compute the grounding on demand: run the
tag's head over the image's max-over-bag (whole-image + concept crops), argmax
→ the region that best explains the tag on this image, mirroring what
score_image records for live suggestions.

- heads.py: extract _image_bag (now shared by score_image) + ground_applied_tag.
  Returns (grounding, has_head): has_head False = no head to localize with →
  no overlay; grounding None = the whole-image vector won → whole-image frame.
- tags.py: GET /api/images/<id>/tags/<id>/grounding → {grounding, has_head}.
- TagChip/TagPanel: applied chips inject fcSuggestionHover and fetch grounding
  on hover (cached per image+tag, race-guarded), reusing Step 3's overlay in
  both the modal and Explore. No new frontend overlay code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 13:19:41 -04:00
bvandeusen 524a26c618 feat(ui): hover a suggestion → highlight the crop region it came from (#133 step 3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m36s
The payoff: hover a suggestion in the rail and the exact crop that produced it
lights up on the image (a booru:head, a panel, a figure); a null-grounding tag
shows a subtle dashed whole-image frame ('global vector won, not a crop').
ImageCanvas gains a grounding overlay that tracks the <img>'s live bounding rect
(correct under object-fit letterboxing + pan/zoom) and draws the normalized bbox
+ a detector/kind label. SuggestionItem sets the hovered grounding via
provide/inject (no 4-level event relay through TagPanel/SuggestionsPanel/group);
ImageViewer AND ExploreView provide it + pass it to their canvas. Overlay is
pointer-events:none so it never blocks pan/zoom/click. Videos out of v1 scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-05 23:24:18 -04:00