Compare commits

...

49 Commits

Author SHA1 Message Date
bvandeusen aea2701c28 feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m52s
The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh ==
cached, verified by probe) confidently mis-detects short ASCII English like
"... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a
re-translate reproduced it. Confidence alone can't separate the 0.86 collision
(genuine German lands there too), and single-word mis-flags sit at a confident
1.0 no floor catches.

Two operator-approved levers:

- Acceptance floor is now a live Settings value (ImportSettings.
  translation_min_confidence, default 0.90; surfaced in the Translation card), so
  it's tunable without a redeploy. _accept takes the threshold as a parameter.

- Per-post sticky override (Post.translation_override: auto/force/original).
  'force' stores a translation even below the floor (rescue a skipped
  legit-foreign title); 'original' keeps the original and clears any stored
  translation (kill a confident mis-flag no floor catches). The sweep honors it
  on every run and _reset_translations skips 'original', so the choice survives a
  Re-translate-all. POST /api/posts/<id>/translation-override applies it
  immediately (translate now when the service is up, else queue for the sweep).
  UI: PostTranslationControl on the posts-feed card.

Migration 0084 (both columns + a CHECK on the override). The feed + provenance
serializers expose translation_override.

With a stricter floor the rollback finally works: raise it -> Re-translate all ->
the 0.86 mis-flags are rejected and restored to the original; force /
keep-original handle the residual either way.

Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor
cases); sweep force/original + re-translate-skips-original; override endpoint
(validation, original clears, force queues when disabled, feed exposes it);
settings min_confidence default/save/validate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 22:38:25 -04:00
bvandeusen a444cf82d1 refactor(tags): unify suggestion source — one canonical DB-tag dropdown, drop dead raw/alias machinery (#154)
CI / backend-lint-and-test (push) Successful in 36s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / integration (push) Successful in 3m45s
Every tag suggestion is a canonical DB tag now (tagging-v2 #114: heads + CCIP
score EXISTING concept tags). The pre-heads apparatus for model-predicted tags
that didn't exist in the DB — creates_new_tag / raw_name / via_alias, the
/suggestions/alias endpoint + add_alias_and_accept, AliasPickerDialog, and the
store's aliasAccept/removeAlias — was dead and is removed.

The type-to-add dropdown was TWO row sources (server autocomplete + the image's
ML suggestions) merged with a dedup that dropped the %-bearing suggestion row
when the debounced server hit landed — the operator's "confidence % flickers
then vanishes". Now it's ONE list of DB-tag matches, each annotated with the
model's confidence (join by canonical_tag_id) when the tag was scored for this
image. No dedup, no flicker; picking a suggested tag still records acceptance
via TagPanel.findPending.

Single per-image fetch: score_image now reports above_threshold per row
(computed vs the head's own suggest cut, separate from the inclusion floor), so
the rail makes ONE min=0 request and derives the panel (above_threshold) and the
dropdown (all, text-filtered) client-side — the two /suggestions calls collapse
to one. Manual "Create 'X' as <kind>" (novel typed names) is unchanged; the
alias table + tag-side alias admin + auto-apply alias matching are untouched.

Tests: gate/serializer assertions updated (above_threshold; dropped dead-field
+ alias-endpoint checks); frontend spec seeds via the single load and covers the
byCategory/aboveByCategory split.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 15:22:51 -04:00
bvandeusen 6ab7fd5c7f tune(translation): set latin acceptance floor to 0.80 (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
Calibrated against fresh probes once Interpreter returned real langdetect
confidence: genuine German detected at 1.0, a correctly-detected but ambiguous
latin string at 0.86. Set _MIN_LATIN_CONFIDENCE to 0.80 (below that band) so
legitimate ambiguous non-English still translates while genuinely-unsure guesses
are rejected. Real langdetect also fixed the original mis-flag at the source, so
this floor is a safety net, not the primary fix. Pin 0.86-accepted in the gate
test to guard against bumping the floor back up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-09 16:40:02 -04:00
bvandeusen 7ddf94220d feat(translation): accept/reject gate on Interpreter's detection confidence (#1376)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m46s
Interpreter now returns a real per-detection confidence (source stays "auto"),
so curator can reject the mis-detections it was blindly storing — e.g. a short
English title mis-labelled as German and rewritten into the archive.

The gate consumes ONLY Interpreter's own reported detection — curator does no
language detection of its own (Scribe rule 133): a field is stored when the
engine actually translated it AND either the detected language is CJK
(script-detected, reliably high — ja/ko/zh trusted outright, incl. pure-kanji
Japanese that lands as zh ~0.75) or the reported confidence clears a
latin-script floor (_MIN_LATIN_CONFIDENCE = 0.90). A latin detection below the
floor keeps the original and marks the post handled; a missing confidence fails
open. The client already sent source="auto" and parsed confidence, so this is
purely the gate + tests.

Tests: pinned interpreter-client test now asserts source stays "auto"; new
pure-unit gate tests (CJK trusted / latin floor / case-insensitive / fail-open)
in the fast lane; end-to-end reject-low-latin, accept-high-latin,
accept-low-cjk sweeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-09 16:17:56 -04:00
bvandeusen ffdbdbaf07 feat(translation): 8h sweep + drain button + detection probe (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 46s
CI / integration (push) Successful in 3m55s
Throughput: translate_posts now runs every 8h (was daily) as the
steady-state cadence for newly-imported posts, and the Settings
"Translate now" button runs it in drain mode (run-until-done, no reset)
so one press clears the whole untranslated backlog instead of a single
300-post chunk. The interrupt/backoff re-enqueue now preserves the drain
flag so a bulk drain resumes cleanly after an Interpreter restart.

Misdetection groundwork: surface the detector's confidence from the
Interpreter client (it was in the detectedLanguage payload but discarded)
and add a read-only "Test translation" box — POST /settings/translation/
probe + TranslationCard UI — that shows detected language + confidence +
engine + result for pasted text, without saving. Lets the operator see
why a short/abbreviation-heavy English title gets mis-detected so the
detection guard (min-length + confidence floor) can be tuned from real
numbers. The guard itself follows once the mis-detected cases are probed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-08 20:13:49 -04:00
bvandeusen a017771621 feat(ui): thin kind-coloured border on tag chips so they don't wash out
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m49s
The tonal fill (esp. character = info) is intentionally faint and blends into
the dark tag rail. Add a thin border in each chip's own kind colour via
color-mix on currentColor (the tonal chip's themed foreground), defining the
edge without changing the fill. Theme-aware in both light and dark.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:31:44 -04:00
bvandeusen 25e555cab6 fix(ui): stop leading tag-chip icon clipping on the left edge
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 3m50s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
The larger size=default chip widened Vuetify's negative start-margin on the
leading kind-icon, placing it left of the .v-chip__content box whose
overflow:hidden (the name-truncation guard) then clipped its left edge.
Zero the icon's negative inline-start margin so it sits inside the clip box;
the chip's 12px padding keeps a comfortable left inset.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:29:28 -04:00
bvandeusen 3c7ab44e74 feat(translation): per-field language detection for mixed-language posts
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m43s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
_translate_one translated [title, description] in ONE Interpreter call and keyed
the whole-post passthrough on the aggregate detected_lang (the FIRST item). So an
English title + non-English description detected "en" and marked the post handled,
leaving the description untranslated. Now each field is translated independently
(its own detected_lang / passthrough) and the non-target field is stored on its
own; translated_source_lang reflects the translated field's language.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:31:04 -04:00
bvandeusen 06f98acf3e feat(tagging): bolder auto-tag accept/reject + larger tag chips in the rail
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m45s
The in-pill ✓/✗ on unconfirmed auto-tags read muted — a faint colored icon on a
tonal chip (worst on character tags, kind=info) that only lit up on hover. Make
them solid green/red circles with a white glyph (22px, icon 15), mirroring the
Suggestions rail's verdict buttons so accept/reject read identically. Also bump
the applied-tag chips from size=small to default and the leading kind icon to
match — bigger, clearer tags throughout the rail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:17:18 -04:00
bvandeusen 4371ddb7e7 feat(translation): live re-translate progress in the Settings card
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m45s
CI / frontend-build (push) Successful in 20s
translation/status now reports `active` (a translate/retranslate sweep is
running, from the TaskRun table) and `last_run` (the most recent finished run's
task + status). The Settings card polls live while a sweep runs, showing a
spinner + "Translating… N remaining" that ticks down, and flags a last run that
ended in error/timeout. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:20:32 -04:00
bvandeusen 40cc11be5b feat(deploy): container healthchecks + Swarm rolling-update auto-rollback
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m45s
web gets a /api/health liveness check; workers a lenient celery-ping check. A
shared deploy policy (update_config order=start-first, failure_action=rollback,
monitor 90s; rollback_config; restart_policy) means a bad image that never goes
healthy is rolled back automatically instead of taking the service down. Ignored
by plain `docker compose up` (deploy: is swarm-only), so the dev override is
unaffected. Assumes prod deploys from this file via docker stack deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 0b78264d62 feat(maintenance): daily janitor for orphaned .part/.partial staging files
Downloads/imports stage into <name>.part / <name>.partial then os.replace() into
place, so a kill mid-write leaves a discardable temp — never a corrupt final.
cleanup_orphaned_temp_files sweeps ones left behind under the images root, only
older than 6h so an in-flight download's staging file is never removed. Daily beat.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 9eae636047 feat(translation): pooled Interpreter session + manual sweep resumes after drain
CI / lint (push) Successful in 4s
CI / backend-lint-and-test (push) Successful in 34s
CI / frontend-build (push) Successful in 21s
CI / integration (push) Successful in 3m45s
- interpreter_client: shared requests.Session with a connect-only retry
  (connect=2, no status retries — we map 429/5xx ourselves) so a proxy reload
  is smoothed and the keep-alive connection is pooled across the sweep.
- translate_posts: on an interrupt (drain), re-enqueue after the Retry-After
  hint / default backoff instead of waiting for the daily beat; self-terminates
  via the health gate. Steady-state one-chunk-per-run on success is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen f8105046dc feat(explore): exclude WIP-tagged work from the Explore rabbit-hole
Explore's neighbour grid (/api/gallery/similar → gallery_service.similar) now
takes an Explore-only exclude_wip flag that drops `wip` system-tagged images
from the candidates, alongside the banner/editor presentation tags. The
gallery's own "similar" button is unchanged (keeps wip, #1274) — only the
Explore store passes exclude_wip=1. The anchor itself may still be a WIP; only
neighbours are filtered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen d631ed023c style(translation): use datetime.UTC alias (ruff UP017)
CI / frontend-build (push) Successful in 18s
CI / lint (push) Successful in 4s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:04:17 -04:00
bvandeusen c64261593d feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience
CI / frontend-build (push) Successful in 19s
CI / lint (push) Failing after 2s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m42s
Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs
mid-flight. Give in-flight work room to drain and make interrupted work
resume cleanly instead of stalling.

- docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s /
  scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown
  can actually drain before SIGKILL.
- celery_app.py: task_reject_on_worker_lost=True — a task killed past the
  grace window is re-queued (safe: idempotent + chunked, recovery sweeps
  re-drive stragglers).
- interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable
  and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter
  behind a reverse proxy no longer raises an opaque HTTPError.
- translation.py: thread retry_after out of _translate_batch; retranslate_posts
  resumes after the Retry-After hint (or 60s default, capped 900s) on an
  interrupt with _reset_done=True, self-terminating via the health gate.
- tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff.

No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:01:00 -04:00
bvandeusen 1f6d94f51d feat(translation): re-translate on model change — artist-scoped + global re-run (#146)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
retranslate_posts resets the 5 translation columns to NULL for a scoped set of posts (all, or WHERE artist_id IN ids) then reuses the untranslated sweep to re-run them, chasing the tail until drained (run-until-done). Interpreter cache keys on engine_version so a changed model re-translates, an unchanged one is cache-fast. Reset only happens when the service is configured+healthy so translations are never wiped when they can't be rebuilt. New POST /settings/translation/retranslate (artist_id | all=true). UI: per-artist 'Re-translate posts' on the Artist Management tab + 'Re-translate all' in the Settings Translation card, both with confirm dialogs. No migration (reuses m143 columns).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:03:19 -04:00
bvandeusen 6a255482ea feat(translation): "Test connection" button — on-demand Interpreter health check (#143)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m43s
New POST /api/settings/translation/test pings /v1/health for a GIVEN base URL (not
the saved one), so the operator can verify a URL before enabling it. TranslationCard
gains a Test-connection button that reports reachable/unreachable inline and
updates the status dot. Endpoint test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 13:00:37 -04:00
bvandeusen af5aa21e45 test(translation): API endpoint tests for status + run (#143 step 6)
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m47s
CI / lint (push) Successful in 3s
/settings/translation/status defaults (off → no health call) + /run 400-when-
unconfigured + 202-when-configured (monkeypatched .delay). requests is already a
backend dep, so no requirements/ci-requirements change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:39:43 -04:00
bvandeusen 83c1745fd0 feat(ui): translation-forward post text + Settings card (#143 step 5)
CI / lint (push) Successful in 3s
CI / integration (push) Successful in 3m45s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
PostCard + modal ProvenancePanel show the English title/description by default when
a translation exists, with a per-card "show original (<lang>)" toggle — translated
bodies render as plain text, originals keep their sanitized HTML. New
TranslationCard in Settings → Ingestion & filters: enable switch, Interpreter base
URL (generic placeholder, no default host), target language, a reachability
indicator + untranslated-posts count + "Translate now".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:38:41 -04:00
bvandeusen ead60978e3 feat(translation): expose translated fields in post feed + provenance payloads (#143 step 4)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m44s
post_feed_service (card + detail) and provenance_service._post_dict now include
post_title_translated, description_translated (card-truncated / detail-uncapped)
and translated_source_lang, keeping the originals for the toggle. Feed
serialization test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:31:46 -04:00
bvandeusen 7a4de7278d feat(translation): backfill sweep + beat + manual trigger (#143 step 3)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m49s
tasks/translation.py — translate_posts: picks untranslated posts (title OR
description non-empty), per-post [title, description] batch via the Interpreter
client, stores translations + detected lang + engine_version; passthrough /
already-target posts are marked handled with no stored translation. 503 or a
connection error interrupts (retry next cycle), 400 stops (fix config), per-post
commit keeps progress; wall-clock bounded. Wired into celery (maintenance_long
lane) + a daily beat. No-op unless enabled + base URL set + healthy. GET
/settings/translation/status + POST .../run for the Settings card. Task tests
(stubbed client, monkeypatched session).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:29:28 -04:00
bvandeusen 7f8073c4c8 feat(translation): Interpreter client — batch translate + health (#143 step 2)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m43s
services/interpreter_client.py: sync (requests) client for the LibreTranslate-
compatible /v1/translate — no new dep, mirrors the platform clients. translate()
maps translatedText[]↔texts (order + length), returns detected_lang +
engine_version (aggregate = first item, fine for a per-post [title, description]
batch); passthrough items come back unchanged in their slot. InterpreterUnavailable
on 503 / connection error (retry later), InterpreterBadRequest on 400. health()
checks /v1/health engines.llm. 10 unit tests with mocked HTTP.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:23:49 -04:00
bvandeusen a3bc98a53c feat(translation): Post translation columns + settings + migration (#143 step 1)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m46s
Post gains post_title_translated / description_translated / translated_source_lang
/ translation_engine_version / translated_at — filled by the translate sweep so
viewing is instant. ImportSettings gains translation_enabled (OFF by default),
interpreter_base_url (EMPTY — no default host; the operator points it at their own
Interpreter proxy behind a reverse proxy) and translation_target_lang (en),
exposed + validated via /settings/import. Migration 0083. Settings defaults +
patch + validation test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 12:20:31 -04:00
bvandeusen e7c3f4e9c9 feat(ui): auto-tag accept/reject as an in-pill yes/no pair; confirm refocuses input
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m38s
Fold the auto-tag accept into the chip: a provisional auto-tag now shows a compact
green ✓ / red ✗ pair IN the pill (replacing the ✕), and the "auto" text label is
dropped — the yes/no is signal enough (operator-asked). ✓ confirms (trains +
shields from retraction), ✗ removes (records a negative). The name still
ellipsis-truncates so the pair stays reachable.

onConfirm now returns focus to the tag input like onRemove already does, so the
input is the cursor's resting position after any chip action in both the modal
and Explore views.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 00:56:15 -04:00
bvandeusen 29f3a485b0 fix(ui): surface auto-hide misfires proactively — review strip no longer gated on Show-hidden (#141)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m44s
The conflict-flag review strip only appeared when "Show hidden" was toggled on, so
misfires could go unnoticed — defeating the point of flagging them. Fetch pending
flags on mount and show the strip whenever there are any, independent of the
toggle (operator-flagged). Gated to the main gallery (not the post-detail view),
matching the filter bar.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-07 00:39:23 -04:00
bvandeusen 2bcaa20b22 feat(ml): schedule presentation auto-hide sweep + retention (#141 step 6)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m41s
scheduled_presentation_auto_apply (daily beat) runs presentation_auto_apply_sweep
— idempotent, so an interrupted run just re-runs next cycle (that's the recovery),
wall-clock bounded by soft/hard task time limits. prune_presentation_reviews
(daily beat) drops RESOLVED review flags older than 30 days (rule 89 retention).
Tests run both tasks via a monkeypatched session factory. Milestone 141 complete:
the presentation-chrome auto-hide + conflict-flagged review is now live end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:46:09 -04:00
bvandeusen 9726d6f4b5 feat(ui): hidden-view review strip — flagged auto-hides with keep / un-hide (#141 step 5)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m40s
When "Show hidden" is on, a review strip appears atop the gallery listing the
auto-hidden chrome flagged "also looks like content" (most-concerning first):
thumbnail + "also looks like <X>" + Keep hidden / Un-hide. Un-hide removes the
presentation tag (image returns to the gallery) and trains the head; Keep
resolves the flag. Self-hides when there's nothing to review; theme-token styled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:39:02 -04:00
bvandeusen 6c34f86477 feat(gallery): hidden-view review endpoints — list + keep + un-hide (#141 step 5)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m46s
GET /api/gallery/hidden-review lists unresolved presentation auto-hide flags
(image + presentation tag + conflict tag/score), most-concerning first. POST
.../keep resolves the flag (the tag stays). POST .../unhide removes the
presentation tag (image returns to the gallery), records a TagSuggestionRejection
so the head learns it misfired, and resolves the flag. Tests for list/keep/unhide.
Frontend review strip (shown when Show-hidden is on) next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:34:18 -04:00
bvandeusen 1f548d8a7b feat(ui): presentation-chrome auto-hide Settings controls (#141 step 4)
CI / backend-lint-and-test (push) Successful in 32s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 3m41s
HeadsCard gains a "Hide presentation chrome" section: on/off switch + "Hide
confidence" (presentation_auto_apply_threshold) + "Flag if content ≥"
(presentation_conflict_threshold), wired to MLSettings via patchSettings and
loaded on mount. Makes the step-4 sweep's thresholds operator-tunable
(config-in-UI). wip is called out as never auto-hidden.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:16:41 -04:00
bvandeusen eedf8d109a feat(ml): presentation-chrome auto-hide sweep + hard-skip + conflict flagging (#141 step 4)
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m39s
CI / lint (push) Successful in 3s
presentation_auto_apply_sweep fires banner/editor-screenshot heads at the FLAT
presentation threshold (source=presentation_auto). Two guards: (1) hard-skip any
image already carrying a human/confirmed content tag — you valued it, so the model
can't bury it; (2) if an auto-hide ALSO scores >= presentation_conflict_threshold
on a content head, hide it but record a PresentationReview row (conflict tag +
score) for the Hidden view.

_auto_apply_heads now excludes system tags, so a graduated wip/banner can't fire
via the content path (and wip never auto-applies at all). presentation_auto added
to _AUTO_SOURCES so auto-hidden chrome never self-trains. Tests: applies,
hard-skip valued, conflict-flag, disabled no-op, ignores wip, content-path
excludes system. Settings UI + scheduling land next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 23:11:26 -04:00
bvandeusen ab63d94249 feat(ml): presentation auto-hide settings + review table (#141 step 3)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m47s
MLSettings gains presentation_auto_apply_enabled / _threshold (default 0.90) +
presentation_conflict_threshold (default 0.50): banner/editor auto-hide with a
FLAT threshold (decoupled from content-head graduation), plus the "also looks
like content" conflict cut. New presentation_review table (image, presentation
tag, conflict tag + score, created/resolved_at) records auto-hides flagged for
review. Migration 0082 (columns + table), ml_admin API (editable + get_settings
+ _validate bounds), settings roundtrip/bounds test. The sweep that reads these
knobs + the Settings UI land in step 4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:59:00 -04:00
bvandeusen eadaa716af feat(gallery): "Show hidden" toggle reveals presentation chrome (#141 step 2)
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m44s
CI / lint (push) Successful in 2s
A Curation-group chip in the facet panel flips include_hidden, threaded through
the gallery filter store (default model, activeFilterParam, applyFilterFromQuery,
cloneFilter, filterToQuery) and counted in the refine badge. Off by default → the
gallery hides banner/editor-screenshot chrome; on → it's revealed. Backend
already honors include_hidden (step 1). The dedicated conflict-flagged review
surface (only the set-aside items) lands in step 5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:43:24 -04:00
bvandeusen 0efb187eb1 fix(gallery): drop include_hidden before the similar() call (#141 step 1)
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m44s
CI / lint (push) Successful in 2s
The /similar route splats **filters into similar(), which doesn't take the new
include_hidden kwarg → TypeError → 500 (test_gallery_similar). Drop it like
post_id; similar() has its own presentation exclusion (#1274), so the
gallery-browse flag doesn't apply there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:32:53 -04:00
bvandeusen e86b91dfe2 feat(gallery): default-hide presentation chrome (banner/editor screenshot) (#141 step 1)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Failing after 3m41s
The default gallery + facets now implicitly exclude images carrying a
presentation system tag (banner / editor screenshot), reusing the tag-scope
EXISTS machinery. Suppressed when the operator explicitly filters FOR a
presentation tag OR passes include_hidden (the Hidden view — step 2). `wip` is
NOT hidden (real, in-progress art). include_hidden threaded through
scroll/timeline/jump_cursor/facets + the gallery API _parse_filters. Test covers
default-hide, include_hidden, explicit-filter-shows, and wip-stays-visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:27:00 -04:00
bvandeusen 4f4ddecf75 fix(ui): long tag chips no longer overflow the rail and clip their ✕
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 26s
CI / integration (push) Successful in 3m42s
CI / backend-lint-and-test (push) Successful in 36s
A character+fandom(+AUTO) chip could exceed the tag-panel width, pushing the
close ✕ off the right edge so the tag couldn't be removed (operator-flagged with
a "Mirko - Rumi Usagiyama → My Hero Academia AUTO" screenshot). Cap the chip at
100% of the rail and make the NAME the elastic part (ellipsis-truncates), so the
✕, fandom, and AUTO badge stay reachable; the full name stays on the hover title.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 22:22:21 -04:00
bvandeusen 04d5d62cfe style(ui): auto-tag Keep button matches the suggestion accept button
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 45s
CI / integration (push) Successful in 3m48s
The ✓ Keep on a provisional auto-tag chip was a smaller tinted-outline variant;
make it the SAME filled green circle (white ✓, 26px, opacity 0.9→1 + scale on
hover, accent focus ring) as the suggestion accept button (.fc-act--yes in
SuggestionItem) so "accept this tag" reads identically across surfaces
(operator-asked).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 21:36:19 -04:00
bvandeusen 18bb25f140 fix: ruff C416 (dict() over comprehension) + frontend test playlistIds rename
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m41s
- heads.py: conf_map = dict(conf) instead of a dict comprehension (ruff C416).
- postCard.spec.js: the modal-playlist rename (postImageIds→playlistIds) missed
  this frontend test (grep was src-only); update the expected call args.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:09:29 -04:00
bvandeusen 17433c69d4 test(tag): serialize_tag unit tests include the new source/confirmed keys (m139)
CI / lint (push) Failing after 3s
CI / integration (push) Successful in 3m46s
CI / frontend-build (push) Failing after 17s
CI / backend-lint-and-test (push) Successful in 27s
The confirm-UI change added source + confirmed to serialize_tag; two exact-dict
unit tests in test_tag_query.py failed on the new keys. Add them (default
None/False for rows without image scope).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:03:51 -04:00
bvandeusen 3bf41ecac3 feat(ui): modal prev/next walks the current gallery filter (#1322)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Failing after 17s
CI / integration (push) Successful in 3m43s
CI / backend-lint-and-test (push) Failing after 27s
The image modal cycled GLOBAL neighbours; now the gallery hands it a snapshot of
the currently-filtered, ordered id list so prev/next moves through exactly what
you're viewing — the filtered-playlist behaviour lost in the ImageRepo→FC move.
Generalized the modal store's post-scoped cycle into a `playlistIds` playlist
reused by both GalleryView and PostCard (falls back to global neighbours when no
playlist is passed, e.g. Explore's "open full viewer").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 19:02:38 -04:00
bvandeusen bae077e323 feat(ml): CCIP references exclude unconfirmed auto character tags + confirm trips detectors (m139)
CI / backend-lint-and-test (push) Failing after 42s
CI / lint (push) Failing after 3s
CI / frontend-build (push) Successful in 20s
CI / integration (push) Successful in 3m46s
Completes "no self-training": unconfirmed auto-applied character tags no longer
seed CCIP references — character_references + the prototype builder
(_current_fingerprints/_rebuild_one) gain a shared _positive_char_tag filter
(human-applied OR operator-confirmed), mirroring the head-positive exclusion.

Confirming a tag also has to move the change-detectors, or an incremental
refresh/Retrain right after a confirm wouldn't fold the tag in (only the nightly
full pass would): the CCIP global gate now counts character confirmations, and
the head training fingerprint counts confirmations. Test for the CCIP path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:59:00 -04:00
bvandeusen 775941609d feat(ui): confirm/keep auto-applied tags (milestone 139)
CI / frontend-build (push) Successful in 19s
CI / integration (push) Successful in 3m39s
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Failing after 28s
Auto-applied tags are provisional (they don't train the model + can be retracted
until confirmed), so surface and confirm them:
- Backend: list_for_image + get_image_with_tags now include `source` + a
  `confirmed` flag on each applied tag (via serialize_tag, image-scoped; defaulted
  for autocomplete/directory callers).
- Frontend: TagChip badges an unconfirmed auto-tag with an "auto" pill + a
  one-click Keep/confirm (✓) → POST /images/<id>/tags/<id>/confirm, which promotes
  it to a training positive and shields it from the retraction sweep; TagPanel
  reloads so the badge + button drop once confirmed.
Contract test for the source/confirmed payload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:51:39 -04:00
bvandeusen d3984ccb0d fix(ui): tag autocomplete no longer shows stale wrong-prefix results (race)
CI / frontend-build (push) Successful in 19s
CI / lint (push) Successful in 2s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m37s
The keystroke debounce cleared the timer but not an already-fired fetch, so a
slower earlier-prefix response ("s") could land after "sex" and overwrite the
dropdown with wrong-prefix matches (operator-flagged with a "sex"→Stockings/
Super Mario screenshot). Gate each autocomplete response on a useInflightToken
(cancel on every keystroke, isCurrent() after the await) so only the latest
query's results are applied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:45:25 -04:00
bvandeusen 7d3a3b4a83 revert(ml): keep head auto-apply precision at 0.97 (operator: general tuning was fine)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
Milestone 139 raised head_auto_apply_precision 0.97→0.98; operator confirmed the
general-tag confidence was already well tuned, so revert that. The support floor
(min_positives 30→50) and CCIP match confidence (0.92→0.95) stay. Migration 0081
(not yet deployed) edited to drop the precision bump.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:42:08 -04:00
bvandeusen 6684907577 feat(ui): "Reject rest" per suggestion category — confirm the good, reject the rest
Each category section header gets a subtle "Reject rest" action that dismisses
every still-unhandled suggestion in it at once (store.dismissRemaining, parallel
dispatch). Canonical tags persist a rejection and stay flagged (reversible,
one-click un-reject); raw creates-new-tag rows drop client-side. Shows only when
the section has unhandled items. No confirm dialog — it's fully reversible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:42:08 -04:00
bvandeusen 2d44a26bdf feat(ml): auto-applied tags don't train a head unless confirmed (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m41s
Makes auto-apply truly "soft" for heads: _ids_with_tag (head positives) and
_eligible_tag_ids (graduation count) now count human-applied + operator-confirmed
tags only, via a shared _AUTO_SOURCES (head_auto/ccip_auto/ml_auto) exclusion.
Unconfirmed auto-applied tags no longer train the head that judges them, so a
misfire can't reinforce itself and the retraction sweep can actually drop it.
Confirming a tag (TagPositiveConfirmation) promotes it to a positive AND protects
it from retraction. sklearn-free tests. CCIP reference exclusion is the companion
piece, next.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:28:25 -04:00
bvandeusen 0de726ed48 test(ml): bump auto-apply test head n_pos default 30→60 past the new floor (m139)
CI / integration (push) Successful in 3m42s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 32s
The stricter head_auto_apply_min_positives (30→50, migration 0081) dropped the
_head helper's default n_pos=30 below the support floor, so the "supported head"
sweep tests saw the head as ineligible (n_applied 0). Move the default to 60; the
explicit n_pos=5 under-supported test stays correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:18:46 -04:00
bvandeusen 3006e84cc0 feat(ml): soft auto-apply — retract auto-tags now below threshold (milestone 139)
CI / integration (push) Failing after 3m41s
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 29s
Daily scheduled_retract_auto_tags re-scores standing auto-applied tags and drops
the ones the model no longer supports:
- retract_auto_applied_heads: per graduated head, re-score its source='head_auto'
  images (bounded — only the images already carrying the auto-tag, not the whole
  library) and remove ones now < auto_apply_threshold.
- retract_auto_applied_ccip: per source='ccip_auto' character tag, max-cosine the
  image's figure vectors vs that character's prototypes; remove ones now below the
  ccip auto-apply threshold.
Both SKIP operator-confirmed tags (TagPositiveConfirmation) and are SILENT — a low
score isn't proof the tag was wrong, so no hard negative is recorded (that's
reserved for an operator removal). No-op unless the relevant auto-apply switch is
on. New daily beat. sklearn-free tests for both paths + the disabled no-op.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:13:37 -04:00
bvandeusen cbc3e11a53 feat(ml): stricter auto-apply defaults to cut misfires (milestone 139)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Failing after 3m48s
head_auto_apply_precision 0.97→0.98, head_auto_apply_min_positives 30→50,
ccip_auto_apply_threshold 0.92→0.95 (operator-asked). Model defaults change for
fresh installs; migration 0081 bumps the existing singleton row IFF still at the
old default (won't clobber a deliberate operator change). ml_admin bounds already
permit these. Fixed a stale comment in the auto-apply test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
2026-07-06 18:08:54 -04:00
78 changed files with 5121 additions and 632 deletions
@@ -0,0 +1,43 @@
"""stricter auto-apply defaults (milestone 139) — cut auto-apply misfires
head_auto_apply_min_positives 30→50 and ccip_auto_apply_threshold 0.92→0.95
(operator-asked 2026-07-06). The head graduation precision bar stays 0.97 — the
operator confirmed the general-tag confidence was already well tuned; only the
support floor + the CCIP match confidence are raised. The model defaults change
for fresh installs; here we bump the existing singleton row IFF it is still at
the previous default, so a deliberate operator change is NOT clobbered.
Revision ID: 0081
Revises: 0080
Create Date: 2026-07-06
"""
from typing import Sequence, Union
from alembic import op
revision: str = "0081"
down_revision: Union[str, None] = "0080"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.execute(
"UPDATE ml_settings SET head_auto_apply_min_positives = 50 "
"WHERE head_auto_apply_min_positives = 30"
)
op.execute(
"UPDATE ml_settings SET ccip_auto_apply_threshold = 0.95 "
"WHERE ccip_auto_apply_threshold = 0.92"
)
def downgrade() -> None:
op.execute(
"UPDATE ml_settings SET head_auto_apply_min_positives = 30 "
"WHERE head_auto_apply_min_positives = 50"
)
op.execute(
"UPDATE ml_settings SET ccip_auto_apply_threshold = 0.92 "
"WHERE ccip_auto_apply_threshold = 0.95"
)
@@ -0,0 +1,85 @@
"""presentation-chrome auto-hide (#141) — settings knobs + review table
MLSettings gains presentation_auto_apply_enabled / _threshold and
presentation_conflict_threshold: banner + editor-screenshot auto-hide on the
sweep with a FLAT threshold (decoupled from content-head graduation), and a
conflict threshold that flags an auto-hide that "also looks like content".
New table presentation_review records an auto-hidden chrome image that also
scored high on a content head, surfaced in the Hidden view for a keep-hidden /
un-hide decision. Resolved rows are pruned by retention.
Revision ID: 0082
Revises: 0081
Create Date: 2026-07-07
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0082"
down_revision: Union[str, None] = "0081"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"ml_settings",
sa.Column(
"presentation_auto_apply_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("true"),
),
)
op.add_column(
"ml_settings",
sa.Column(
"presentation_auto_apply_threshold", sa.Float(), nullable=False,
server_default=sa.text("0.90"),
),
)
op.add_column(
"ml_settings",
sa.Column(
"presentation_conflict_threshold", sa.Float(), nullable=False,
server_default=sa.text("0.50"),
),
)
op.create_table(
"presentation_review",
sa.Column(
"image_record_id", sa.Integer(),
sa.ForeignKey("image_record.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"tag_id", sa.Integer(),
sa.ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True,
),
sa.Column(
"conflict_tag_id", sa.Integer(),
sa.ForeignKey("tag.id", ondelete="SET NULL"), nullable=True,
),
sa.Column("conflict_score", sa.Float(), nullable=False),
sa.Column(
"created_at", sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now(),
),
sa.Column("resolved_at", sa.DateTime(timezone=True), nullable=True),
)
# The review list queries the unresolved flags (resolved_at IS NULL).
op.create_index(
"ix_presentation_review_resolved_at", "presentation_review",
["resolved_at"],
)
def downgrade() -> None:
op.drop_index(
"ix_presentation_review_resolved_at", table_name="presentation_review"
)
op.drop_table("presentation_review")
op.drop_column("ml_settings", "presentation_conflict_threshold")
op.drop_column("ml_settings", "presentation_auto_apply_threshold")
op.drop_column("ml_settings", "presentation_auto_apply_enabled")
+73
View File
@@ -0,0 +1,73 @@
"""post-text translation via Interpreter (milestone 143) — Post columns + settings
Post gains the translated title/description + the detected source language,
Interpreter engine_version (cache key), and translated_at — filled by the
translate sweep. ImportSettings gains translation_enabled (OFF by default),
interpreter_base_url (EMPTY — the operator sets their own, behind a reverse
proxy), and translation_target_lang (en).
Revision ID: 0083
Revises: 0082
Create Date: 2026-07-07
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0083"
down_revision: Union[str, None] = "0082"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"post", sa.Column("post_title_translated", sa.Text(), nullable=True)
)
op.add_column(
"post", sa.Column("description_translated", sa.Text(), nullable=True)
)
op.add_column(
"post",
sa.Column("translated_source_lang", sa.String(8), nullable=True),
)
op.add_column(
"post",
sa.Column("translation_engine_version", sa.String(128), nullable=True),
)
op.add_column(
"post",
sa.Column("translated_at", sa.DateTime(timezone=True), nullable=True),
)
op.add_column(
"import_settings",
sa.Column(
"translation_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
op.add_column(
"import_settings",
sa.Column(
"interpreter_base_url", sa.Text(), nullable=False, server_default="",
),
)
op.add_column(
"import_settings",
sa.Column(
"translation_target_lang", sa.Text(), nullable=False,
server_default="en",
),
)
def downgrade() -> None:
op.drop_column("import_settings", "translation_target_lang")
op.drop_column("import_settings", "interpreter_base_url")
op.drop_column("import_settings", "translation_enabled")
op.drop_column("post", "translated_at")
op.drop_column("post", "translation_engine_version")
op.drop_column("post", "translated_source_lang")
op.drop_column("post", "description_translated")
op.drop_column("post", "post_title_translated")
@@ -0,0 +1,51 @@
"""translation strictness setting + per-post translation override (milestone 155)
ImportSettings gains ``translation_min_confidence`` (the latin-script acceptance
floor, now operator-tunable in the UI; default 0.9 — stricter than the old
hardcoded 0.8, since Interpreter confidently mis-detects short ASCII English at
~0.86). Post gains ``translation_override`` — a sticky per-post choice of
auto / force / original so the operator can force a skipped translation on, or
knock a wrongly-translated one back to the original, and have it survive a
Re-translate-all.
Revision ID: 0084
Revises: 0083
Create Date: 2026-07-10
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0084"
down_revision: Union[str, None] = "0083"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"translation_min_confidence", sa.Float(), nullable=False,
server_default=sa.text("0.9"),
),
)
op.add_column(
"post",
sa.Column(
"translation_override", sa.String(16), nullable=False,
server_default="auto",
),
)
op.create_check_constraint(
"ck_post_translation_override",
"post",
"translation_override IN ('auto', 'force', 'original')",
)
def downgrade() -> None:
op.drop_constraint("ck_post_translation_override", "post", type_="check")
op.drop_column("post", "translation_override")
op.drop_column("import_settings", "translation_min_confidence")
+115 -3
View File
@@ -3,9 +3,18 @@
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from quart import Blueprint, jsonify, request from quart import Blueprint, jsonify, request
from sqlalchemy import delete, select, update
from sqlalchemy.orm import aliased
from ..extensions import get_session from ..extensions import get_session
from ..services.gallery_service import GalleryService from ..models import (
ImageRecord,
PresentationReview,
Tag,
TagSuggestionRejection,
)
from ..models.tag import image_tag
from ..services.gallery_service import GalleryService, image_url, thumbnail_url
gallery_bp = Blueprint("gallery", __name__, url_prefix="/api/gallery") gallery_bp = Blueprint("gallery", __name__, url_prefix="/api/gallery")
@@ -77,6 +86,9 @@ def _parse_filters():
platform = request.args.get("platform") or None platform = request.args.get("platform") or None
untagged = request.args.get("untagged") in ("1", "true", "yes") untagged = request.args.get("untagged") in ("1", "true", "yes")
no_artist = request.args.get("no_artist") in ("1", "true", "yes") no_artist = request.args.get("no_artist") in ("1", "true", "yes")
# Show the presentation chrome (banner / editor screenshot) that the default
# gallery hides — the Hidden view sets this (milestone 141).
include_hidden = request.args.get("include_hidden") in ("1", "true", "yes")
date_from = _parse_date(request.args.get("date_from")) date_from = _parse_date(request.args.get("date_from"))
date_to = _parse_date(request.args.get("date_to")) date_to = _parse_date(request.args.get("date_to"))
if date_to is not None: if date_to is not None:
@@ -88,6 +100,7 @@ def _parse_filters():
"platform": platform, "platform": platform,
"untagged": untagged, "no_artist": no_artist, "untagged": untagged, "no_artist": no_artist,
"date_from": date_from, "date_to": date_to, "date_from": date_from, "date_to": date_to,
"include_hidden": include_hidden,
} }
return filters, sort return filters, sort
@@ -132,12 +145,20 @@ async def similar():
filters, _sort = _parse_filters() filters, _sort = _parse_filters()
except (KeyError, ValueError): except (KeyError, ValueError):
return jsonify({"error": "similar_to query param required"}), 400 return jsonify({"error": "similar_to query param required"}), 400
# Explore passes exclude_wip=1 to also drop work-in-progress from the
# rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274).
exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True")
# post_id is the exclusive post-detail view — not a similarity scope. # post_id is the exclusive post-detail view — not a similarity scope.
scope = {k: v for k, v in filters.items() if k != "post_id"} # include_hidden is a gallery-browse flag; similar() has its OWN presentation
# exclusion (a similarity-quality concern, #1274), so drop it here (#141).
scope = {
k: v for k, v in filters.items() if k not in ("post_id", "include_hidden")
}
async with get_session() as session: async with get_session() as session:
svc = GalleryService(session) svc = GalleryService(session)
try: try:
images = await svc.similar(image_id=similar_to, limit=limit, **scope) images = await svc.similar(
image_id=similar_to, limit=limit, exclude_wip=exclude_wip, **scope)
except ValueError as exc: except ValueError as exc:
return jsonify({"error": str(exc)}), 400 return jsonify({"error": str(exc)}), 400
if images is None: if images is None:
@@ -211,6 +232,97 @@ async def jump():
return jsonify({"cursor": cursor}) return jsonify({"cursor": cursor})
# -- Hidden-view review (#141): auto-hidden chrome flagged "also looks like
# content", surfaced in the gallery's Show-hidden review strip. -----------
@gallery_bp.route("/hidden-review", methods=["GET"])
async def hidden_review():
"""Unresolved presentation auto-hide flags, most-concerning first (highest
content score) — for the gallery's Hidden-view review strip."""
ptag = aliased(Tag)
ctag = aliased(Tag)
async with get_session() as session:
rows = (await session.execute(
select(
PresentationReview.image_record_id,
PresentationReview.tag_id,
PresentationReview.conflict_tag_id,
PresentationReview.conflict_score,
ImageRecord.path, ImageRecord.thumbnail_path,
ImageRecord.sha256, ImageRecord.mime,
ptag.name.label("tag_name"),
ctag.name.label("conflict_name"),
)
.join(ImageRecord, ImageRecord.id == PresentationReview.image_record_id)
.join(ptag, ptag.id == PresentationReview.tag_id)
.outerjoin(ctag, ctag.id == PresentationReview.conflict_tag_id)
.where(PresentationReview.resolved_at.is_(None))
.order_by(PresentationReview.conflict_score.desc())
)).all()
return jsonify({"items": [
{
"image_id": r.image_record_id,
"tag_id": r.tag_id,
"tag_name": r.tag_name,
"conflict_tag_id": r.conflict_tag_id,
"conflict_name": r.conflict_name,
"conflict_score": r.conflict_score,
"thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
"image_url": image_url(r.path),
}
for r in rows
]})
@gallery_bp.route(
"/hidden-review/<int:image_id>/<int:tag_id>/keep", methods=["POST"]
)
async def hidden_review_keep(image_id, tag_id):
"""Keep the auto-hide: resolve the flag; the tag stays applied (#141)."""
async with get_session() as session:
await session.execute(
update(PresentationReview)
.where(
PresentationReview.image_record_id == image_id,
PresentationReview.tag_id == tag_id,
)
.values(resolved_at=datetime.now(UTC))
)
await session.commit()
return "", 204
@gallery_bp.route(
"/hidden-review/<int:image_id>/<int:tag_id>/unhide", methods=["POST"]
)
async def hidden_review_unhide(image_id, tag_id):
"""Un-hide: remove the presentation tag (image returns to the gallery), record
a rejection so the head LEARNS it misfired, and resolve the flag (#141)."""
from sqlalchemy.dialects.postgresql import insert as pg_insert
async with get_session() as session:
await session.execute(
delete(image_tag).where(
image_tag.c.image_record_id == image_id,
image_tag.c.tag_id == tag_id,
)
)
await session.execute(
pg_insert(TagSuggestionRejection)
.values(image_record_id=image_id, tag_id=tag_id)
.on_conflict_do_nothing()
)
await session.execute(
update(PresentationReview)
.where(
PresentationReview.image_record_id == image_id,
PresentationReview.tag_id == tag_id,
)
.values(resolved_at=datetime.now(UTC))
)
await session.commit()
return "", 204
@gallery_bp.route("/image/<int:image_id>", methods=["GET"]) @gallery_bp.route("/image/<int:image_id>", methods=["GET"])
async def image_detail(image_id: int): async def image_detail(image_id: int):
async with get_session() as session: async with get_session() as session:
+12
View File
@@ -39,6 +39,9 @@ _EDITABLE = (
"ccip_match_threshold", "ccip_match_threshold",
"ccip_auto_apply_enabled", "ccip_auto_apply_enabled",
"ccip_auto_apply_threshold", "ccip_auto_apply_threshold",
"presentation_auto_apply_enabled",
"presentation_auto_apply_threshold",
"presentation_conflict_threshold",
"embedder_model_name", "embedder_model_name",
"embedder_model_version", "embedder_model_version",
*_DETECTOR_FIELDS, *_DETECTOR_FIELDS,
@@ -96,6 +99,9 @@ async def get_settings():
"ccip_match_threshold": s.ccip_match_threshold, "ccip_match_threshold": s.ccip_match_threshold,
"ccip_auto_apply_enabled": s.ccip_auto_apply_enabled, "ccip_auto_apply_enabled": s.ccip_auto_apply_enabled,
"ccip_auto_apply_threshold": s.ccip_auto_apply_threshold, "ccip_auto_apply_threshold": s.ccip_auto_apply_threshold,
"presentation_auto_apply_enabled": s.presentation_auto_apply_enabled,
"presentation_auto_apply_threshold": s.presentation_auto_apply_threshold,
"presentation_conflict_threshold": s.presentation_conflict_threshold,
"embedder_model_name": s.embedder_model_name, "embedder_model_name": s.embedder_model_name,
**{f: getattr(s, f) for f in _DETECTOR_FIELDS}, **{f: getattr(s, f) for f in _DETECTOR_FIELDS},
} }
@@ -150,6 +156,12 @@ def _validate(p: dict) -> str | None:
return "ccip_match_threshold must be between 0.5 and 0.999" return "ccip_match_threshold must be between 0.5 and 0.999"
if not (0.5 <= float(p["ccip_auto_apply_threshold"]) <= 0.999): if not (0.5 <= float(p["ccip_auto_apply_threshold"]) <= 0.999):
return "ccip_auto_apply_threshold must be between 0.5 and 0.999" return "ccip_auto_apply_threshold must be between 0.5 and 0.999"
# Presentation chrome auto-hide (#141). Auto-apply runs high (hiding is
# consequential); the conflict cut is a plain probability [0,1].
if not (0.5 <= float(p["presentation_auto_apply_threshold"]) <= 0.999):
return "presentation_auto_apply_threshold must be between 0.5 and 0.999"
if not (0.0 <= float(p["presentation_conflict_threshold"]) <= 1.0):
return "presentation_conflict_threshold must be between 0 and 1"
# Embedder model swap (#1190): both must be non-empty. Changing them means a # Embedder model swap (#1190): both must be non-empty. Changing them means a
# different embedding space — the operator must re-embed + retrain after. # different embedding space — the operator must re-embed + retrain after.
for key in ("embedder_model_name", "embedder_model_version"): for key in ("embedder_model_name", "embedder_model_version"):
+83
View File
@@ -3,12 +3,28 @@
from quart import Blueprint, jsonify, request from quart import Blueprint, jsonify, request
from ..extensions import get_session from ..extensions import get_session
from ..models import ImportSettings, Post
from ..services import interpreter_client as ic
from ..services.post_feed_service import PostFeedService from ..services.post_feed_service import PostFeedService
from ..services.source_service import KNOWN_PLATFORMS from ..services.source_service import KNOWN_PLATFORMS
from ..utils.text import html_to_plain
from ._responses import error_response as _bad from ._responses import error_response as _bad
posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts") posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts")
_TRANSLATION_OVERRIDES = ("auto", "force", "original")
def _queue_for_sweep(post: Post) -> None:
"""Mark a post untranslated (all translation columns NULL) so the periodic
sweep re-runs it under its new override — used when Interpreter is down and we
can't translate inline."""
post.post_title_translated = None
post.description_translated = None
post.translated_source_lang = None
post.translation_engine_version = None
post.translated_at = None
@posts_bp.route("", methods=["GET"]) @posts_bp.route("", methods=["GET"])
async def list_posts(): async def list_posts():
@@ -82,3 +98,70 @@ async def get_post(post_id: int):
if item is None: if item is None:
return _bad("not_found", status=404, detail=f"post id={post_id}") return _bad("not_found", status=404, detail=f"post id={post_id}")
return jsonify(item) return jsonify(item)
@posts_bp.route("/<int:post_id>/translation-override", methods=["POST"])
async def set_translation_override(post_id: int):
"""Sticky per-post translation override (milestone 155). Body:
``{"override": "auto" | "force" | "original"}``.
'original' keeps the original (clears any stored translation now — no
Interpreter needed). 'force'/'auto' translate the post immediately if the
service is up (force bypasses the acceptance floor; auto re-runs the gate);
if it's down we save the flag and mark the post untranslated so the next sweep
applies it. The override persists, so the sweep + Re-translate-all keep
honoring it. Returns the updated translation fields + an ``applied`` status."""
body = await request.get_json(silent=True) or {}
override = body.get("override")
if override not in _TRANSLATION_OVERRIDES:
return _bad(
"invalid_override",
detail=f"override must be one of {list(_TRANSLATION_OVERRIDES)}",
)
# Lazy import (mirrors settings.py) so the API module doesn't pull the celery
# task graph at import time.
from ..tasks.translation import _store_translation, _translate_field
async with get_session() as session:
post = await session.get(Post, post_id)
if post is None:
return _bad("not_found", status=404, detail=f"post id={post_id}")
post.translation_override = override
cfg = await ImportSettings.load(session)
target = (cfg.translation_target_lang or "en").strip() or "en"
if override == "original":
_store_translation(post, (None, None, None), (None, None, None), target)
applied = "cleared"
else:
base_url = (cfg.interpreter_base_url or "").strip()
if cfg.translation_enabled and base_url and ic.health(base_url):
force = override == "force"
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
mc = cfg.translation_min_confidence
try:
title_res = _translate_field(title, base_url, target, mc, force=force)
desc_res = _translate_field(desc, base_url, target, mc, force=force)
except ic.InterpreterUnavailable:
_queue_for_sweep(post)
applied = "queued"
else:
_store_translation(post, title_res, desc_res, target)
applied = "translated"
else:
# Disabled / no URL / unhealthy → let the sweep apply it later.
_queue_for_sweep(post)
applied = "queued"
await session.commit()
return jsonify({
"id": post.id,
"translation_override": post.translation_override,
"post_title_translated": post.post_title_translated,
"description_translated": post.description_translated,
"translated_source_lang": post.translated_source_lang,
"applied": applied,
})
+195 -2
View File
@@ -1,12 +1,24 @@
"""Settings API: import filters, system stats.""" """Settings API: import filters, system stats."""
import asyncio
import secrets import secrets
from quart import Blueprint, jsonify, request from quart import Blueprint, jsonify, request
from sqlalchemy import func, select from sqlalchemy import func, or_, select
from ..extensions import get_session from ..extensions import get_session
from ..models import AppSetting, Artist, ImageRecord, ImportBatch, ImportSettings, ImportTask, Tag from ..models import (
AppSetting,
Artist,
ImageRecord,
ImportBatch,
ImportSettings,
ImportTask,
Post,
Tag,
TaskRun,
)
from ..services import interpreter_client as ic
settings_bp = Blueprint("settings", __name__, url_prefix="/api") settings_bp = Blueprint("settings", __name__, url_prefix="/api")
@@ -32,6 +44,10 @@ _EDITABLE_FIELDS = (
"extdl_mediafire_enabled", "extdl_mediafire_enabled",
"extdl_dropbox_enabled", "extdl_dropbox_enabled",
"extdl_pixeldrain_enabled", "extdl_pixeldrain_enabled",
"translation_enabled",
"interpreter_base_url",
"translation_target_lang",
"translation_min_confidence",
) )
# Per-host external-download toggles — all plain booleans, validated uniformly. # Per-host external-download toggles — all plain booleans, validated uniformly.
@@ -69,6 +85,10 @@ async def get_import_settings():
"extdl_mediafire_enabled": row.extdl_mediafire_enabled, "extdl_mediafire_enabled": row.extdl_mediafire_enabled,
"extdl_dropbox_enabled": row.extdl_dropbox_enabled, "extdl_dropbox_enabled": row.extdl_dropbox_enabled,
"extdl_pixeldrain_enabled": row.extdl_pixeldrain_enabled, "extdl_pixeldrain_enabled": row.extdl_pixeldrain_enabled,
"translation_enabled": row.translation_enabled,
"interpreter_base_url": row.interpreter_base_url,
"translation_target_lang": row.translation_target_lang,
"translation_min_confidence": row.translation_min_confidence,
}) })
@@ -128,6 +148,23 @@ async def update_import_settings():
for tog in _EXTDL_TOGGLE_FIELDS: for tog in _EXTDL_TOGGLE_FIELDS:
if tog in body and not isinstance(body[tog], bool): if tog in body and not isinstance(body[tog], bool):
return jsonify({"error": f"{tog} must be a boolean"}), 400 return jsonify({"error": f"{tog} must be a boolean"}), 400
# Translation (#143): base URL may be empty (feature off until set — no
# default host; the operator points it at their own Interpreter proxy).
if "translation_enabled" in body and not isinstance(
body["translation_enabled"], bool
):
return jsonify({"error": "translation_enabled must be a boolean"}), 400
for key in ("interpreter_base_url", "translation_target_lang"):
if key in body and not isinstance(body[key], str):
return jsonify({"error": f"{key} must be a string"}), 400
# Acceptance floor (milestone 155): latin-script translations below this
# Interpreter confidence are kept as the original.
if "translation_min_confidence" in body:
v = body["translation_min_confidence"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
return jsonify(
{"error": "translation_min_confidence must be a number in [0, 1]"}
), 400
if "series_suggest_threshold" in body: if "series_suggest_threshold" in body:
v = body["series_suggest_threshold"] v = body["series_suggest_threshold"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1: if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
@@ -270,3 +307,159 @@ async def rotate_extension_api_key():
row.value = new_value row.value = new_value
await session.commit() await session.commit()
return jsonify({"key": new_value}) return jsonify({"key": new_value})
# --- Translation (#143): live status + manual "Translate now" --------------
@settings_bp.route("/settings/translation/status", methods=["GET"])
async def translation_status():
"""For the Settings card: is it on, is a URL set, is the service reachable,
and how many posts still await translation. Health runs the sync client in a
thread so the event loop isn't blocked."""
translation_tasks = (
"backend.app.tasks.translation.translate_posts",
"backend.app.tasks.translation.retranslate_posts",
)
async with get_session() as session:
cfg = await ImportSettings.load(session)
untranslated = (await session.execute(
select(func.count(Post.id))
.where(Post.translated_source_lang.is_(None))
.where(or_(
Post.post_title.is_not(None), Post.description.is_not(None),
))
)).scalar_one()
# Live progress: is a sweep running now, and what did the last one do?
# (run-until-done re-enqueues itself, so `active` stays true across a
# bulk re-translate; `last_run` surfaces a completed run's outcome.)
active = (await session.execute(
select(func.count(TaskRun.id))
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.status == "running")
)).scalar_one()
last = (await session.execute(
select(TaskRun.task_name, TaskRun.status, TaskRun.finished_at)
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.finished_at.is_not(None))
.order_by(TaskRun.finished_at.desc())
.limit(1)
)).first()
base_url = (cfg.interpreter_base_url or "").strip()
healthy = await asyncio.to_thread(ic.health, base_url) if base_url else False
return jsonify({
"enabled": cfg.translation_enabled,
"base_url_set": bool(base_url),
"healthy": healthy,
"untranslated_count": int(untranslated),
"active": int(active) > 0,
"last_run": {
"task": last[0].rsplit(".", 1)[-1],
"status": last[1],
"finished_at": last[2].isoformat() if last[2] else None,
} if last else None,
})
@settings_bp.route("/settings/translation/test", methods=["POST"])
async def translation_test():
"""On-demand reachability check for a GIVEN Interpreter base URL (the Settings
'Test connection' button) — pings /v1/health without saving, so the operator
can verify a URL before enabling. Health runs in a thread (sync client)."""
body = await request.get_json()
base_url = ""
if isinstance(body, dict):
base_url = (body.get("base_url") or "").strip()
healthy = await asyncio.to_thread(ic.health, base_url) if base_url else False
return jsonify({"healthy": healthy})
@settings_bp.route("/settings/translation/probe", methods=["POST"])
async def translation_probe():
"""Diagnostic for the Settings 'Test translation' box: translate a pasted
snippet WITHOUT saving anything, returning what Interpreter *detected*
(language + confidence) alongside the result. Lets the operator see why a
given string was (mis-)detected — e.g. a short English title flagged as
another language — so a detection guard can be tuned from real numbers.
Read-only: no post is touched. Uses the currently-saved base URL + target."""
body = await request.get_json(silent=True)
body = body if isinstance(body, dict) else {}
text = (body.get("text") or "").strip()
if not text:
return jsonify({"error": "provide text to translate"}), 400
async with get_session() as session:
cfg = await ImportSettings.load(session)
base_url = (cfg.interpreter_base_url or "").strip()
if not base_url:
return jsonify({"error": "no Interpreter base URL is set"}), 400
target = (cfg.translation_target_lang or "en").strip() or "en"
try:
res = await asyncio.to_thread(
ic.translate, [text], base_url=base_url, target=target,
)
except ic.InterpreterUnavailable as e:
return jsonify({"error": f"Interpreter unavailable: {e}"}), 503
except ic.InterpreterBadRequest as e:
return jsonify({"error": f"Interpreter rejected the request: {e}"}), 400
translations = res.get("translations") or []
return jsonify({
"target": target,
"detected_lang": res.get("detected_lang"),
"detected_confidence": res.get("detected_confidence"),
"engine": res.get("engine"),
"engine_version": res.get("engine_version"),
"translated": translations[0] if translations else None,
})
@settings_bp.route("/settings/translation/run", methods=["POST"])
async def translation_run():
"""Enqueue the translate sweep now (the Settings 'Translate now' button).
Runs in drain mode — run-until-done — so one press chases the whole
untranslated backlog to zero rather than a single 300-post chunk."""
async with get_session() as session:
cfg = await ImportSettings.load(session)
if not cfg.translation_enabled or not (cfg.interpreter_base_url or "").strip():
return jsonify(
{"error": "translation is disabled or no base URL is set"}
), 400
from ..tasks.translation import translate_posts
r = translate_posts.delay(drain=True)
return jsonify({"celery_task_id": r.id}), 202
@settings_bp.route("/settings/translation/retranslate", methods=["POST"])
async def translation_retranslate():
"""Re-translate stored translations after a model change (m146). Body:
``{"artist_id": <int>}`` aims at one artist; ``{"all": true}`` re-runs every
artist. ``all`` must be explicit so an empty/typo body can't wipe everything.
Clears the scoped translations and enqueues the run-until-done retranslate
sweep (the Interpreter cache re-translates on a changed model, is cache-fast
otherwise). Same enabled + base-URL guard as 'Translate now'."""
body = await request.get_json(silent=True)
body = body if isinstance(body, dict) else {}
artist_id = body.get("artist_id")
do_all = bool(body.get("all"))
if artist_id is None and not do_all:
return jsonify(
{"error": "provide artist_id, or all=true to re-translate everything"}
), 400
if artist_id is not None:
try:
artist_id = int(artist_id)
except (TypeError, ValueError):
return jsonify({"error": "artist_id must be an integer"}), 400
async with get_session() as session:
cfg = await ImportSettings.load(session)
if not cfg.translation_enabled or not (cfg.interpreter_base_url or "").strip():
return jsonify(
{"error": "translation is disabled or no base URL is set"}
), 400
from ..tasks.translation import retranslate_posts
# artist_id wins when both are sent; otherwise all=true → None (every artist).
artist_ids = [artist_id] if artist_id is not None else None
r = retranslate_posts.delay(artist_ids=artist_ids)
return jsonify({"celery_task_id": r.id}), 202
+10 -30
View File
@@ -11,10 +11,11 @@ suggestions_bp = Blueprint("suggestions", __name__, url_prefix="/api")
@suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"]) @suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"])
async def get_suggestions(image_id: int): async def get_suggestions(image_id: int):
# ?min=<float> overrides the configured per-category thresholds so the typed # ?min=<float> overrides the per-head suggest thresholds for INCLUSION. The
# tag-input dropdown can surface EVERY stored prediction (min=0), including # rail sends min=0 in its single per-image fetch to get EVERY head (each row
# low-confidence actions/features, in canonical formatting. Omitted → the # still carries above_threshold vs its natural cut), then derives the panel
# curated above-threshold list the Suggestions panel uses. # (above_threshold) and the typed dropdown (all, filtered by text) client-side
# — no second request. Omitted → only above-threshold rows.
override = None override = None
raw_min = request.args.get("min") raw_min = request.args.get("min")
if raw_min is not None: if raw_min is not None:
@@ -36,12 +37,11 @@ async def get_suggestions(image_id: int):
"category": s.category, "category": s.category,
"score": round(s.score, 4), "score": round(s.score, 4),
"source": s.source, "source": s.source,
"creates_new_tag": s.creates_new_tag, # whether the score cleared the head's own suggest cut.
# raw model key (alias is stored under this) + whether an # The single min=0 fetch returns every head; the panel
# operator alias produced this suggestion — drive the # shows above_threshold, the typed dropdown shows all and
# modal's "Treat as alias"/"Remove alias" affordances. # annotates each match with its score.
"raw_name": s.raw_name, "above_threshold": s.above_threshold,
"via_alias": s.via_alias,
# operator dismissed this tag for this image — surfaced # operator dismissed this tag for this image — surfaced
# (not dropped) so the rail can show it rejected + offer # (not dropped) so the rail can show it rejected + offer
# one-click un-reject. # one-click un-reject.
@@ -73,26 +73,6 @@ async def accept_suggestion(image_id: int):
return jsonify({"accepted": True, "tag_id": tag_id}) return jsonify({"accepted": True, "tag_id": tag_id})
@suggestions_bp.route(
"/images/<int:image_id>/suggestions/alias", methods=["POST"]
)
async def alias_suggestion(image_id: int):
body = await request.get_json()
required = {"alias_string", "alias_category", "canonical_tag_id"}
if not body or not required.issubset(body):
return jsonify({"error": f"required: {sorted(required)}"}), 400
canonical_tag_id = body["canonical_tag_id"]
async with get_session() as session:
await AllowlistService(session).add_alias_and_accept(
image_id,
body["alias_string"],
body["alias_category"],
canonical_tag_id,
)
await session.commit()
return jsonify({"accepted": True, "tag_id": canonical_tag_id})
@suggestions_bp.route( @suggestions_bp.route(
"/images/<int:image_id>/suggestions/dismiss", methods=["POST"] "/images/<int:image_id>/suggestions/dismiss", methods=["POST"]
) )
+39
View File
@@ -35,6 +35,7 @@ def make_celery() -> Celery:
"backend.app.tasks.backup", "backend.app.tasks.backup",
"backend.app.tasks.admin", "backend.app.tasks.admin",
"backend.app.tasks.library_audit", "backend.app.tasks.library_audit",
"backend.app.tasks.translation",
], ],
) )
app.conf.update( app.conf.update(
@@ -63,9 +64,20 @@ def make_celery() -> Celery:
"backend.app.tasks.backup.*": {"queue": "maintenance_long"}, "backend.app.tasks.backup.*": {"queue": "maintenance_long"},
"backend.app.tasks.admin.*": {"queue": "maintenance_long"}, "backend.app.tasks.admin.*": {"queue": "maintenance_long"},
"backend.app.tasks.library_audit.*": {"queue": "maintenance_long"}, "backend.app.tasks.library_audit.*": {"queue": "maintenance_long"},
# Translation backfill hits the LLM (~16s/item) → the long lane so it
# never starves the quick self-healing sweeps (#143).
"backend.app.tasks.translation.*": {"queue": "maintenance_long"},
}, },
# Heavy ML tasks need fair dispatch — see ImageRepo's precedent. # Heavy ML tasks need fair dispatch — see ImageRepo's precedent.
task_acks_late=True, task_acks_late=True,
# Deploy graceful-shutdown safety: with acks_late, a task killed because
# it outran the container's stop-grace window (SIGKILL) is re-queued
# rather than silently lost. Safe because our long tasks are idempotent +
# chunked (translation per-post commit, downloads terminal-status, audits
# chunk) and the 5-min recovery sweeps re-drive anything left non-terminal
# — a re-run resumes cleanly and never corrupts. No redeliver-loop risk:
# heavy GPU work is tombstoned via gpu_queue, not run inline in a worker.
task_reject_on_worker_lost=True,
worker_prefetch_multiplier=1, worker_prefetch_multiplier=1,
# Broker resilience (2026-06-24): a swarm overlay-network blip after a # Broker resilience (2026-06-24): a swarm overlay-network blip after a
# redeploy left Redis healthy but transiently unreachable, and a worker # redeploy left Redis healthy but transiently unreachable, and a worker
@@ -103,6 +115,11 @@ def make_celery() -> Celery:
"task": "backend.app.tasks.maintenance.cleanup_old_tasks", "task": "backend.app.tasks.maintenance.cleanup_old_tasks",
"schedule": 86400.0, # daily "schedule": 86400.0, # daily
}, },
"cleanup-orphaned-temp-files": {
"task": "backend.app.tasks.maintenance.cleanup_orphaned_temp_files",
"schedule": 86400.0, # daily — sweep .part/.partial left by a
# download/import killed mid-write (graceful-shutdown fallout)
},
"train-heads-nightly": { "train-heads-nightly": {
"task": "backend.app.tasks.ml.scheduled_train_heads", "task": "backend.app.tasks.ml.scheduled_train_heads",
"schedule": 86400.0, # passive cadence; manual retrain stays available "schedule": 86400.0, # passive cadence; manual retrain stays available
@@ -147,6 +164,28 @@ def make_celery() -> Celery:
"task": "backend.app.tasks.ml.scheduled_ccip_auto_apply", "task": "backend.app.tasks.ml.scheduled_ccip_auto_apply",
"schedule": 86400.0, # no-op unless ccip_auto_apply_enabled "schedule": 86400.0, # no-op unless ccip_auto_apply_enabled
}, },
"retract-auto-tags-daily": {
"task": "backend.app.tasks.ml.scheduled_retract_auto_tags",
"schedule": 86400.0, # soft auto-apply: drop auto-tags now below
# their threshold (m139); no-op unless the auto-apply switch is on
},
"presentation-auto-apply-daily": {
"task": "backend.app.tasks.ml.scheduled_presentation_auto_apply",
"schedule": 86400.0, # auto-hide banner/editor chrome (#141);
# no-op unless presentation_auto_apply_enabled
},
"prune-presentation-reviews-daily": {
"task": "backend.app.tasks.ml.prune_presentation_reviews",
"schedule": 86400.0, # retention: drop resolved review flags >30d
},
"translate-posts-8h": {
"task": "backend.app.tasks.translation.translate_posts",
"schedule": 28800.0, # every 8h: steady-state cadence for the
# trickle of newly-imported posts (no-op unless translation
# configured + healthy). One bounded 300-chunk per fire — the
# one-time backlog drains via the "Translate now" button
# (drain=True, run-until-done), not this sweep.
},
"snapshot-head-metrics-daily": { "snapshot-head-metrics-daily": {
"task": "backend.app.tasks.maintenance.snapshot_head_metrics", "task": "backend.app.tasks.maintenance.snapshot_head_metrics",
"schedule": 86400.0, "schedule": 86400.0,
+2
View File
@@ -28,6 +28,7 @@ from .pixiv_failed_media import PixivFailedMedia
from .pixiv_seen_media import PixivSeenMedia from .pixiv_seen_media import PixivSeenMedia
from .post import Post from .post import Post
from .post_attachment import PostAttachment from .post_attachment import PostAttachment
from .presentation_review import PresentationReview
from .series_chapter import SeriesChapter from .series_chapter import SeriesChapter
from .series_page import SeriesPage from .series_page import SeriesPage
from .series_suggestion import SeriesSuggestion from .series_suggestion import SeriesSuggestion
@@ -57,6 +58,7 @@ __all__ = [
"SubscribeStarSeenMedia", "SubscribeStarSeenMedia",
"Post", "Post",
"PostAttachment", "PostAttachment",
"PresentationReview",
"SeriesChapter", "SeriesChapter",
"SeriesPage", "SeriesPage",
"SeriesSuggestion", "SeriesSuggestion",
+24
View File
@@ -92,6 +92,30 @@ class ImportSettings(Base):
Boolean, nullable=False, default=True, server_default="true", Boolean, nullable=False, default=True, server_default="true",
) )
# -- Post-text translation via the Interpreter LAN service (milestone 143).
# Off by default with NO default host — it needs a reachable Interpreter
# service (the operator's, behind a reverse proxy), which not every install
# has; the operator sets the URL and flips it on. Empty base_url OR disabled
# → the translate sweep no-ops.
translation_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="false",
)
interpreter_base_url: Mapped[str] = mapped_column(
Text, nullable=False, default="", server_default="",
)
translation_target_lang: Mapped[str] = mapped_column(
Text, nullable=False, default="en", server_default="en",
)
# The latin-script acceptance floor for the translation gate: a translation
# whose Interpreter-reported confidence is below this is kept as the original
# (operator-tunable, milestone 155). Default 0.9 — stricter than the old
# hardcoded 0.8, because Interpreter confidently mis-detects short ASCII
# English (e.g. "… WIP Part 1") as a European language at ~0.86. CJK stays
# trusted regardless (script-detected). Per-post overrides handle the misses.
translation_min_confidence: Mapped[float] = mapped_column(
Float, nullable=False, default=0.9, server_default="0.9",
)
@classmethod @classmethod
async def load(cls, session) -> ImportSettings: async def load(cls, session) -> ImportSettings:
"""The singleton settings row (id=1), via an async session.""" """The singleton settings row (id=1), via an async session."""
+22 -2
View File
@@ -63,7 +63,9 @@ class MLSettings(Base):
Boolean, nullable=False, default=True Boolean, nullable=False, default=True
) )
head_auto_apply_min_positives: Mapped[int] = mapped_column( head_auto_apply_min_positives: Mapped[int] = mapped_column(
Integer, nullable=False, default=30 # Support floor raised 30→50 (operator-asked 2026-07-06): a head needs
# more human labels before it may fire without a human.
Integer, nullable=False, default=50
) )
# CCIP character-match cosine cut (#114). 0.85 default — the v1 flat 0.75 # CCIP character-match cosine cut (#114). 0.85 default — the v1 flat 0.75
# over-fired (high-reference characters matched a scatter of images); 0.85 # over-fired (high-reference characters matched a scatter of images); 0.85
@@ -78,7 +80,25 @@ class MLSettings(Base):
Boolean, nullable=False, default=True Boolean, nullable=False, default=True
) )
ccip_auto_apply_threshold: Mapped[float] = mapped_column( ccip_auto_apply_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.92 # Raised 0.92→0.95 (operator-asked 2026-07-06) so only very confident
# character matches auto-tag.
Float, nullable=False, default=0.95
)
# -- Presentation chrome auto-hide (#141) -------------------------------
# banner / editor screenshot auto-apply on the sweep with their OWN flat
# threshold (decoupled from content-head graduation). Hiding is consequential
# so it runs HIGH. `wip` is never auto-applied. When an image would be
# auto-hidden but ALSO scores >= presentation_conflict_threshold on a content
# head, it's still hidden but flagged for review (PresentationReview) instead
# of buried silently. ON by default (opt-out); every auto-tag is reversible.
presentation_auto_apply_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True
)
presentation_auto_apply_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.90
)
presentation_conflict_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.50
) )
# Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069); # Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069);
# existing libraries keep their stored value until the operator re-embeds. # existing libraries keep their stored value until the operator re-embeds.
+43 -1
View File
@@ -8,7 +8,17 @@ artist-filter queries don't depend on the Source detour).
from datetime import datetime from datetime import datetime
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func from sqlalchemy import (
JSON,
CheckConstraint,
DateTime,
ForeignKey,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from .base import Base from .base import Base
@@ -23,6 +33,10 @@ class Post(Base):
# (created in alembic 0030) covers that case via # (created in alembic 0030) covers that case via
# (artist_id, external_post_id). # (artist_id, external_post_id).
UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"), UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"),
CheckConstraint(
"translation_override IN ('auto', 'force', 'original')",
name="ck_post_translation_override",
),
) )
id: Mapped[int] = mapped_column(Integer, primary_key=True) id: Mapped[int] = mapped_column(Integer, primary_key=True)
@@ -47,6 +61,34 @@ class Post(Base):
description: Mapped[str | None] = mapped_column(Text, nullable=True) description: Mapped[str | None] = mapped_column(Text, nullable=True)
attachment_count: Mapped[int | None] = mapped_column(Integer, nullable=True) attachment_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
# -- Post-text translation (milestone 143). Filled by the translate_posts
# sweep via the Interpreter LAN service so viewing is instant.
# translated_source_lang is the DETECTED original language; "en" (or a
# passthrough) means nothing to translate and the *_translated columns stay
# NULL. engine_version keys the Interpreter cache — re-runs are ~1ms and a
# model upgrade re-translates instead of serving stale.
post_title_translated: Mapped[str | None] = mapped_column(Text, nullable=True)
description_translated: Mapped[str | None] = mapped_column(Text, nullable=True)
translated_source_lang: Mapped[str | None] = mapped_column(
String(8), nullable=True
)
translation_engine_version: Mapped[str | None] = mapped_column(
String(128), nullable=True
)
translated_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
# Sticky per-post override of the translation decision (milestone 155):
# 'auto' = the acceptance gate decides; 'force' = always store Interpreter's
# translation even below the confidence floor (rescue a skipped legit-foreign
# title); 'original' = never translate, keep the original (kill a confidently
# mis-flagged one the floor can't catch). The sweep reads this on every run,
# and re-translate leaves 'original' posts alone, so the choice survives a
# Re-translate-all.
translation_override: Mapped[str] = mapped_column(
String(16), nullable=False, default="auto", server_default="auto",
)
downloaded_at: Mapped[datetime] = mapped_column( downloaded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
) )
+40
View File
@@ -0,0 +1,40 @@
"""PresentationReview — an auto-hidden presentation tag that ALSO looked like
real content, flagged for operator review (milestone 141).
When the auto-apply sweep hides an image as chrome (banner / editor screenshot)
but the image ALSO scores highly on a content head, it still hides it but records
this row so the Hidden view can surface it ("⚠ also looks like <conflict tag>")
for a keep-hidden / un-hide decision. Resolved rows are pruned by retention.
"""
from datetime import datetime
from sqlalchemy import DateTime, Float, ForeignKey, func
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class PresentationReview(Base):
__tablename__ = "presentation_review"
image_record_id: Mapped[int] = mapped_column(
ForeignKey("image_record.id", ondelete="CASCADE"), primary_key=True
)
# The presentation tag that was auto-applied (banner / editor screenshot).
tag_id: Mapped[int] = mapped_column(
ForeignKey("tag.id", ondelete="CASCADE"), primary_key=True
)
# The content tag the image ALSO scored high on — the "concerning" signal.
# SET NULL (not CASCADE): losing the conflict tag shouldn't erase the flag.
conflict_tag_id: Mapped[int | None] = mapped_column(
ForeignKey("tag.id", ondelete="SET NULL"), nullable=True
)
conflict_score: Mapped[float] = mapped_column(Float, nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now()
)
# Set when the operator keeps-hidden or un-hides; retention prunes resolved.
resolved_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
+4
View File
@@ -48,6 +48,10 @@ class TagKind(StrEnum):
# content. `wip` is real art: only the training pipelines exclude it. # content. `wip` is real art: only the training pipelines exclude it.
SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot") SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot")
PRESENTATION_SYSTEM_TAGS = ("banner", "editor screenshot") PRESENTATION_SYSTEM_TAGS = ("banner", "editor screenshot")
# `wip` marks real-but-unfinished art. It's kept in the gallery's own "similar"
# results (#1274), but the Explore rabbit-hole opts to hide it (exclude_wip) so a
# browse doesn't keep surfacing work-in-progress (operator, 2026-07-08).
WIP_SYSTEM_TAG = "wip"
image_tag = Table( image_tag = Table(
"image_tag", "image_tag",
+77 -11
View File
@@ -22,8 +22,16 @@ from sqlalchemy import Select, and_, distinct, exists, func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import aliased from sqlalchemy.orm import aliased
from ..models import Artist, ImageProvenance, ImageRecord, Post, Source, Tag from ..models import (
from ..models.tag import PRESENTATION_SYSTEM_TAGS, image_tag Artist,
ImageProvenance,
ImageRecord,
Post,
Source,
Tag,
TagPositiveConfirmation,
)
from ..models.tag import PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG, image_tag
from .pagination import decode_cursor, encode_cursor from .pagination import decode_cursor, encode_cursor
from .tag_query import ( from .tag_query import (
fandom_join_alias, fandom_join_alias,
@@ -179,7 +187,7 @@ def _apply_scope(
stmt, *, tag_ids, post_id, artist_id, media_type, stmt, *, tag_ids, post_id, artist_id, media_type,
tag_or_groups=None, tag_exclude=None, tag_or_groups=None, tag_exclude=None,
platform=None, untagged=False, no_artist=False, platform=None, untagged=False, no_artist=False,
date_from=None, date_to=None, date_from=None, date_to=None, hidden_tag_ids=None,
): ):
"""Apply the composable gallery filters to a statement. """Apply the composable gallery filters to a statement.
@@ -216,6 +224,12 @@ def _apply_scope(
stmt = stmt.where(image_in_any_tag_scope(group)) stmt = stmt.where(image_in_any_tag_scope(group))
if tag_exclude: if tag_exclude:
stmt = stmt.where(~image_in_any_tag_scope(tag_exclude)) stmt = stmt.where(~image_in_any_tag_scope(tag_exclude))
# Presentation chrome (banner / editor screenshot) is hidden from the default
# gallery — an implicit exclude the caller supplies unless the operator asked
# to include hidden or is explicitly filtering for a presentation tag
# (milestone 141). `wip` is NOT hidden. Resolved to ids by _hidden_tag_ids.
if hidden_tag_ids:
stmt = stmt.where(~image_in_any_tag_scope(hidden_tag_ids))
prov = _provenance_clause(post_id, artist_id) prov = _provenance_clause(post_id, artist_id)
if prov is not None: if prov is not None:
stmt = stmt.where(prov) stmt = stmt.where(prov)
@@ -402,6 +416,31 @@ class GalleryService:
def __init__(self, session: AsyncSession): def __init__(self, session: AsyncSession):
self.session = session self.session = session
async def _hidden_tag_ids(
self, include_hidden, tag_ids, tag_or_groups,
) -> list[int] | None:
"""Presentation-chrome tag ids to implicitly exclude from a gallery query,
or None. None when the caller asked to include hidden, when the operator
is explicitly filtering FOR a presentation tag (they clearly want to see
it), or when no presentation tags exist. (milestone 141)"""
if include_hidden:
return None
rows = await self.session.execute(
select(Tag.id).where(
Tag.is_system.is_(True),
Tag.name.in_(PRESENTATION_SYSTEM_TAGS),
)
)
pres = [r[0] for r in rows]
if not pres:
return None
explicit = set(tag_ids or [])
for group in tag_or_groups or []:
explicit.update(group)
if explicit & set(pres):
return None
return pres
async def scroll( async def scroll(
self, self,
cursor: str | None, cursor: str | None,
@@ -418,12 +457,16 @@ class GalleryService:
no_artist: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_from: datetime | None = None,
date_to: datetime | None = None, date_to: datetime | None = None,
include_hidden: bool = False,
) -> GalleryPage: ) -> GalleryPage:
if limit < 1 or limit > 200: if limit < 1 or limit > 200:
raise ValueError("limit must be between 1 and 200") raise ValueError("limit must be between 1 and 200")
_require_single_filter( _require_single_filter(
tag_ids, post_id, artist_id, tag_or_groups, tag_exclude, tag_ids, post_id, artist_id, tag_or_groups, tag_exclude,
) )
hidden = await self._hidden_tag_ids(
include_hidden, tag_ids, tag_or_groups,
)
# eff is the ACTIVE sort column (effective_date or earliest_post_date); # eff is the ACTIVE sort column (effective_date or earliest_post_date);
# the cursor, ordering and year/month grouping all key off it, so the # the cursor, ordering and year/month grouping all key off it, so the
@@ -436,7 +479,7 @@ class GalleryService:
artist_id=artist_id, media_type=media_type, artist_id=artist_id, media_type=media_type,
tag_or_groups=tag_or_groups, tag_exclude=tag_exclude, tag_or_groups=tag_or_groups, tag_exclude=tag_exclude,
platform=platform, untagged=untagged, no_artist=no_artist, platform=platform, untagged=untagged, no_artist=no_artist,
date_from=date_from, date_to=date_to, date_from=date_from, date_to=date_to, hidden_tag_ids=hidden,
) )
descending = sort not in _ASCENDING_SORTS descending = sort not in _ASCENDING_SORTS
@@ -489,6 +532,7 @@ class GalleryService:
no_artist: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_from: datetime | None = None,
date_to: datetime | None = None, date_to: datetime | None = None,
include_hidden: bool = False,
) -> list[TimelineBucket]: ) -> list[TimelineBucket]:
eff = _effective_date_col() eff = _effective_date_col()
year_col = func.date_part("year", eff).label("yr") year_col = func.date_part("year", eff).label("yr")
@@ -500,12 +544,15 @@ class GalleryService:
_require_single_filter( _require_single_filter(
tag_ids, post_id, artist_id, tag_or_groups, tag_exclude, tag_ids, post_id, artist_id, tag_or_groups, tag_exclude,
) )
hidden = await self._hidden_tag_ids(
include_hidden, tag_ids, tag_or_groups,
)
stmt = _apply_scope( stmt = _apply_scope(
stmt, tag_ids=tag_ids, post_id=post_id, stmt, tag_ids=tag_ids, post_id=post_id,
artist_id=artist_id, media_type=media_type, artist_id=artist_id, media_type=media_type,
tag_or_groups=tag_or_groups, tag_exclude=tag_exclude, tag_or_groups=tag_or_groups, tag_exclude=tag_exclude,
platform=platform, untagged=untagged, no_artist=no_artist, platform=platform, untagged=untagged, no_artist=no_artist,
date_from=date_from, date_to=date_to, date_from=date_from, date_to=date_to, hidden_tag_ids=hidden,
) )
stmt = stmt.group_by(year_col, month_col).order_by(year_col.desc(), month_col.desc()) stmt = stmt.group_by(year_col, month_col).order_by(year_col.desc(), month_col.desc())
rows = (await self.session.execute(stmt)).all() rows = (await self.session.execute(stmt)).all()
@@ -519,7 +566,7 @@ class GalleryService:
tag_exclude: list[int] | None = None, tag_exclude: list[int] | None = None,
platform: str | None = None, untagged: bool = False, platform: str | None = None, untagged: bool = False,
no_artist: bool = False, date_from: datetime | None = None, no_artist: bool = False, date_from: datetime | None = None,
date_to: datetime | None = None, date_to: datetime | None = None, include_hidden: bool = False,
) -> str | None: ) -> str | None:
"""Returns a cursor that, when passed to scroll() with the same sort, """Returns a cursor that, when passed to scroll() with the same sort,
positions at the first image of the given year-month. None if the positions at the first image of the given year-month. None if the
@@ -536,12 +583,15 @@ class GalleryService:
_require_single_filter( _require_single_filter(
tag_ids, post_id, artist_id, tag_or_groups, tag_exclude, tag_ids, post_id, artist_id, tag_or_groups, tag_exclude,
) )
hidden = await self._hidden_tag_ids(
include_hidden, tag_ids, tag_or_groups,
)
stmt = _apply_scope( stmt = _apply_scope(
stmt, tag_ids=tag_ids, post_id=post_id, stmt, tag_ids=tag_ids, post_id=post_id,
artist_id=artist_id, media_type=media_type, artist_id=artist_id, media_type=media_type,
tag_or_groups=tag_or_groups, tag_exclude=tag_exclude, tag_or_groups=tag_or_groups, tag_exclude=tag_exclude,
platform=platform, untagged=untagged, no_artist=no_artist, platform=platform, untagged=untagged, no_artist=no_artist,
date_from=date_from, date_to=date_to, date_from=date_from, date_to=date_to, hidden_tag_ids=hidden,
) )
descending = sort != "oldest" descending = sort != "oldest"
if descending: if descending:
@@ -566,6 +616,7 @@ class GalleryService:
platform: str | None = None, platform: str | None = None,
untagged: bool = False, no_artist: bool = False, untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None, date_from: datetime | None = None, date_to: datetime | None = None,
include_hidden: bool = False,
) -> GalleryFacets: ) -> GalleryFacets:
"""Live facet counts scoped to the current filter. Each facet GROUP is """Live facet counts scoped to the current filter. Each facet GROUP is
computed with all OTHER active filters applied but its OWN selection computed with all OTHER active filters applied but its OWN selection
@@ -576,10 +627,14 @@ class GalleryService:
_require_single_filter( _require_single_filter(
tag_ids, post_id, artist_id, tag_or_groups, tag_exclude, tag_ids, post_id, artist_id, tag_or_groups, tag_exclude,
) )
hidden = await self._hidden_tag_ids(
include_hidden, tag_ids, tag_or_groups,
)
common = { common = {
"tag_ids": tag_ids, "post_id": post_id, "tag_ids": tag_ids, "post_id": post_id,
"artist_id": artist_id, "media_type": media_type, "artist_id": artist_id, "media_type": media_type,
"tag_or_groups": tag_or_groups, "tag_exclude": tag_exclude, "tag_or_groups": tag_or_groups, "tag_exclude": tag_exclude,
"hidden_tag_ids": hidden,
} }
# total — the full active filter (the headline result count). # total — the full active filter (the headline result count).
@@ -660,6 +715,7 @@ class GalleryService:
platform: str | None = None, platform: str | None = None,
untagged: bool = False, no_artist: bool = False, untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None, date_from: datetime | None = None, date_to: datetime | None = None,
exclude_wip: bool = False,
) -> list[GalleryImage] | None: ) -> list[GalleryImage] | None:
"""Visual "more like this": images near `image_id`'s SigLIP embedding """Visual "more like this": images near `image_id`'s SigLIP embedding
(pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result (pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result
@@ -696,14 +752,18 @@ class GalleryService:
# Presentation images (banner / editor-screenshot system tags, #128) # Presentation images (banner / editor-screenshot system tags, #128)
# cluster on UI chrome rather than content, so near any one of them # cluster on UI chrome rather than content, so near any one of them
# they'd fill the grid. Excluded from CANDIDATES only — the anchor # they'd fill the grid. Excluded from CANDIDATES only — the anchor
# itself may be a banner, and `wip` stays surfaced (real art; only # itself may be a banner. `wip` stays surfaced here by default (real art;
# the training pipelines exclude it). # only the training pipelines exclude it), but the Explore rabbit-hole
# passes exclude_wip to also drop work-in-progress (operator, 2026-07-08).
excluded_system_tags = PRESENTATION_SYSTEM_TAGS
if exclude_wip:
excluded_system_tags = (*PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG)
presentation = ( presentation = (
select(image_tag.c.image_record_id) select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
.where( .where(
Tag.is_system.is_(True), Tag.is_system.is_(True),
Tag.name.in_(PRESENTATION_SYSTEM_TAGS), Tag.name.in_(excluded_system_tags),
) )
) )
stmt = stmt.where( stmt = stmt.where(
@@ -731,8 +791,14 @@ class GalleryService:
# Self-join Tag to resolve a character's fandom NAME (not just id) so the # Self-join Tag to resolve a character's fandom NAME (not just id) so the
# modal chip can label it without an N+1 (shared tag_query helpers). # modal chip can label it without an N+1 (shared tag_query helpers).
fandom_alias = fandom_join_alias() fandom_alias = fandom_join_alias()
# source drives the auto-applied badge; confirmed = operator affirmed the
# tag (positive + retraction-shielded, milestone 139).
confirmed = exists().where(
TagPositiveConfirmation.image_record_id == image_id,
TagPositiveConfirmation.tag_id == Tag.id,
).label("confirmed")
tag_stmt = ( tag_stmt = (
select(*tag_columns(fandom_alias)) select(*tag_columns(fandom_alias), image_tag.c.source, confirmed)
.select_from( .select_from(
Tag.__table__ Tag.__table__
.join(image_tag, image_tag.c.tag_id == Tag.id) .join(image_tag, image_tag.c.tag_id == Tag.id)
+152
View File
@@ -0,0 +1,152 @@
"""Interpreter translation client (milestone 143) — a thin SYNC wrapper over the
self-hosted Interpreter LAN service (LibreTranslate-compatible `/v1/translate`).
Sync (requests) because the only caller is the sync celery translate sweep, and
it mirrors FC's other platform clients. The service knows nothing about Curator —
all Curator logic stays here. base_url is operator-configured (empty until set;
behind a reverse proxy). Full API contract: Scribe note #1347.
"""
from __future__ import annotations
from datetime import UTC, datetime
from email.utils import parsedate_to_datetime
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
class InterpreterUnavailable(Exception):
"""The translation engine is down / unreachable / draining — a connection
error, HTTP 429, or a 5xx (commonly 502/503/504 through a reverse proxy while
the service restarts). Retry later, don't drop the item. ``retry_after`` holds
the server's Retry-After hint in seconds when it sent one, so the caller can
back off exactly as long as it's asked to."""
def __init__(self, message: str, *, retry_after: float | None = None):
super().__init__(message)
self.retry_after = retry_after
class InterpreterBadRequest(Exception):
"""Bad request params (HTTP 400)."""
def _url(base_url: str, path: str) -> str:
return f"{base_url.rstrip('/')}{path}"
def _parse_retry_after(resp) -> float | None:
"""Parse a Retry-After header (RFC 7231: delta-seconds or an HTTP-date) into
non-negative seconds, or None if absent/unparseable — so a gracefully-
draining Interpreter can tell Curator exactly how long to wait before it
tries again."""
raw = (resp.headers.get("Retry-After") or "").strip()
if not raw:
return None
try:
return max(0.0, float(int(raw))) # delta-seconds form
except ValueError:
pass
try: # HTTP-date form
when = parsedate_to_datetime(raw)
except (TypeError, ValueError):
return None
if when is None:
return None
if when.tzinfo is None:
when = when.replace(tzinfo=UTC)
return max(0.0, (when - datetime.now(UTC)).total_seconds())
# A shared session pools the keep-alive connection across the per-post sweep
# calls, and retries CONNECT failures only (connect=2, short backoff) — smoothing
# the instant a reverse proxy reloads. Status codes are deliberately NOT retried
# (status=0, raise_on_status=False): translate() maps 429/5xx → InterpreterUnavailable
# itself, and letting urllib3 retry a draining 503 would defeat the Retry-After
# backoff we honour upstream.
_retry = Retry(
total=None, connect=2, read=0, redirect=0, status=0,
backoff_factor=0.3, raise_on_status=False,
)
session = requests.Session()
_adapter = HTTPAdapter(max_retries=_retry)
session.mount("http://", _adapter)
session.mount("https://", _adapter)
def health(base_url: str, *, timeout: float = 5.0) -> bool:
"""True iff the Interpreter LLM engine is up. Any error (unset URL, network,
non-200, engine down) → False, so the sweep just no-ops rather than raising."""
if not base_url:
return False
try:
r = session.get(_url(base_url, "/v1/health"), timeout=timeout)
except requests.RequestException:
return False
if r.status_code != 200:
return False
engines = (r.json() or {}).get("engines") or {}
return bool(engines.get("llm"))
def translate(
texts: list[str], *, base_url: str, target: str = "en",
source: str = "auto", timeout: float = 120.0,
) -> dict:
"""Translate a batch. Returns::
{"translations": [str, ...], # SAME order & length as `texts`
"detected_lang": str | None, # aggregate: describes the FIRST item
"detected_confidence": float | None, # detector's confidence, if given
"engine": str | None,
"engine_version": str | None}
Interpreter batch metadata is aggregate (first item only) — fine for one
post's ``[title, description]`` batch since they share a language. Passthrough
items (already target-language / emoji-only) come back UNCHANGED in their
slot. Raises InterpreterUnavailable on a connection error / 429 / 5xx (retry
later, honouring Retry-After), InterpreterBadRequest on 400. (Scribe note
#1347.)
"""
if not texts:
return {"translations": [], "detected_lang": None,
"detected_confidence": None,
"engine": None, "engine_version": None}
try:
r = session.post(
_url(base_url, "/v1/translate"),
json={"q": list(texts), "source": source,
"target": target, "engine": "auto"},
timeout=timeout,
)
except requests.RequestException as e:
raise InterpreterUnavailable(str(e)) from e
if r.status_code == 400:
raise InterpreterBadRequest((r.json() or {}).get("error", "bad request"))
# A gracefully-draining service — often behind a reverse proxy — returns 429
# or a 5xx gateway error (502/503/504), not always a clean 503. Treat every
# one as "unavailable, retry later" and honour any Retry-After it sends, so a
# rolling Interpreter restart interrupts the sweep cleanly (instead of raising
# an opaque HTTPError) and resumes exactly when the service says it's ready.
if r.status_code == 429 or r.status_code >= 500:
raise InterpreterUnavailable(
f"interpreter unavailable (HTTP {r.status_code})",
retry_after=_parse_retry_after(r),
)
r.raise_for_status()
body = r.json() or {}
out = body.get("translatedText")
# q was an array → translatedText is an array (same order & length). Guard a
# scalar reply just in case (shouldn't happen for an array q).
if not isinstance(out, list):
out = [out]
interp = body.get("interpreter") or {}
detected = body.get("detectedLanguage") or {}
return {
"translations": out,
"detected_lang": detected.get("language"),
"detected_confidence": detected.get("confidence"),
"engine": interp.get("engine"),
"engine_version": interp.get("engine_version"),
}
-14
View File
@@ -12,13 +12,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
from ...models import TagSuggestionRejection from ...models import TagSuggestionRejection
from ...models.tag import image_tag from ...models.tag import image_tag
from .aliases import AliasService
class AllowlistService: class AllowlistService:
def __init__(self, session: AsyncSession): def __init__(self, session: AsyncSession):
self.session = session self.session = session
self.aliases = AliasService(session)
async def _apply_image_tag(self, image_id: int, tag_id: int, source: str): async def _apply_image_tag(self, image_id: int, tag_id: int, source: str):
stmt = insert(image_tag).values( stmt = insert(image_tag).values(
@@ -41,18 +39,6 @@ class AllowlistService:
await self._apply_image_tag(image_id, tag_id, source="ml_accepted") await self._apply_image_tag(image_id, tag_id, source="ml_accepted")
await self._clear_rejection(image_id, tag_id) await self._clear_rejection(image_id, tag_id)
async def add_alias_and_accept(
self,
image_id: int,
alias_string: str,
alias_category: str,
canonical_tag_id: int,
) -> None:
await self.aliases.create(
alias_string, alias_category, canonical_tag_id
)
await self.accept(image_id, canonical_tag_id)
async def dismiss(self, image_id: int, tag_id: int) -> None: async def dismiss(self, image_id: int, tag_id: int) -> None:
stmt = insert(TagSuggestionRejection).values( stmt = insert(TagSuggestionRejection).values(
image_record_id=image_id, tag_id=tag_id image_record_id=image_id, tag_id=tag_id
+15 -1
View File
@@ -13,7 +13,7 @@ exact CCIP difference metric/threshold gets validated against the model during
the hands-on eval. numpy is imported lazily (API worker has it via pgvector). the hands-on eval. numpy is imported lazily (API worker has it via pgvector).
""" """
from sqlalchemy import func, select from sqlalchemy import exists, func, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from ...models import ( from ...models import (
@@ -23,8 +23,10 @@ from ...models import (
MLSettings, MLSettings,
Tag, Tag,
TagKind, TagKind,
TagPositiveConfirmation,
) )
from ...models.tag import image_tag from ...models.tag import image_tag
from .training_data import _AUTO_SOURCES
# Cosine-similarity floor to call a figure the same character. The live setting # Cosine-similarity floor to call a figure the same character. The live setting
# (ml_settings.ccip_match_threshold) drives it; this is only the fallback when no # (ml_settings.ccip_match_threshold) drives it; this is only the fallback when no
@@ -111,6 +113,17 @@ async def _ref_signature(session: AsyncSession) -> tuple:
return (n_tags, n_regs, max_id, n_hygiene) return (n_tags, n_regs, max_id, n_hygiene)
def _positive_char_tag():
"""Condition on the joined character image_tag: HUMAN-applied or operator-
confirmed — NOT an unconfirmed auto-apply. Keeps an auto-tagged character from
self-seeding CCIP references, so a ccip_auto misfire can't reinforce itself
(milestone 139) — mirrors the head-training positive exclusion."""
return image_tag.c.source.not_in(_AUTO_SOURCES) | exists().where(
TagPositiveConfirmation.image_record_id == image_tag.c.image_record_id,
TagPositiveConfirmation.tag_id == image_tag.c.tag_id,
)
async def character_references(session: AsyncSession) -> dict[int, list]: async def character_references(session: AsyncSession) -> dict[int, list]:
"""Per character-tag CCIP reference vectors: figure/face-region CCIP """Per character-tag CCIP reference vectors: figure/face-region CCIP
embeddings on UNAMBIGUOUS (single-character) images carrying that tag. embeddings on UNAMBIGUOUS (single-character) images carrying that tag.
@@ -128,6 +141,7 @@ async def character_references(session: AsyncSession) -> dict[int, list]:
) )
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.kind == TagKind.character) .where(Tag.kind == TagKind.character)
.where(_positive_char_tag())
.where(ImageRegion.kind.in_(_FIGURE_KINDS)) .where(ImageRegion.kind.in_(_FIGURE_KINDS))
.where(ImageRegion.ccip_embedding.is_not(None)) .where(ImageRegion.ccip_embedding.is_not(None))
.where(ImageRegion.image_record_id.in_(_single_character_images())) .where(ImageRegion.image_record_id.in_(_single_character_images()))
@@ -31,9 +31,16 @@ from ...models import (
MLSettings, MLSettings,
Tag, Tag,
TagKind, TagKind,
TagPositiveConfirmation,
) )
from ...models.tag import image_tag from ...models.tag import image_tag
from .ccip import _FIGURE_KINDS, _hygiene_tagged_images, _single_character_images from .ccip import (
_FIGURE_KINDS,
_hygiene_tagged_images,
_l2norm,
_positive_char_tag,
_single_character_images,
)
# Deterministic per-tag capping so a rebuild of an UNCHANGED reference set # Deterministic per-tag capping so a rebuild of an UNCHANGED reference set
# resamples identically (stable prototypes, no churn between refreshes). # resamples identically (stable prototypes, no churn between refreshes).
@@ -63,7 +70,16 @@ def _global_signature(session: Session) -> str:
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.is_system.is_(True)) .where(Tag.is_system.is_(True))
).scalar_one() ).scalar_one()
return f"{n_tags}:{n_regs}:{max_id or 0}:{n_hygiene}" # Character confirmations affect the reference set now that auto-tags only
# seed references once confirmed (milestone 139) — so a confirm must trip the
# gate, or the per-character diff (which reflects it) never runs.
n_conf = session.execute(
select(func.count())
.select_from(TagPositiveConfirmation)
.join(Tag, Tag.id == TagPositiveConfirmation.tag_id)
.where(Tag.kind == TagKind.character)
).scalar_one()
return f"{n_tags}:{n_regs}:{max_id or 0}:{n_hygiene}:{n_conf}"
def _current_fingerprints(session: Session) -> dict[int, str]: def _current_fingerprints(session: Session) -> dict[int, str]:
@@ -84,6 +100,7 @@ def _current_fingerprints(session: Session) -> dict[int, str]:
) )
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
.where(Tag.kind == TagKind.character) .where(Tag.kind == TagKind.character)
.where(_positive_char_tag())
.where(ImageRegion.kind.in_(_FIGURE_KINDS)) .where(ImageRegion.kind.in_(_FIGURE_KINDS))
.where(ImageRegion.ccip_embedding.is_not(None)) .where(ImageRegion.ccip_embedding.is_not(None))
.where(ImageRegion.image_record_id.in_(_single_character_images())) .where(ImageRegion.image_record_id.in_(_single_character_images()))
@@ -104,6 +121,7 @@ def _rebuild_one(session: Session, tag_id: int, cap: int) -> int:
image_tag.c.image_record_id == ImageRegion.image_record_id, image_tag.c.image_record_id == ImageRegion.image_record_id,
) )
.where(image_tag.c.tag_id == tag_id) .where(image_tag.c.tag_id == tag_id)
.where(_positive_char_tag())
.where(ImageRegion.kind.in_(_FIGURE_KINDS)) .where(ImageRegion.kind.in_(_FIGURE_KINDS))
.where(ImageRegion.ccip_embedding.is_not(None)) .where(ImageRegion.ccip_embedding.is_not(None))
.where(ImageRegion.image_record_id.in_(_single_character_images())) .where(ImageRegion.image_record_id.in_(_single_character_images()))
@@ -173,3 +191,76 @@ def refresh_character_prototypes(
settings.ccip_ref_signature = sig settings.ccip_ref_signature = sig
session.commit() session.commit()
return {"skipped": False, "rebuilt": rebuilt, "removed": removed} return {"skipped": False, "rebuilt": rebuilt, "removed": removed}
def retract_auto_applied_ccip(session: Session) -> int:
"""Soft auto-apply for CCIP character tags (milestone 139): re-score every
standing source='ccip_auto' character tag against that character's prototypes
and REMOVE the ones whose best figure match is now BELOW
ccip_auto_apply_threshold. Skips operator-confirmed tags. SILENT — a low score
isn't proof the tag was wrong (that's reserved for an operator removal). No-op
unless ccip_auto_apply_enabled. A character with no prototypes yet, or an image
with no figure vectors, is left alone (can't judge → keep). Returns
n_retracted."""
import numpy as np
settings = session.execute(
select(MLSettings).where(MLSettings.id == 1)
).scalar_one()
if not settings.ccip_auto_apply_enabled:
return 0
thr = float(settings.ccip_auto_apply_threshold)
pairs = session.execute(
select(image_tag.c.image_record_id, image_tag.c.tag_id)
.where(image_tag.c.source == "ccip_auto")
).all()
if not pairs:
return 0
confirmed = {
(iid, tid) for iid, tid in session.execute(
select(
TagPositiveConfirmation.image_record_id,
TagPositiveConfirmation.tag_id,
)
).all()
}
# Each involved character's normalized prototype matrix, loaded once.
proto: dict[int, object] = {}
for tid in {tid for _iid, tid in pairs}:
vecs = [
v for (v,) in session.execute(
select(CharacterPrototype.ccip_embedding)
.where(CharacterPrototype.tag_id == tid)
)
]
if vecs:
proto[tid] = _l2norm(
np.vstack([np.asarray(v, dtype=np.float32) for v in vecs]), np
)
retracted = 0
for iid, tid in pairs:
if (iid, tid) in confirmed or tid not in proto:
continue # confirmed / no prototypes
qvecs = [
v for (v,) in session.execute(
select(ImageRegion.ccip_embedding)
.where(ImageRegion.image_record_id == iid)
.where(ImageRegion.kind.in_(_FIGURE_KINDS))
.where(ImageRegion.ccip_embedding.is_not(None))
)
]
if not qvecs:
continue # no figure vectors → keep
Q = _l2norm(
np.vstack([np.asarray(v, dtype=np.float32) for v in qvecs]), np
)
if float((Q @ proto[tid].T).max()) < thr:
session.execute(
image_tag.delete()
.where(image_tag.c.image_record_id == iid)
.where(image_tag.c.tag_id == tid)
.where(image_tag.c.source == "ccip_auto")
)
retracted += 1
session.commit()
return retracted
+274 -24
View File
@@ -22,7 +22,7 @@ import logging
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import Any from typing import Any
from sqlalchemy import delete, func, select from sqlalchemy import delete, exists, func, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -32,13 +32,16 @@ from ...models import (
ImageRecord, ImageRecord,
ImageRegion, ImageRegion,
MLSettings, MLSettings,
PresentationReview,
Tag, Tag,
TagHead, TagHead,
TagKind, TagKind,
TagPositiveConfirmation,
TagSuggestionRejection, TagSuggestionRejection,
) )
from ...models.tag import image_tag from ...models.tag import PRESENTATION_SYSTEM_TAGS, image_tag
from .training_data import ( from .training_data import (
_AUTO_SOURCES,
_auto_apply_point, _auto_apply_point,
_hygiene_excluded_ids, _hygiene_excluded_ids,
_ids_with_tag, _ids_with_tag,
@@ -137,13 +140,20 @@ def _embedder_version(session: Session) -> str:
def _eligible_tag_ids(session: Session, min_pos: int) -> list[int]: def _eligible_tag_ids(session: Session, min_pos: int) -> list[int]:
"""Concept tags (general/character) with >= min_pos labelled images — the """Concept tags (general/character) with >= min_pos POSITIVE images — the set
set that gets a head. Counts all sources; source-aware filtering (#1133) is that gets a head. Counts human-applied + operator-confirmed tags only;
a separate, optional refinement.""" unconfirmed auto-applied predictions do NOT count toward eligibility (they
don't train the head — milestone 139), so a concept can't graduate on its own
guesses."""
confirmed = exists().where(
TagPositiveConfirmation.image_record_id == image_tag.c.image_record_id,
TagPositiveConfirmation.tag_id == image_tag.c.tag_id,
)
rows = session.execute( rows = session.execute(
select(Tag.id) select(Tag.id)
.join(image_tag, image_tag.c.tag_id == Tag.id) .join(image_tag, image_tag.c.tag_id == Tag.id)
.where(Tag.kind.in_(_HEAD_KINDS)) .where(Tag.kind.in_(_HEAD_KINDS))
.where(image_tag.c.source.not_in(_AUTO_SOURCES) | confirmed)
.group_by(Tag.id) .group_by(Tag.id)
.having(func.count(image_tag.c.image_record_id) >= min_pos) .having(func.count(image_tag.c.image_record_id) >= min_pos)
).all() ).all()
@@ -180,11 +190,20 @@ def _head_fingerprints(session: Session, tag_ids: list[int]) -> dict[int, str]:
.group_by(TagSuggestionRejection.tag_id) .group_by(TagSuggestionRejection.tag_id)
).all() ).all()
rej_map = {t: (c, m) for t, c, m in rej} rej_map = {t: (c, m) for t, c, m in rej}
# Confirmations promote an auto-applied tag to a positive (milestone 139), so
# a confirm must move the fingerprint too — else a manual Retrain right after
# confirming wouldn't fold the tag in (the nightly full run would).
conf = session.execute(
select(TagPositiveConfirmation.tag_id, func.count())
.where(TagPositiveConfirmation.tag_id.in_(tag_ids))
.group_by(TagPositiveConfirmation.tag_id)
).all()
conf_map = dict(conf)
out = {} out = {}
for t in tag_ids: for t in tag_ids:
pc, pm = pos_map.get(t, (0, None)) pc, pm = pos_map.get(t, (0, None))
rc, rm = rej_map.get(t, (0, None)) rc, rm = rej_map.get(t, (0, None))
out[t] = f"{pc}:{pm}:{rc}:{rm}" out[t] = f"{pc}:{pm}:{rc}:{rm}:{conf_map.get(t, 0)}"
return out return out
@@ -481,13 +500,19 @@ async def score_image(
session: AsyncSession, image_id: int, threshold_override: float | None = None, session: AsyncSession, image_id: int, threshold_override: float | None = None,
) -> list[dict]: ) -> list[dict]:
"""Suggestions for one image from the trained heads: [{tag_id, name, """Suggestions for one image from the trained heads: [{tag_id, name,
category, score}], ranked. A concept surfaces when its score clears the category, score, above_threshold, grounding}], ranked. A concept is INCLUDED
head's own suggest_threshold — or, when threshold_override is given (the when its score clears the head's own suggest_threshold — or, when
typed-dropdown "show everything" mode), that flat floor instead (0 → every threshold_override is given (the typed-dropdown "show everything" mode), that
head). System-tag heads (wip/banner/editor) instead use a flat flat floor instead (0 → every head). System-tag heads (wip/banner/editor)
_SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface for rejection instead use a flat _SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface
(still overridden by threshold_override). Empty if the image has no for rejection (still overridden by threshold_override). Empty if the image has
embedding or no heads exist yet. no embedding or no heads exist yet.
``above_threshold`` is reported SEPARATELY from inclusion: it's always whether
the score cleared the head's NATURAL cut (suggest_threshold, or the system
floor), regardless of any override. So the single min=0 fetch returns every
head, and the caller can split panel (above_threshold) from dropdown (all)
without a second request.
MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image
vector PLUS every concept-region crop the agent embedded (same model vector PLUS every concept-region crop the agent embedded (same model
@@ -518,21 +543,25 @@ async def score_image(
winners = probs_bag.argmax(axis=0) # (H,) winners = probs_bag.argmax(axis=0) # (H,)
out = [] out = []
for i, p in enumerate(probs): for i, p in enumerate(probs):
if threshold_override is not None: m = heads["meta"][i]
cut = threshold_override # The head's NATURAL suggest cut — system tags use the flat floor (see
elif heads["meta"][i]["is_system"]: # _SYSTEM_TAG_SUGGEST_FLOOR) so their false positives show up for the
# System tags surface at the flat floor (see _SYSTEM_TAG_SUGGEST_FLOOR) # operator to reject; content heads use their own precision-tuned
# so their false positives show up for the operator to reject. # threshold. This is what "above threshold" means (drives the panel).
cut = _SYSTEM_TAG_SUGGEST_FLOOR natural = (
else: _SYSTEM_TAG_SUGGEST_FLOOR if m["is_system"] else float(heads["thr"][i])
cut = heads["thr"][i] )
# INCLUSION is looser under threshold_override (dropdown show-all,
# override=0): every head comes back so a low-confidence concept can still
# be typed + picked, each carrying its own above_threshold flag.
cut = threshold_override if threshold_override is not None else natural
if p >= cut: if p >= cut:
m = heads["meta"][i]
out.append({ out.append({
"tag_id": m["tag_id"], "tag_id": m["tag_id"],
"name": m["name"], "name": m["name"],
"category": m["category"], "category": m["category"],
"score": float(p), "score": float(p),
"above_threshold": bool(p >= natural),
"grounding": bag_meta[int(winners[i])], "grounding": bag_meta[int(winners[i])],
}) })
out.sort(key=lambda d: d["score"], reverse=True) out.sort(key=lambda d: d["score"], reverse=True)
@@ -631,8 +660,11 @@ def start_head_auto_apply_run(session: Session, params: dict[str, Any]) -> int:
def _auto_apply_heads(session: Session, embedding_version: str, min_pos: int): def _auto_apply_heads(session: Session, embedding_version: str, min_pos: int):
"""Eligible heads to fire: graduated (auto_apply_threshold set), enough """Eligible CONTENT heads to fire: graduated (auto_apply_threshold set),
support, current embedding. Returns the row list (tag_id/name/weights/...).""" enough support, current embedding, NON-system. System tags never auto-apply
via this path — `wip` never auto-applies at all, and banner/editor screenshot
go through the presentation path at their own flat threshold (#141). Returns
the row list (tag_id/name/weights/...)."""
return session.execute( return session.execute(
select( select(
TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias, TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias,
@@ -642,6 +674,7 @@ def _auto_apply_heads(session: Session, embedding_version: str, min_pos: int):
.where(TagHead.embedding_version == embedding_version) .where(TagHead.embedding_version == embedding_version)
.where(TagHead.auto_apply_threshold.is_not(None)) .where(TagHead.auto_apply_threshold.is_not(None))
.where(TagHead.n_pos >= min_pos) .where(TagHead.n_pos >= min_pos)
.where(~Tag.is_system)
).all() ).all()
@@ -723,3 +756,220 @@ def auto_apply_sweep(
for h in range(len(rows)) for h in range(len(rows))
] ]
return {"n_applied": sum(applied), "concepts": concepts} return {"n_applied": sum(applied), "concepts": concepts}
_PRESENTATION_SOURCE = "presentation_auto"
def _presentation_heads(session: Session, embedding_version: str):
"""Trained heads for the presentation chrome tags (banner / editor screenshot).
They fire at the FLAT presentation threshold regardless of graduation — a head
exists once the operator has labelled enough chrome (head_min_positives)."""
return session.execute(
select(TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias)
.join(Tag, Tag.id == TagHead.tag_id)
.where(TagHead.embedding_version == embedding_version)
.where(Tag.is_system.is_(True))
.where(Tag.name.in_(PRESENTATION_SYSTEM_TAGS))
).all()
def _conflict_heads(session: Session, embedding_version: str):
"""ALL content (non-system) heads — the "does this ALSO look like real
content" signal for the presentation conflict guard (#141)."""
return session.execute(
select(TagHead.tag_id, TagHead.weights, TagHead.bias)
.join(Tag, Tag.id == TagHead.tag_id)
.where(TagHead.embedding_version == embedding_version)
.where(~Tag.is_system)
).all()
def _valued_image_ids(session: Session) -> set[int]:
"""Images the operator has shown they value: carrying a HUMAN or CONFIRMED
content (non-system) tag. The presentation sweep never auto-hides these
(guard 1) — you tagged it, so the model doesn't get to bury it (#141)."""
confirmed = exists().where(
TagPositiveConfirmation.image_record_id == image_tag.c.image_record_id,
TagPositiveConfirmation.tag_id == image_tag.c.tag_id,
)
rows = session.execute(
select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id)
.where(~Tag.is_system)
.where(image_tag.c.source.not_in(_AUTO_SOURCES) | confirmed)
).all()
return {r[0] for r in rows}
def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> dict:
"""Auto-hide presentation chrome (banner / editor screenshot) at the FLAT
presentation threshold (#141) — NOT the per-head graduated threshold. Two
guards keep it safe: (1) never hide an image carrying a human/confirmed content
tag; (2) if an image about to be hidden ALSO scores >= the conflict threshold
on a content head, still hide it but flag it (PresentationReview) so the Hidden
view surfaces "also looks like <X>" for review. No-op unless
presentation_auto_apply_enabled. numpy-only (no sklearn). Returns
{n_applied, n_flagged, concepts}."""
import numpy as np
from sqlalchemy.dialects.postgresql import insert as pg_insert
settings = _settings(session)
if not dry_run and not settings.presentation_auto_apply_enabled:
return {"n_applied": 0, "n_flagged": 0, "concepts": []}
ver = settings.embedder_model_version
pres = _presentation_heads(session, ver)
if not pres:
return {"n_applied": 0, "n_flagged": 0, "concepts": []}
thr = float(settings.presentation_auto_apply_threshold)
conflict_thr = float(settings.presentation_conflict_threshold)
Wp = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in pres])
bp = np.asarray([r.bias for r in pres], dtype=np.float32)
pres_tag_ids = [r.tag_id for r in pres]
pres_names = [r.name for r in pres]
conf = _conflict_heads(session, ver)
Wc = bc = conf_tag_ids = None
if conf:
Wc = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in conf])
bc = np.asarray([r.bias for r in conf], dtype=np.float32)
conf_tag_ids = [r.tag_id for r in conf]
valued = _valued_image_ids(session)
# Skip images that already carry, or have rejected, each presentation tag.
skip = {tid: set() for tid in pres_tag_ids}
for tid in pres_tag_ids:
for (iid,) in session.execute(
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tid)
):
skip[tid].add(iid)
for (iid,) in session.execute(
select(TagSuggestionRejection.image_record_id).where(
TagSuggestionRejection.tag_id == tid
)
):
skip[tid].add(iid)
applied = [0] * len(pres)
n_flagged = 0
scanned = 0
all_ids = list(session.execute(
select(ImageRecord.id).where(ImageRecord.siglip_embedding.is_not(None))
).scalars())
for start in range(0, len(all_ids), _AUTO_APPLY_CHUNK):
chunk = all_ids[start:start + _AUTO_APPLY_CHUNK]
emb = _load_embeddings(session, chunk)
cids = [i for i in chunk if i in emb]
if not cids:
continue
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
probs = 1.0 / (1.0 + np.exp(-(Xn @ Wp.T + bp))) # (N, P)
if Wc is not None:
cprobs = 1.0 / (1.0 + np.exp(-(Xn @ Wc.T + bc))) # (N, C)
max_c = cprobs.max(axis=1)
arg_c = cprobs.argmax(axis=1)
scanned += len(cids)
for p in range(len(pres)):
tid = pres_tag_ids[p]
for idx in np.where(probs[:, p] >= thr)[0]:
iid = cids[int(idx)]
if iid in skip[tid] or iid in valued:
continue
skip[tid].add(iid)
applied[p] += 1
if not dry_run:
session.execute(
pg_insert(image_tag)
.values(
image_record_id=iid, tag_id=tid,
source=_PRESENTATION_SOURCE,
)
.on_conflict_do_nothing()
)
# Guard 2: also looks like content → hide but flag for review.
if Wc is not None and float(max_c[idx]) >= conflict_thr:
n_flagged += 1
if not dry_run:
session.execute(
pg_insert(PresentationReview)
.values(
image_record_id=iid, tag_id=tid,
conflict_tag_id=conf_tag_ids[int(arg_c[idx])],
conflict_score=float(max_c[idx]),
)
.on_conflict_do_nothing()
)
if not dry_run:
session.commit()
concepts = [
{"tag_id": pres_tag_ids[p], "name": pres_names[p],
"applied": applied[p], "scanned": scanned, "threshold": thr}
for p in range(len(pres))
]
return {
"n_applied": sum(applied), "n_flagged": n_flagged, "concepts": concepts,
}
def retract_auto_applied_heads(session: Session) -> int:
"""Soft auto-apply (milestone 139): re-score every standing source='head_auto'
tag against its CURRENT head and REMOVE the ones now BELOW the head's
auto_apply_threshold — i.e. the head sharpened (or the operator raised the bar)
and no longer supports them. Skips operator-confirmed tags
(TagPositiveConfirmation). SILENT: a low score isn't proof the tag was wrong,
so no hard negative is recorded — that's reserved for an operator removal.
No-op unless head_auto_apply_enabled. Only re-scores the images that ALREADY
carry the auto-tag (bounded), never the whole library. Returns n_retracted."""
import numpy as np
settings = _settings(session)
if not settings.head_auto_apply_enabled:
return 0
heads = session.execute(
select(
TagHead.tag_id, TagHead.weights, TagHead.bias,
TagHead.auto_apply_threshold,
)
.where(TagHead.embedding_version == settings.embedder_model_version)
.where(TagHead.auto_apply_threshold.is_not(None))
).all()
retracted = 0
for tag_id, weights, bias, thr in heads:
auto_ids = [
iid for (iid,) in session.execute(
select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == tag_id)
.where(image_tag.c.source == "head_auto")
)
]
if not auto_ids:
continue
confirmed = {
iid for (iid,) in session.execute(
select(TagPositiveConfirmation.image_record_id)
.where(TagPositiveConfirmation.tag_id == tag_id)
.where(TagPositiveConfirmation.image_record_id.in_(auto_ids))
)
}
candidates = [i for i in auto_ids if i not in confirmed]
emb = _load_embeddings(session, candidates)
cids = [i for i in candidates if i in emb]
if not cids:
continue
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
w = np.asarray(weights, dtype=np.float32)
probs = 1.0 / (1.0 + np.exp(-(Xn @ w + float(bias))))
below = [cids[k] for k in np.where(probs < float(thr))[0]]
for iid in below:
session.execute(
image_tag.delete()
.where(image_tag.c.image_record_id == iid)
.where(image_tag.c.tag_id == tag_id)
.where(image_tag.c.source == "head_auto")
)
retracted += 1
session.commit()
return retracted
+17 -17
View File
@@ -22,22 +22,20 @@ from .heads import score_image
@dataclass(frozen=True) @dataclass(frozen=True)
class Suggestion: class Suggestion:
# canonical_tag_id is None when this is a raw Camie tag with no alias and # Every suggestion is a canonical Tag: heads/CCIP only score EXISTING concept
# no existing Tag row — accepting it will create the tag. # tags (tagging-v2, #114). The old raw-model-key / creates-new / alias-remap
canonical_tag_id: int | None # cases are gone — a suggestion always maps to a real tag id.
canonical_tag_id: int
display_name: str display_name: str
category: str category: str
score: float score: float
source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2) source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2)
creates_new_tag: bool # above_threshold = the score cleared the head's own suggest cut (or the
# raw_name = the booru model vocab key behind this suggestion. It's the key # system floor). The Suggestions PANEL shows only these; the typed-tag
# an alias MUST be stored under (resolution looks up the raw key), so the # dropdown fetches ALL suggestions (every head, min=0) and just annotates each
# modal needs it to author an alias correctly. None for centroid-only hits # matching row with its score, so a low-confidence concept can still be typed
# (no underlying prediction → nothing to alias). # and picked. CCIP character matches are always above their match threshold.
raw_name: str | None = None above_threshold: bool
# via_alias = this suggestion was surfaced because an operator alias remapped
# the raw prediction to this canonical tag. Lets the UI mark it + offer undo.
via_alias: bool = False
# rejected = the operator dismissed this tag for this image (a stored # rejected = the operator dismissed this tag for this image (a stored
# TagSuggestionRejection). It stays in the list — flagged, not dropped — so # TagSuggestionRejection). It stays in the list — flagged, not dropped — so
# the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery, # the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery,
@@ -108,6 +106,7 @@ class SuggestionService:
for h in hits: for h in hits:
merged[(h["category"], h["tag_id"])] = { merged[(h["category"], h["tag_id"])] = {
"name": h["name"], "score": h["score"], "source": "head", "name": h["name"], "score": h["score"], "source": "head",
"above_threshold": h["above_threshold"],
"grounding": h.get("grounding"), "grounding": h.get("grounding"),
} }
for c in ccip_hits: for c in ccip_hits:
@@ -116,12 +115,16 @@ class SuggestionService:
if ex is not None: if ex is not None:
ex["source"] = "both" ex["source"] = "both"
ex["score"] = max(ex["score"], c["score"]) ex["score"] = max(ex["score"], c["score"])
# CCIP only returns matches above its own threshold, so a CCIP
# corroboration always makes the merged suggestion above-threshold.
ex["above_threshold"] = True
# Keep the head's localized crop if it had one; else fall back to # Keep the head's localized crop if it had one; else fall back to
# the CCIP figure so a corroborated character still grounds (#1206). # the CCIP figure so a corroborated character still grounds (#1206).
ex["grounding"] = ex.get("grounding") or c.get("grounding") ex["grounding"] = ex.get("grounding") or c.get("grounding")
else: else:
merged[key] = { merged[key] = {
"name": c["name"], "score": c["score"], "source": "ccip", "name": c["name"], "score": c["score"], "source": "ccip",
"above_threshold": True,
"grounding": c.get("grounding"), "grounding": c.get("grounding"),
} }
@@ -136,7 +139,7 @@ class SuggestionService:
category=cat, category=cat,
score=m["score"], score=m["score"],
source=m["source"], source=m["source"],
creates_new_tag=False, above_threshold=m["above_threshold"],
rejected=tag_id in rejected, rejected=tag_id in rejected,
grounding=m.get("grounding"), grounding=m.get("grounding"),
) )
@@ -157,8 +160,7 @@ class SuggestionService:
was suggested for (or already applied to) >= threshold fraction of was suggested for (or already applied to) >= threshold fraction of
the selection AND was acceptable on >= 1 image. Confidence is the the selection AND was acceptable on >= 1 image. Confidence is the
mean over images where it was suggested. Aggregated by mean over images where it was suggested. Aggregated by
canonical_tag_id; creates-new (no canonical id) suggestions are canonical_tag_id (every suggestion is a canonical tag now)."""
skipped (bulk Accept applies by tag id)."""
if not image_ids: if not image_ids:
return {} return {}
threshold = min(1.0, max(0.0, threshold)) threshold = min(1.0, max(0.0, threshold))
@@ -169,8 +171,6 @@ class SuggestionService:
sl = await self.for_image(image_id) sl = await self.for_image(image_id)
for category, items in sl.by_category.items(): for category, items in sl.by_category.items():
for s in items: for s in items:
if s.canonical_tag_id is None or s.creates_new_tag:
continue
# for_image keeps rejected tags (flagged) for the rail; # for_image keeps rejected tags (flagged) for the rail;
# bulk consensus must still ignore them — a tag dismissed on # bulk consensus must still ignore them — a tag dismissed on
# an image isn't a suggestion for that image. # an image isn't a suggestion for that image.
+27 -2
View File
@@ -17,9 +17,20 @@ from typing import Any
from sqlalchemy import func, select from sqlalchemy import func, select
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from ...models import ImageRecord, Tag, TagSuggestionRejection from ...models import (
ImageRecord,
Tag,
TagPositiveConfirmation,
TagSuggestionRejection,
)
from ...models.tag import image_tag from ...models.tag import image_tag
# Auto-apply sources whose tags are PROVISIONAL: they never train a head (or seed
# a CCIP reference) unless the operator confirms them (milestone 139). Keeping
# auto-applied predictions out of training is what makes them "soft" — a misfire
# can't reinforce itself, so the retraction sweep can actually drop it.
_AUTO_SOURCES = ("head_auto", "ccip_auto", "ml_auto", "presentation_auto")
def _hygiene_excluded_ids(session: Session) -> set[int]: def _hygiene_excluded_ids(session: Session) -> set[int]:
"""Ids of images carrying ANY system tag (wip / banner / editor """Ids of images carrying ANY system tag (wip / banner / editor
@@ -45,9 +56,23 @@ def _hygiene_excluded_ids(session: Session) -> set[int]:
def _ids_with_tag(session: Session, tag_id: int) -> list[int]: def _ids_with_tag(session: Session, tag_id: int) -> list[int]:
"""Image ids that count as POSITIVES for this tag's head: human-applied
(manual / accepted) tags PLUS any auto-applied tag the operator explicitly
confirmed (TagPositiveConfirmation). Unconfirmed auto-applied tags are
EXCLUDED — they are provisional and must not train the head that judges
them (milestone 139)."""
confirmed = (
select(TagPositiveConfirmation.image_record_id)
.where(TagPositiveConfirmation.tag_id == tag_id)
)
return [ return [
r[0] for r in session.execute( r[0] for r in session.execute(
select(image_tag.c.image_record_id).where(image_tag.c.tag_id == tag_id) select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == tag_id)
.where(
image_tag.c.source.not_in(_AUTO_SOURCES)
| image_tag.c.image_record_id.in_(confirmed)
)
).all() ).all()
] ]
+15
View File
@@ -200,6 +200,8 @@ class PostFeedService:
atts_map = await self._attachments_for([post.id]) atts_map = await self._attachments_for([post.id])
item = self._to_dict(post, artist, source, thumbs_map, atts_map) item = self._to_dict(post, artist, source, thumbs_map, atts_map)
item["description_full"] = html_to_plain(post.description) item["description_full"] = html_to_plain(post.description)
# Full (uncapped) translated description for the detail view (#143).
item["description_translated_full"] = post.description_translated
# Sanitized HTML body for faithful (semantic) rendering in the post view; # Sanitized HTML body for faithful (semantic) rendering in the post view;
# detail-only (the feed list stays lightweight plain text). None when the # detail-only (the feed list stays lightweight plain text). None when the
# post has no body. Inline `<img>` sources are remapped to locally-served # post has no body. Inline `<img>` sources are remapped to locally-served
@@ -371,6 +373,12 @@ class PostFeedService:
description_plain, truncated = None, False description_plain, truncated = None, False
else: else:
description_plain, truncated = truncate_at_word(plain_full, DESCRIPTION_LIMIT) description_plain, truncated = truncate_at_word(plain_full, DESCRIPTION_LIMIT)
# Translation (#143): the stored translated description is already plain
# text; truncate it the same way for the card.
desc_trans = post.description_translated
desc_trans_short = (
truncate_at_word(desc_trans, DESCRIPTION_LIMIT)[0] if desc_trans else None
)
thumbs_entry = thumbs_map.get(post.id, {"thumbs": [], "more": 0}) thumbs_entry = thumbs_map.get(post.id, {"thumbs": [], "more": 0})
# `source` is null for filesystem-imported posts with no live # `source` is null for filesystem-imported posts with no live
# subscription (alembic 0030). Frontend renders that as a # subscription (alembic 0030). Frontend renders that as a
@@ -384,6 +392,13 @@ class PostFeedService:
"downloaded_at": post.downloaded_at.isoformat(), "downloaded_at": post.downloaded_at.isoformat(),
"description_plain": description_plain, "description_plain": description_plain,
"description_truncated": truncated, "description_truncated": truncated,
# Translation-forward fields (#143): shown by default when present;
# UI toggles back to the originals above. Source lang labels the toggle.
"post_title_translated": post.post_title_translated,
"description_translated": desc_trans_short,
"translated_source_lang": post.translated_source_lang,
# Sticky per-post translation choice (auto/force/original, #155).
"translation_override": post.translation_override,
"artist": {"id": artist.id, "name": artist.name, "slug": artist.slug}, "artist": {"id": artist.id, "name": artist.name, "slug": artist.slug},
"source": ( "source": (
{"id": source.id, "platform": source.platform} {"id": source.id, "platform": source.platform}
@@ -29,6 +29,13 @@ def _post_dict(p: Post) -> dict:
"date": p.post_date.isoformat() if p.post_date else None, "date": p.post_date.isoformat() if p.post_date else None,
"description_html": sanitize_post_html(p.description), "description_html": sanitize_post_html(p.description),
"attachment_count": p.attachment_count, "attachment_count": p.attachment_count,
# Translation (#143): the English title/description shown by default when
# a translation exists; the UI toggles to the original. Source lang labels
# the original.
"title_translated": p.post_title_translated,
"description_translated": p.description_translated,
"translated_source_lang": p.translated_source_lang,
"translation_override": p.translation_override,
} }
+7
View File
@@ -91,4 +91,11 @@ def serialize_tag(row) -> dict:
"fandom_id": row.fandom_id, "fandom_id": row.fandom_id,
"fandom_name": row.fandom_name, "fandom_name": row.fandom_name,
"is_system": bool(getattr(row, "is_system", False)), "is_system": bool(getattr(row, "is_system", False)),
# Applied-tag context: only the image-scoped selects (list_for_image /
# get_image_with_tags) provide these; autocomplete / directory don't →
# default. `source` drives the auto-applied badge; `confirmed` = the
# operator affirmed the tag (a training positive, shielded from the
# retraction sweep — milestone 139).
"source": getattr(row, "source", None),
"confirmed": bool(getattr(row, "confirmed", False)),
} }
+15 -2
View File
@@ -9,7 +9,14 @@ from sqlalchemy import and_, case, exists, func, select, text, update
from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from ..models import HeadMetric, Tag, TagHead, TagKind, image_tag from ..models import (
HeadMetric,
Tag,
TagHead,
TagKind,
TagPositiveConfirmation,
image_tag,
)
from .db_helpers import get_or_create from .db_helpers import get_or_create
from .tag_query import fandom_join_alias, tag_columns from .tag_query import fandom_join_alias, tag_columns
@@ -288,8 +295,14 @@ class TagService:
character chip with its fandom without an N+1 (mirrors the character chip with its fandom without an N+1 (mirrors the
autocomplete/directory resolution).""" autocomplete/directory resolution)."""
fandom_alias = fandom_join_alias() fandom_alias = fandom_join_alias()
# source drives the auto-applied badge; confirmed = operator affirmed the
# tag (positive + retraction-shielded, milestone 139).
confirmed = exists().where(
TagPositiveConfirmation.image_record_id == image_id,
TagPositiveConfirmation.tag_id == Tag.id,
).label("confirmed")
stmt = ( stmt = (
select(*tag_columns(fandom_alias)) select(*tag_columns(fandom_alias), image_tag.c.source, confirmed)
.select_from( .select_from(
Tag.__table__ Tag.__table__
.join(image_tag, image_tag.c.tag_id == Tag.id) .join(image_tag, image_tag.c.tag_id == Tag.id)
+36
View File
@@ -79,6 +79,14 @@ VERIFY_PAGE = 200
FFPROBE_TIMEOUT_SECONDS = 10 FFPROBE_TIMEOUT_SECONDS = 10
TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h
TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days
# Orphaned staging files: downloads/imports stage into <name>.part / <name>.partial
# then os.replace() into place (importer / external_fetch / native_ingest_common /
# attachment_store), so a kill mid-write leaves a discardable temp, never a corrupt
# final. cleanup_orphaned_temp_files sweeps ones left behind; the min-age guard
# keeps it from deleting an in-flight download's staging file mid-write.
IMAGES_ROOT = Path("/images")
TEMP_STAGING_SUFFIXES = (".part", ".partial")
ORPHAN_TEMP_MIN_AGE_HOURS = 6
# Audit 2026-06-02: per-entity recovery sweep thresholds. Each must be # Audit 2026-06-02: per-entity recovery sweep thresholds. Each must be
# > the entity's longest legitimate runtime (its task's time_limit + a # > the entity's longest legitimate runtime (its task's time_limit + a
@@ -339,6 +347,34 @@ def cleanup_old_tasks() -> int:
return result.rowcount or 0 return result.rowcount or 0
@celery.task(name="backend.app.tasks.maintenance.cleanup_orphaned_temp_files")
def cleanup_orphaned_temp_files() -> int:
"""Delete orphaned .part/.partial staging files under the images root, left by
a download/import killed mid-write. Only removes files older than
ORPHAN_TEMP_MIN_AGE_HOURS so an in-flight download's staging file is never
pulled out from under it. Returns the count removed."""
if not IMAGES_ROOT.is_dir():
return 0
cutoff = datetime.now(UTC).timestamp() - ORPHAN_TEMP_MIN_AGE_HOURS * 3600
removed = 0
for path in IMAGES_ROOT.rglob("*"):
if path.suffix not in TEMP_STAGING_SUFFIXES or not path.is_file():
continue
try:
if path.stat().st_mtime >= cutoff:
continue # too fresh — may be an active download
path.unlink()
removed += 1
except OSError as exc:
log.warning("cleanup_orphaned_temp_files: %s: %s", path, exc)
if removed:
log.info(
"cleanup_orphaned_temp_files: removed %d orphaned staging file(s)",
removed,
)
return removed
@celery.task(name="backend.app.tasks.maintenance.recover_stalled_task_runs") @celery.task(name="backend.app.tasks.maintenance.recover_stalled_task_runs")
def recover_stalled_task_runs() -> int: def recover_stalled_task_runs() -> int:
"""Flip task_run rows stuck in 'running' past their queue-specific """Flip task_run rows stuck in 'running' past their queue-specific
+60
View File
@@ -592,3 +592,63 @@ def scheduled_ccip_auto_apply() -> str:
applied += 1 applied += 1
session.commit() session.commit()
return f"applied={applied}" return f"applied={applied}"
@celery.task(
name="backend.app.tasks.ml.scheduled_presentation_auto_apply",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_presentation_auto_apply() -> str:
"""Auto-hide presentation chrome (banner / editor screenshot) on a daily
passive sweep (#141). No-op unless presentation_auto_apply_enabled. Idempotent
— already-hidden images are skipped — so an interrupted run simply re-runs next
cycle (that IS the recovery). Wall-clock bounded by the task time limits."""
from ..services.ml.heads import presentation_auto_apply_sweep
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
result = presentation_auto_apply_sweep(session)
return f"applied={result['n_applied']} flagged={result['n_flagged']}"
@celery.task(name="backend.app.tasks.ml.prune_presentation_reviews")
def prune_presentation_reviews() -> str:
"""Retention (rule 89): drop RESOLVED presentation-review flags older than 30
days — the operator has acted on them, so they're just history (#141)."""
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete as sa_delete
from ..models import PresentationReview
cutoff = datetime.now(UTC) - timedelta(days=30)
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
res = session.execute(
sa_delete(PresentationReview).where(
PresentationReview.resolved_at.is_not(None),
PresentationReview.resolved_at < cutoff,
)
)
session.commit()
return f"pruned={res.rowcount}"
@celery.task(
name="backend.app.tasks.ml.scheduled_retract_auto_tags",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_retract_auto_tags() -> str:
"""Soft auto-apply (milestone 139): retract standing head_auto/ccip_auto tags
the model no longer supports (score now below the auto-apply threshold),
skipping operator-confirmed ones. Silent (no hard negative). No-op unless the
respective auto-apply switch is on. Returns 'head=N ccip=M'."""
from ..services.ml.character_prototypes import retract_auto_applied_ccip
from ..services.ml.heads import retract_auto_applied_heads
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
n_head = retract_auto_applied_heads(session)
with SessionLocal() as session:
n_ccip = retract_auto_applied_ccip(session)
return f"head={n_head} ccip={n_ccip}"
+398
View File
@@ -0,0 +1,398 @@
"""Post-text translation Celery tasks (milestone 143 + re-translate m146).
Backfills non-English post title/description to the target language via the
self-hosted Interpreter LAN service, storing the result + engine_version so
viewing is instant and re-runs are cache-fast. Sync (Celery workers are sync),
same pattern as the other backfill sweeps. No-op unless translation is enabled,
a base URL is set, and the service is healthy.
Curator does NO language detection of its own: each field's translation is
accepted or rejected purely on Interpreter's reported engine + detected language
+ confidence (see ``_accept``) — a short English title Interpreter mis-labels as
a European language scores below the latin-script floor and is kept as-is.
Two entry points:
- ``translate_posts`` — the periodic untranslated sweep (every 8h;
translated_source_lang IS NULL only), one bounded chunk per fire. The Settings
"Translate now" button calls it with ``drain=True`` to chase the tail
run-until-done and clear the whole backlog in a single press.
- ``retranslate_posts`` — after a model change, resets the stored translation
columns for a scoped set of posts (all, or a given set of artists) so the
untranslated selection re-runs them, then chases the tail until drained
(run-until-done). The Interpreter cache keys on engine_version: a changed
model genuinely re-translates, an unchanged one is ~1ms cache-fast.
"""
import logging
from datetime import UTC, datetime
from celery.exceptions import SoftTimeLimitExceeded
from sqlalchemy import func, or_, select, update
from ..celery_app import celery
from ..models import ImportSettings, Post
from ..services import interpreter_client as ic
from ..utils.text import html_to_plain
from ._sync_engine import sync_session_factory as _sync_session_factory
log = logging.getLogger(__name__)
# Bound one run's work so it commits progress rather than dying at the time
# limit; the rest resumes next cycle (idempotent — only untranslated posts are
# picked, so an interrupted run just re-runs = rule-89 recovery).
_MAX_POSTS_PER_RUN = 300
# Short gap between run-until-done chunks so a big re-translate finishes on its
# own without monopolising the maintenance_long worker in one uninterrupted run.
_RETRANSLATE_COUNTDOWN = 5
# When Interpreter drains mid-chunk (graceful restart), resume the bulk
# re-translate after the service's Retry-After hint if it sent one, else this
# default; capped so a bogus/huge hint can't park the work longer than the daily
# beat's own safety net would.
_INTERRUPT_BACKOFF = 60
_INTERRUPT_BACKOFF_MAX = 900
# --- Translation acceptance gate (Scribe rule 133; contract note #1347) -------
# Curator does NO language detection of its own — it accepts or rejects a
# translation purely on Interpreter's reported detection. CJK is script-detected
# (kana/hangul/kanji) and reliably high-confidence, so ja/ko/zh are trusted
# outright (pure-kanji Japanese can even come back labelled zh ~0.75, still
# correctly translated). Latin-script detection is a real statistical
# probability, so a short English title Interpreter mis-labels as a European
# language scores low; require it to clear this floor, else keep the original.
# Calibrated 2026-07-09 against fresh (uncached) probes once Interpreter returned
# real langdetect confidence: genuine German detected at 1.0, while a correctly-
# detected but ambiguous latin string dipped to 0.86 — so a single constant can't win.
# The floor is now operator-tunable (ImportSettings.translation_min_confidence,
# milestone 155) with a stricter 0.90 default; per-post overrides
# (Post.translation_override) rescue the false negatives it skips. Genuine German
# and mis-flagged short English both land ~0.86, and single-word mis-flags sit at
# a confident 1.0 no floor catches. Real langdetect fixed some cases at the
# source (some short English titles now detect as English → passthrough), so this
# floor is a safety net, not the primary fix. Re-tune via the "Test translation"
# box (send fresh text — cache hits report 1.0).
_CJK_LANGS = frozenset({"ja", "ko", "zh"})
_DEFAULT_MIN_LATIN_CONFIDENCE = 0.90
def _interrupt_backoff(retry_after) -> int:
"""Seconds to wait before resuming after an Interpreter interruption: the
server's Retry-After hint (capped), else the default."""
if retry_after and retry_after > 0:
return int(min(retry_after, _INTERRUPT_BACKOFF_MAX))
return _INTERRUPT_BACKOFF
@celery.task(
name="backend.app.tasks.translation.translate_posts",
soft_time_limit=1800, time_limit=2100,
)
def translate_posts(drain: bool = False) -> str:
"""Translate untranslated non-English post title/description (default target
en) via Interpreter. Per-post [title, description] batch → per-post detected
language. Passthrough / already-target posts are marked handled (source ==
target) with no stored translation. 503 or a connection error interrupts the
run (retry next cycle); 400 stops it (fix config); posts done so far stay
committed. No-op unless configured + healthy. Returns a summary string.
``drain`` (the Settings "Translate now" button) chases the tail
run-until-done: on a clean chunk with work still remaining it re-enqueues
itself until the untranslated backlog is zero, like ``retranslate_posts``. The
periodic beat leaves it False → one bounded chunk per fire (the 8h cadence
drives the rest, enough for the trickle of newly-imported posts)."""
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
ready = _translation_config(session)
if isinstance(ready, str):
return ready
base_url, target, min_confidence = ready
posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch(
session, posts, base_url, target, min_confidence,
)
# Drain mode (manual "Translate now"): chase the tail until the backlog is
# zero, so one press clears the whole pile instead of one 300-chunk.
# Termination is guaranteed — every handled post leaves the untranslated
# set, so the remaining count strictly decreases each chunk.
if status == "ok" and drain:
remaining = _count_untranslated(session, None)
if remaining:
translate_posts.apply_async(
(), {"drain": True}, countdown=_RETRANSLATE_COUNTDOWN,
)
log.info(
"translate_posts: draining — %d remaining → re-enqueued",
remaining,
)
# Interpreter drained mid-chunk (graceful restart): don't make a manual
# "Translate now" wait a whole beat cycle — re-enqueue after its
# Retry-After hint (or the default backoff), preserving `drain` so a bulk
# drain resumes cleanly. Self-terminating: once the service is down the
# health gate returns "interpreter unavailable" early.
elif status == "interrupted" and _count_untranslated(session, None):
delay = _interrupt_backoff(retry_after)
translate_posts.apply_async((), {"drain": drain}, countdown=delay)
log.info(
"translate_posts: interrupted (service draining) → retry in %ss",
delay,
)
return _summary(status, translated, len(posts))
@celery.task(
name="backend.app.tasks.translation.retranslate_posts",
soft_time_limit=1800, time_limit=2100,
)
def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
"""Re-translate posts after a model change. On the first call resets the
stored translation columns to NULL for the scoped posts — all artists when
``artist_ids`` is falsy, else ``WHERE artist_id IN artist_ids`` — so the
normal untranslated selection re-runs them through Interpreter. Then handles
one bounded chunk and re-enqueues itself until the scoped backlog is drained
(run-until-done; ``_reset_done`` guards against wiping fresh work on the
follow-up chunks). No reset happens unless the service is configured +
healthy, so translations are never cleared when they can't be rebuilt."""
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
ready = _translation_config(session)
if isinstance(ready, str):
return ready
base_url, target, min_confidence = ready
if not _reset_done:
n = _reset_translations(session, artist_ids)
log.info(
"retranslate_posts: reset %d post(s) (artist_ids=%s)",
n, artist_ids,
)
posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch(
session, posts, base_url, target, min_confidence,
)
# run-until-done: chase the tail when the chunk finished cleanly.
# Termination is guaranteed: every handled post leaves the NULL set, so
# the remaining count strictly decreases each chunk.
if status == "ok":
remaining = _count_untranslated(session, artist_ids)
if remaining:
retranslate_posts.apply_async(
(),
{"artist_ids": artist_ids, "_reset_done": True},
countdown=_RETRANSLATE_COUNTDOWN,
)
log.info(
"retranslate_posts: %d remaining → re-enqueued", remaining,
)
# Interpreter drained mid-chunk (graceful restart / 429 / 5xx): don't
# stall the bulk re-translate until the daily beat — resume it after the
# service's Retry-After hint (or the default backoff), with
# _reset_done=True so the fresh work is never re-wiped. Self-terminating:
# if the service is still down next run, _translation_config's health gate
# returns "interpreter unavailable" early and nothing re-enqueues.
elif status == "interrupted" and _count_untranslated(session, artist_ids):
delay = _interrupt_backoff(retry_after)
retranslate_posts.apply_async(
(),
{"artist_ids": artist_ids, "_reset_done": True},
countdown=delay,
)
log.info(
"retranslate_posts: interrupted (service draining) → retry in %ss",
delay,
)
return _summary(status, translated, len(posts))
def _translation_config(session):
"""Resolve (base_url, target, min_confidence) if translation is enabled +
healthy, else a short status string ("disabled" / "interpreter unavailable")
for the caller to return. Centralises the guard so translate/retranslate agree
exactly, including the operator-tunable acceptance floor."""
cfg = ImportSettings.load_sync(session)
if not cfg.translation_enabled or not cfg.interpreter_base_url.strip():
return "disabled"
base_url = cfg.interpreter_base_url.strip()
target = (cfg.translation_target_lang or "en").strip() or "en"
min_confidence = cfg.translation_min_confidence
if not ic.health(base_url):
return "interpreter unavailable"
return base_url, target, min_confidence
def _untranslated_filter(stmt, artist_ids):
"""Add the untranslated-post predicate (+ optional artist scope) to a
select/count over Post. Untranslated = translated_source_lang IS NULL with
some text to translate."""
stmt = stmt.where(
Post.translated_source_lang.is_(None)
).where(or_(
Post.post_title.is_not(None), Post.description.is_not(None),
))
if artist_ids:
stmt = stmt.where(Post.artist_id.in_(artist_ids))
return stmt
def _select_untranslated(session, artist_ids, limit):
return session.execute(
_untranslated_filter(select(Post), artist_ids).limit(limit)
).scalars().all()
def _count_untranslated(session, artist_ids) -> int:
return int(session.execute(
_untranslated_filter(select(func.count(Post.id)), artist_ids)
).scalar_one())
def _reset_translations(session, artist_ids) -> int:
"""Clear the stored translation columns for scoped, already-handled posts so
the untranslated sweep re-runs them. Only touches rows that were translated
(translated_source_lang IS NOT NULL) — untranslated rows are already NULL.
Skips 'keep original' posts (translation_override = 'original') so that choice
survives a Re-translate-all (milestone 155). Returns the row count reset
(commits it)."""
stmt = (
update(Post)
.where(Post.translated_source_lang.is_not(None))
.where(Post.translation_override != "original")
.values(
post_title_translated=None,
description_translated=None,
translated_source_lang=None,
translation_engine_version=None,
translated_at=None,
)
)
if artist_ids:
stmt = stmt.where(Post.artist_id.in_(artist_ids))
n = session.execute(stmt).rowcount
session.commit()
return n
def _translate_batch(session, posts, base_url: str, target: str, min_confidence: float):
"""Translate each post in the chunk with a per-post commit (an interrupted
run keeps progress). Returns (status, translated, retry_after) where status is
one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout";
retry_after is the server's Retry-After hint in seconds on an interrupt, else
None."""
translated = 0
for post in posts:
try:
translated += _translate_one(session, post, base_url, target, min_confidence)
session.commit()
except ic.InterpreterUnavailable as e:
session.rollback()
return "interrupted", translated, getattr(e, "retry_after", None)
except ic.InterpreterBadRequest as e:
session.rollback()
log.warning("translate bad request: %s", e)
return "stopped", translated, None
except SoftTimeLimitExceeded:
return "timeout", translated, None
return "ok", translated, None
def _summary(status: str, translated: int, scanned: int) -> str:
if status == "interrupted":
return f"interrupted (service down) — translated={translated}"
if status == "stopped":
return f"stopped (bad request) — translated={translated}"
if status == "timeout":
return f"time limit — translated={translated}"
return f"translated={translated} scanned={scanned}"
def _accept(
language: str, confidence, min_confidence: float = _DEFAULT_MIN_LATIN_CONFIDENCE,
) -> bool:
"""Should curator store Interpreter's translation, or keep the original?
Consumes ONLY Interpreter's own detection (curator does none, Scribe rule
133): CJK languages are trusted regardless of the reported number; a
latin-script detection must clear ``min_confidence`` (the operator-tunable
ImportSettings.translation_min_confidence). A missing confidence fails OPEN
(accept), so a service that omits the field never silently drops a
translation. Per-post ``force`` overrides bypass this entirely (see
``_translate_one``)."""
if (language or "").split("-", 1)[0].lower() in _CJK_LANGS:
return True
if confidence is None:
return True
return confidence >= min_confidence
def _translate_field(
text: str, base_url: str, target: str, min_confidence: float, force: bool = False,
):
"""Translate ONE field independently. Returns (translated, source_lang,
engine_version), or (None, None, None) when there's nothing to store — empty
text, already the target language, a passthrough (engine "none"), or a
latin-script detection the acceptance gate rejects as too low-confidence (kept
as the original). ``force`` (a per-post 'force' override) bypasses the
confidence gate so a legitimately-foreign title Interpreter is only ~0.86 sure
about is still stored; a genuine passthrough stores nothing regardless (there
is no translation to force). Per-field (not aggregate first-item) detection so
a non-English description still translates when the title is already English."""
if not text:
return None, None, None
res = ic.translate([text], base_url=base_url, target=target)
detected = res["detected_lang"] or target
if detected == target or res["engine"] == "none":
return None, None, None
# Trust only Interpreter's own detection (Scribe rule 133): keep the original
# when a latin-script detection doesn't clear the confidence floor, so a short
# English title mis-labelled as e.g. German isn't rewritten into the archive.
# A 'force' override skips the gate — the operator is overriding a rejection.
if not force and not _accept(
detected, res.get("detected_confidence"), min_confidence,
):
return None, None, None
return res["translations"][0], detected, res["engine_version"]
def _store_translation(post, title_res, desc_res, target: str) -> int:
"""Apply per-field (translated, source_lang, engine_version) results to a post
in place. Returns 1 if a translation was stored, 0 when the post is all
passthrough/empty — in which case the translated columns are CLEARED and the
post is marked handled (translated_source_lang = target). Clearing (not just
leaving) matters on re-apply: a 'keep original' override, or a re-translate
that now rejects a previously-accepted mis-flag, must remove the stale
translation, not keep it."""
title_tr, title_lang, title_ev = title_res
desc_tr, desc_lang, desc_ev = desc_res
if title_tr is None and desc_tr is None:
post.post_title_translated = None
post.description_translated = None
post.translated_source_lang = target
post.translation_engine_version = None
post.translated_at = None
return 0
post.post_title_translated = title_tr
post.description_translated = desc_tr
# Source lang = whichever field was actually non-target (for a mixed post,
# the translated field's language, not the already-English one).
post.translated_source_lang = title_lang or desc_lang
post.translation_engine_version = title_ev or desc_ev
post.translated_at = datetime.now(UTC)
return 1
def _translate_one(
session, post, base_url: str, target: str, min_confidence: float,
) -> int:
"""Translate a post's title/description in place, each field independently,
honoring the per-post override (milestone 155): 'original' keeps the original
(clears any stored translation, no Interpreter call); 'force' translates even
below the confidence floor; 'auto' (default) runs the acceptance gate. Returns
1 if it stored a translation, else 0 (still marks the post handled so the sweep
won't revisit it)."""
if post.translation_override == "original":
return _store_translation(post, (None, None, None), (None, None, None), target)
force = post.translation_override == "force"
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
title_res = _translate_field(title, base_url, target, min_confidence, force=force)
desc_res = _translate_field(desc, base_url, target, min_confidence, force=force)
return _store_translation(post, title_res, desc_res, target)
+65
View File
@@ -8,6 +8,35 @@
# run `docker compose up` from this directory and switches images to # run `docker compose up` from this directory and switches images to
# local builds + DEBUG logging. # local builds + DEBUG logging.
# Rolling-deploy safety (Swarm / `docker stack deploy`): update one task at a
# time, START the new task before stopping the old (zero-downtime via the ingress
# mesh), and if the new task doesn't reach a healthy state within `monitor`, roll
# back to the previous image automatically. `monitor` is sized above web's
# healthcheck start_period so a broken image that never goes healthy is caught.
# Plain `docker compose up` ignores `deploy:` (it warns + skips), so the dev
# override is unaffected. Referenced by each long-lived service below.
x-deploy-policy: &deploy_policy
update_config:
order: start-first
failure_action: rollback
monitor: 90s
rollback_config:
order: start-first
restart_policy:
condition: any
delay: 10s
# Worker liveness: ping THIS container's celery node over the broker. Lenient
# (60s interval, 3 retries, 60s start_period) so a transient broker blip never
# false-flags a worker into a rollback. `$$HOSTNAME` → `$HOSTNAME` for the shell;
# celery's default node name is celery@<hostname> (the container id).
x-celery-healthcheck: &celery_healthcheck
test: ["CMD-SHELL", "celery -A backend.app.celery_app:celery inspect ping -d celery@$$HOSTNAME --timeout 10 >/dev/null 2>&1"]
interval: 60s
timeout: 15s
retries: 3
start_period: 60s
services: services:
redis: redis:
image: redis:7-alpine image: redis:7-alpine
@@ -47,6 +76,24 @@ services:
web: web:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["web"] command: ["web"]
# Graceful shutdown: give the container time to drain in-flight work on a
# deploy (docker SIGTERMs, then SIGKILLs after this window — default is only
# 10s, far too short for real jobs). Hypercorn/Celery both warm-shut-down on
# SIGTERM; per-lane values sized to typical task length. Anything that still
# outruns the window is re-queued (task_reject_on_worker_lost) and re-driven
# by the 5-min recovery sweeps, so a kill never corrupts. web = short HTTP
# requests + the occasional file download.
stop_grace_period: 30s
# Liveness for rolling deploys: /api/health is a no-DB 200 (just proves the
# app booted + serves HTTP after `alembic upgrade head`). start_period covers
# the migration + boot so a slow start isn't mis-flagged.
healthcheck:
test: ["CMD-SHELL", "python -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/api/health', timeout=5).status==200 else 1)\""]
interval: 15s
timeout: 6s
retries: 3
start_period: 40s
deploy: *deploy_policy
ports: ports:
- "${PORT:-8080}:8080" - "${PORT:-8080}:8080"
environment: &app_env environment: &app_env
@@ -77,6 +124,10 @@ services:
worker: worker:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["worker"] command: ["worker"]
# Drain in-flight import/thumbnail/download tasks before SIGKILL on deploy.
stop_grace_period: 90s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment: environment:
<<: *app_env <<: *app_env
CELERY_QUEUES: default,import,thumbnail,download CELERY_QUEUES: default,import,thumbnail,download
@@ -93,6 +144,10 @@ services:
scheduler: scheduler:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["scheduler"] command: ["scheduler"]
# Quick maintenance/scan lane + beat — short tasks, modest drain window.
stop_grace_period: 60s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment: environment:
<<: *app_env <<: *app_env
CELERY_QUEUES: maintenance,scan CELERY_QUEUES: maintenance,scan
@@ -110,6 +165,12 @@ services:
maintenance-long: maintenance-long:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["worker"] command: ["worker"]
# Longest lane (DB backups, library audits, translation backfill) — give it
# the most room to finish a chunk gracefully. Chunked + idempotent, so a job
# that still outruns this resumes cleanly next run rather than corrupting.
stop_grace_period: 180s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment: environment:
<<: *app_env <<: *app_env
CELERY_QUEUES: maintenance_long CELERY_QUEUES: maintenance_long
@@ -125,6 +186,10 @@ services:
ml-worker: ml-worker:
image: git.fabledsword.com/bvandeusen/fabledcurator-ml:dev image: git.fabledsword.com/bvandeusen/fabledcurator-ml:dev
command: ["ml-worker"] command: ["ml-worker"]
# A single GPU inference pass can run tens of seconds — let it finish.
stop_grace_period: 120s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment: environment:
<<: *app_env <<: *app_env
volumes: volumes:
@@ -101,6 +101,47 @@
</v-card> </v-card>
</v-dialog> </v-dialog>
<section class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Translation</h2>
<p class="fc-muted text-body-2 mb-3">
Re-translate every post by {{ overview.name }} through the current
Interpreter model — clears the stored translations and rebuilds them.
Use this after switching translation models to refresh just this artist.
</p>
<div class="d-flex align-center flex-wrap" style="gap: 10px;">
<v-btn
size="small" variant="tonal" rounded="pill"
prepend-icon="mdi-translate"
:loading="retranslating" :disabled="!translationEnabled"
@click="confirmRetranslate = true"
>Re-translate posts</v-btn>
<span v-if="!translationEnabled" class="fc-muted text-caption">
Translation is off — enable it in Settings → Maintenance.
</span>
</div>
</section>
<!-- m146: per-artist re-translate — clears + rebuilds this artist's stored
translations (used after a translation-model change). -->
<v-dialog v-model="confirmRetranslate" max-width="440">
<v-card>
<v-card-title class="fc-h2">Re-translate {{ overview.name }}?</v-card-title>
<v-card-text class="text-body-2">
Clears the stored translation for every post by
<strong>{{ overview.name }}</strong> and re-runs them through your
current Interpreter model. Unchanged text comes back from the cache
instantly. Runs in the background until done.
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="confirmRetranslate = false">Cancel</v-btn>
<v-btn color="accent" :loading="retranslating" @click="onRetranslate">
Re-translate
</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
<section class="fc-artist-mgmt__sec"> <section class="fc-artist-mgmt__sec">
<h2 class="fc-h2">Danger zone</h2> <h2 class="fc-h2">Danger zone</h2>
<ArtistDangerZone <ArtistDangerZone
@@ -113,9 +154,11 @@
</template> </template>
<script setup> <script setup>
import { computed, ref } from 'vue' import { computed, onMounted, ref } from 'vue'
import { useRouter, RouterLink } from 'vue-router' import { useRouter, RouterLink } from 'vue-router'
import { useApi } from '../../composables/useApi.js'
import { useImportStore } from '../../stores/import.js'
import { useSourcesStore } from '../../stores/sources.js' import { useSourcesStore } from '../../stores/sources.js'
import { toast } from '../../utils/toast.js' import { toast } from '../../utils/toast.js'
import ArtistDangerZone from './ArtistDangerZone.vue' import ArtistDangerZone from './ArtistDangerZone.vue'
@@ -125,8 +168,38 @@ const props = defineProps({
}) })
const router = useRouter() const router = useRouter()
const api = useApi()
const importStore = useImportStore()
const sources = useSourcesStore() const sources = useSourcesStore()
// m146: per-artist re-translate. Gated on translation being enabled globally
// (the endpoint 400s otherwise) — read the shared import settings once.
const translationEnabled = ref(false)
const retranslating = ref(false)
const confirmRetranslate = ref(false)
onMounted(async () => {
try {
await importStore.loadSettings()
translationEnabled.value = !!importStore.settings?.translation_enabled
} catch { /* non-fatal — the button just stays disabled */ }
})
async function onRetranslate () {
retranslating.value = true
try {
await api.post('/api/settings/translation/retranslate', {
body: { artist_id: props.overview.id },
})
confirmRetranslate.value = false
toast({ text: `Re-translating ${props.overview.name}`, type: 'success' })
} catch (e) {
toast({ text: `Re-translate failed: ${e.message}`, type: 'error' })
} finally {
retranslating.value = false
}
}
// #130: move a source into another artist. // #130: move a source into another artist.
const moveOpen = ref(false) const moveOpen = ref(false)
const moveSource = ref(null) const moveSource = ref(null)
@@ -34,6 +34,15 @@
:color="store.filter.no_artist ? 'accent' : undefined" :color="store.filter.no_artist ? 'accent' : undefined"
@click="toggleFlag('no_artist')" @click="toggleFlag('no_artist')"
>No artist<span class="fc-facets__count">{{ facetCount('no_artist') }}</span></v-chip> >No artist<span class="fc-facets__count">{{ facetCount('no_artist') }}</span></v-chip>
<!-- Reveal presentation chrome (banner / editor screenshot) the gallery
hides by default (milestone 141). A browse-mode toggle, not a count. -->
<v-chip
size="small" label
:variant="store.filter.include_hidden ? 'flat' : 'tonal'"
:color="store.filter.include_hidden ? 'accent' : undefined"
title="Show hidden — reveal banners / editor screenshots the gallery sets aside by default"
@click="toggleFlag('include_hidden')"
>Show hidden</v-chip>
</div> </div>
</div> </div>
@@ -160,7 +160,7 @@ let debounce = null
const refineCount = computed(() => { const refineCount = computed(() => {
const f = store.filter const f = store.filter
return (f.platform ? 1 : 0) + (f.untagged ? 1 : 0) + (f.no_artist ? 1 : 0) return (f.platform ? 1 : 0) + (f.untagged ? 1 : 0) + (f.no_artist ? 1 : 0)
+ (f.date_from ? 1 : 0) + (f.date_to ? 1 : 0) + (f.include_hidden ? 1 : 0) + (f.date_from ? 1 : 0) + (f.date_to ? 1 : 0)
}) })
const hasRefineFilters = computed(() => refineCount.value > 0) const hasRefineFilters = computed(() => refineCount.value > 0)
@@ -0,0 +1,150 @@
<template>
<!-- Auto-hidden chrome that ALSO looked like real content surfaced PROACTIVELY
atop the gallery whenever there's something to review (NOT gated on the
Show-hidden toggle, so misfires can't go unnoticed), most-concerning first,
with keep / un-hide (#141). Renders nothing when there's nothing to review. -->
<section v-if="items.length" class="fc-review" aria-label="Hidden images to review">
<div class="fc-review__head">
<v-icon size="18" color="warning">mdi-alert-outline</v-icon>
<span class="fc-review__title">
{{ items.length }} auto-hidden {{ items.length === 1 ? 'image' : 'images' }}
may be real content — review before they stay hidden
</span>
</div>
<div class="fc-review__cards">
<div
v-for="it in items" :key="keyOf(it)" class="fc-review-card"
>
<img
:src="it.thumbnail_url" :alt="it.tag_name"
class="fc-review-card__thumb" loading="lazy"
>
<div class="fc-review-card__body">
<div
class="fc-review-card__conflict"
:title="`Scored ${Math.round(it.conflict_score * 100)}% on “${it.conflict_name || 'a content tag'}”`"
>
also looks like <strong>{{ it.conflict_name || 'content' }}</strong>
</div>
<div class="fc-review-card__tag">hidden as {{ it.tag_name }}</div>
<div class="fc-review-card__acts">
<button
type="button" class="fc-review-btn fc-review-btn--keep"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')"
>Keep hidden</button>
<button
type="button" class="fc-review-btn fc-review-btn--unhide"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')"
>Un-hide</button>
</div>
</div>
</div>
</div>
</section>
</template>
<script setup>
import { onMounted, ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { toast } from '../../utils/toast.js'
const api = useApi()
const items = ref([])
const busy = ref([])
function keyOf(it) { return `${it.image_id}:${it.tag_id}` }
async function load() {
// Fetched unconditionally on mount — the strip prompts for pending misfires
// even when the operator is browsing normally (Show-hidden off).
try {
const body = await api.get('/api/gallery/hidden-review')
items.value = body.items || []
} catch { items.value = [] }
}
async function resolve(it, action) {
const k = keyOf(it)
busy.value = [...busy.value, k]
try {
await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`)
items.value = items.value.filter((x) => keyOf(x) !== k)
if (action === 'unhide') {
toast({ text: `Un-hidden — “${it.tag_name}” removed; it'll train the head`, type: 'success' })
}
} catch (e) {
toast({
text: `Could not ${action === 'keep' ? 'keep hidden' : 'un-hide'}: ${e.message}`,
type: 'error',
})
} finally {
busy.value = busy.value.filter((x) => x !== k)
}
}
onMounted(load)
</script>
<style scoped>
.fc-review {
margin-bottom: 14px;
padding: 12px;
border: 1px solid rgb(var(--v-theme-warning), 0.4);
background: rgb(var(--v-theme-warning), 0.06);
border-radius: 8px;
}
.fc-review__head {
display: flex; align-items: center; gap: 8px; margin-bottom: 10px;
}
.fc-review__title {
font-size: 14px; font-weight: 600;
color: rgb(var(--v-theme-on-surface));
}
.fc-review__cards {
display: flex; gap: 10px; overflow-x: auto; padding-bottom: 4px;
}
.fc-review-card {
flex: 0 0 auto; width: 150px;
display: flex; flex-direction: column;
border: 1px solid rgb(var(--v-theme-surface-light));
border-radius: 6px; overflow: hidden;
background: rgb(var(--v-theme-surface));
}
.fc-review-card__thumb {
width: 100%; height: 96px; object-fit: cover; display: block;
background: rgb(var(--v-theme-surface-light));
}
.fc-review-card__body { padding: 6px 8px; }
.fc-review-card__conflict {
font-size: 11px; color: rgb(var(--v-theme-on-surface));
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
}
.fc-review-card__conflict strong { color: rgb(var(--v-theme-warning)); }
.fc-review-card__tag {
font-size: 10px; color: rgb(var(--v-theme-on-surface-variant));
margin: 1px 0 6px;
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
}
.fc-review-card__acts { display: flex; gap: 4px; }
.fc-review-btn {
flex: 1; font-size: 11px; padding: 3px 4px; border-radius: 4px;
cursor: pointer; border: 1px solid transparent;
}
.fc-review-btn:disabled { opacity: 0.5; cursor: default; }
.fc-review-btn--keep {
background: transparent;
color: rgb(var(--v-theme-on-surface-variant));
border-color: rgb(var(--v-theme-on-surface-variant), 0.4);
}
.fc-review-btn--keep:hover:not(:disabled) {
background: rgb(var(--v-theme-on-surface-variant), 0.1);
}
.fc-review-btn--unhide {
background: rgb(var(--v-theme-accent)); color: #fff;
}
.fc-review-btn--unhide:hover:not(:disabled) { opacity: 0.9; }
.fc-review-btn:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
}
</style>
@@ -1,72 +0,0 @@
<template>
<v-card>
<v-card-title>Treat as alias for</v-card-title>
<v-card-text>
<p class="text-caption mb-2">
Pick the existing tag the model's prediction should map to. All
future predictions of this name will resolve to your tag.
</p>
<v-autocomplete
v-model="selectedId"
v-model:menu="menuOpen"
:items="results"
:item-title="(t) => t.fandom_name ? `${t.name} — ${t.fandom_name}` : t.name"
:item-value="(t) => t.id"
:loading="loading"
label="Canonical tag"
no-filter clearable density="compact"
@update:search="onSearch"
@keydown.enter.capture="onEnter"
/>
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn @click="$emit('cancel')">Cancel</v-btn>
<v-btn
color="primary" rounded="pill" :disabled="!selectedId"
@click="$emit('confirm', selectedId)"
>Use this tag</v-btn>
</v-card-actions>
</v-card>
</template>
<script setup>
import { ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { useAcceptOnEnter } from '../../composables/useAcceptOnEnter.js'
const props = defineProps({ category: { type: String, required: true } })
const emit = defineEmits(['confirm', 'cancel'])
const api = useApi()
const results = ref([])
const loading = ref(false)
const selectedId = ref(null)
let debounce = null
// Enter on the closed dropdown confirms the selection instead of re-opening it.
const { menuOpen, onEnter } = useAcceptOnEnter(() => {
if (selectedId.value != null) emit('confirm', selectedId.value)
})
function onSearch(q) {
if (debounce) clearTimeout(debounce)
debounce = setTimeout(async () => {
const query = (q || '').trim()
if (!query) { results.value = []; return }
loading.value = true
try {
// Scope the autocomplete to the prediction's category where it
// maps to a tag kind. Only 'character' surfaces as both a
// suggestion category and a tag kind now ('artist' + 'copyright'
// retired); other categories search unscoped.
const kind = props.category === 'character' ? 'character' : null
const params = { q: query, limit: 20 }
if (kind) params.kind = kind
results.value = await api.get('/api/tags/autocomplete', { params })
} finally {
loading.value = false
}
}, 200)
}
</script>
@@ -42,13 +42,23 @@
· {{ e.post.attachment_count }} files · {{ e.post.attachment_count }} files
</span> </span>
</div> </div>
<div v-if="hasTranslation(e)" class="fc-prov__actions">
<a
href="#" @click.prevent="toggleOrig(e.provenance_id)"
>{{ showOrig[e.provenance_id] ? 'Show translation' : `Show original${langLabel(e)}` }}</a>
</div>
<div v-if="e.post.description_html" class="fc-prov__actions"> <div v-if="e.post.description_html" class="fc-prov__actions">
<a <a
href="#" @click.prevent="toggleDesc(e.provenance_id)" href="#" @click.prevent="toggleDesc(e.provenance_id)"
>{{ expanded[e.provenance_id] ? 'Hide description ' : 'Show description ' }}</a> >{{ expanded[e.provenance_id] ? 'Hide description ' : 'Show description ' }}</a>
</div> </div>
<!-- Translated body = plain text; original = sanitized HTML (#143). -->
<p
v-if="showTranslated(e) && expanded[e.provenance_id] && e.post.description_translated"
class="fc-prov__desc"
>{{ e.post.description_translated }}</p>
<div <div
v-if="e.post.description_html && expanded[e.provenance_id]" v-else-if="e.post.description_html && expanded[e.provenance_id]"
class="fc-prov__desc" v-html="e.post.description_html" class="fc-prov__desc" v-html="e.post.description_html"
/> />
</article> </article>
@@ -117,8 +127,12 @@ const effectiveImage = computed(() => props.image ?? modal.current)
// 2026-05-28. Reset when the viewed image changes. // 2026-05-28. Reset when the viewed image changes.
const expanded = reactive({}) const expanded = reactive({})
function toggleDesc(id) { expanded[id] = !expanded[id] } function toggleDesc(id) { expanded[id] = !expanded[id] }
// Per-entry "show the original (untranslated) text" toggle (#143).
const showOrig = reactive({})
function toggleOrig(id) { showOrig[id] = !showOrig[id] }
watch(() => effectiveId.value, () => { watch(() => effectiveId.value, () => {
for (const k of Object.keys(expanded)) delete expanded[k] for (const k of Object.keys(expanded)) delete expanded[k]
for (const k of Object.keys(showOrig)) delete showOrig[k]
}) })
watch( watch(
@@ -157,10 +171,23 @@ const show = computed(() => {
const attachments = computed(() => state.value?.attachments || []) const attachments = computed(() => state.value?.attachments || [])
function postDate(e) { return formatPostDate(e.post.date) } function postDate(e) { return formatPostDate(e.post.date) }
// Translation-forward (#143): show the English by default when present.
function hasTranslation(e) {
return !!(e.post.title_translated || e.post.description_translated)
}
function showTranslated(e) {
return hasTranslation(e) && !showOrig[e.provenance_id]
}
function langLabel(e) {
return e.post.translated_source_lang ? ` (${e.post.translated_source_lang})` : ''
}
function postTitle(e) { function postTitle(e) {
// Titles can arrive as stored HTML (e.g. "<strong>…</strong>"); render // Titles can arrive as stored HTML (e.g. "<strong>…</strong>"); render
// as plain text (the CSS makes it bold). // as plain text (the CSS makes it bold).
return toPlainText(e.post.title) || `Post ${e.post.external_post_id}` const t = showTranslated(e) && e.post.title_translated
? e.post.title_translated
: e.post.title
return toPlainText(t) || `Post ${e.post.external_post_id}`
} }
function openPost(postId, artistId) { function openPost(postId, artistId) {
@@ -11,10 +11,6 @@
{{ suggestion.display_name }} {{ suggestion.display_name }}
<span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag" <span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag"
title="You rejected this for this image — un-reject to recover">rejected</span> title="You rejected this for this image — un-reject to recover">rejected</span>
<span v-else-if="suggestion.creates_new_tag" class="fc-suggestion__new"
title="No matching tag yet — accepting creates it">+ new</span>
<span v-else-if="suggestion.via_alias" class="fc-suggestion__alias"
:title="`Mapped from the tagger's “${suggestion.raw_name}” via an alias`">alias</span>
</span> </span>
<span class="fc-suggestion__score">{{ scorePct }}</span> <span class="fc-suggestion__score">{{ scorePct }}</span>
<!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval <!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval
@@ -46,40 +42,14 @@
@click="$emit('dismiss', suggestion)" @click="$emit('dismiss', suggestion)"
><v-icon size="16">mdi-close</v-icon></button> ><v-icon size="16">mdi-close</v-icon></button>
</div> </div>
<!-- Modal-safe kebab is baked into KebabMenu (this row lives in the
teleported image modal — #711). Only rendered when an alias action
applies — dismiss now lives on the red ✗, so a centroid hit with no
alias option has no menu. -->
<KebabMenu
v-if="hasMenu"
class="fc-suggestion__menu" size="small" variant="outlined"
:label="`More actions for ${suggestion.display_name}`"
>
<!-- Alias is a tagger-prediction remap, so only offer it for tagger
suggestions with a raw model key that aren't already aliased.
Centroid hits (raw_name null) have nothing to alias. -->
<v-list-item
v-if="suggestion.raw_name && !suggestion.via_alias"
@click="$emit('alias', suggestion)"
>
<v-list-item-title>Treat as alias for</v-list-item-title>
</v-list-item>
<v-list-item
v-if="suggestion.via_alias"
@click="$emit('remove-alias', suggestion)"
>
<v-list-item-title>Remove alias</v-list-item-title>
</v-list-item>
</KebabMenu>
</div> </div>
</template> </template>
<script setup> <script setup>
import { computed, inject } from 'vue' import { computed, inject } from 'vue'
import KebabMenu from '../common/KebabMenu.vue'
const props = defineProps({ suggestion: { type: Object, required: true } }) const props = defineProps({ suggestion: { type: Object, required: true } })
defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss']) defineEmits(['accept', 'dismiss', 'undismiss'])
// #1206: on hover, tell the image viewer which crop produced this suggestion so // #1206: on hover, tell the image viewer which crop produced this suggestion so
// it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere. // it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere.
@@ -92,12 +62,6 @@ function onLeave () {
} }
const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`) const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`)
// Kebab now only carries alias actions: show it when this suggestion can be
// aliased (raw model key, not yet aliased) or is already aliased (so it can be
// un-aliased). Centroid hits (no raw_name, no alias) have an empty menu → hide.
const hasMenu = computed(() =>
Boolean(props.suggestion.raw_name) || Boolean(props.suggestion.via_alias)
)
</script> </script>
<style scoped> <style scoped>
@@ -119,26 +83,6 @@ const hasMenu = computed(() =>
color: rgb(var(--v-theme-on-surface)); color: rgb(var(--v-theme-on-surface));
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
} }
.fc-suggestion__new {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-accent));
background: rgba(var(--v-theme-accent), 0.12);
border: 1px solid rgb(var(--v-theme-accent), 0.4);
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__alias {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-on-surface-variant));
background: rgb(var(--v-theme-surface-light));
border: 1px solid rgb(var(--v-theme-surface-light));
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__score { .fc-suggestion__score {
flex: 0 0 auto; min-width: 38px; text-align: right; flex: 0 0 auto; min-width: 38px; text-align: right;
font-size: 11px; font-size: 11px;
@@ -166,10 +110,6 @@ const hasMenu = computed(() =>
background: transparent; color: rgb(var(--v-theme-on-surface-variant)); background: transparent; color: rgb(var(--v-theme-on-surface-variant));
border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5); border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5);
} }
.fc-suggestion__menu {
flex: 0 0 auto;
}
/* Rejected state: the row stays put (recovery), dimmed + red-edged so it /* Rejected state: the row stays put (recovery), dimmed + red-edged so it
reads as "handled, negative" without shouting over live suggestions. */ reads as "handled, negative" without shouting over live suggestions. */
.fc-suggestion--rejected { .fc-suggestion--rejected {
@@ -1,22 +1,31 @@
<template> <template>
<div class="fc-sgroup"> <div class="fc-sgroup">
<button <div class="fc-sgroup__header-row">
v-if="collapsible" <button
class="fc-sgroup__header fc-sgroup__header--btn" v-if="collapsible"
@click="open = !open" class="fc-sgroup__header fc-sgroup__header--btn"
> @click="open = !open"
<v-icon size="small">{{ open ? 'mdi-chevron-down' : 'mdi-chevron-right' }}</v-icon> >
{{ label }} ({{ items.length }}) <v-icon size="small">{{ open ? 'mdi-chevron-down' : 'mdi-chevron-right' }}</v-icon>
</button> {{ label }} ({{ items.length }})
<div v-else class="fc-sgroup__header">{{ label }}</div> </button>
<div v-else class="fc-sgroup__header">{{ label }}</div>
<!-- "Confirm what's right, reject the rest": clears every still-unhandled
suggestion in this section at once. Reversible (each stays flagged
rejected with one-click un-reject), so no confirm dialog. -->
<button
v-if="rejectableCount > 0"
class="fc-sgroup__reject-rest" type="button"
:title="`Reject the ${rejectableCount} remaining ${label} suggestion${rejectableCount === 1 ? '' : 's'}`"
@click="$emit('reject-all')"
>Reject rest</button>
</div>
<div v-show="open" class="fc-sgroup__items"> <div v-show="open" class="fc-sgroup__items">
<SuggestionItem <SuggestionItem
v-for="(s, i) in items" :key="`${s.display_name}-${i}`" v-for="(s, i) in items" :key="`${s.display_name}-${i}`"
:suggestion="s" :suggestion="s"
@accept="$emit('accept', $event)" @accept="$emit('accept', $event)"
@alias="$emit('alias', $event)"
@remove-alias="$emit('remove-alias', $event)"
@dismiss="$emit('dismiss', $event)" @dismiss="$emit('dismiss', $event)"
@undismiss="$emit('undismiss', $event)" @undismiss="$emit('undismiss', $event)"
/> />
@@ -25,7 +34,7 @@
</template> </template>
<script setup> <script setup>
import { ref } from 'vue' import { computed, ref } from 'vue'
import SuggestionItem from './SuggestionItem.vue' import SuggestionItem from './SuggestionItem.vue'
const props = defineProps({ const props = defineProps({
@@ -34,24 +43,45 @@ const props = defineProps({
collapsible: { type: Boolean, default: false }, collapsible: { type: Boolean, default: false },
defaultOpen: { type: Boolean, default: true } defaultOpen: { type: Boolean, default: true }
}) })
defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss']) defineEmits(['accept', 'dismiss', 'undismiss', 'reject-all'])
// Still-unhandled suggestions (not yet rejected) — how many "Reject rest" clears.
const rejectableCount = computed(() => props.items.filter((s) => !s.rejected).length)
const open = ref(props.collapsible ? props.defaultOpen : true) const open = ref(props.collapsible ? props.defaultOpen : true)
</script> </script>
<style scoped> <style scoped>
.fc-sgroup { margin-bottom: 10px; } .fc-sgroup { margin-bottom: 10px; }
.fc-sgroup__header-row {
display: flex; align-items: center; gap: 8px;
margin-bottom: 4px;
}
.fc-sgroup__header { .fc-sgroup__header {
flex: 1 1 auto; min-width: 0;
font-family: 'Inter', sans-serif; font-family: 'Inter', sans-serif;
font-size: 11px; font-weight: 600; font-size: 11px; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.06em; text-transform: uppercase; letter-spacing: 0.06em;
color: rgb(var(--v-theme-on-surface-variant, var(--v-theme-on-surface))); color: rgb(var(--v-theme-on-surface-variant, var(--v-theme-on-surface)));
margin-bottom: 4px;
} }
.fc-sgroup__header--btn { .fc-sgroup__header--btn {
display: flex; align-items: center; gap: 4px; display: flex; align-items: center; gap: 4px;
background: none; border: none; cursor: pointer; background: none; border: none; cursor: pointer;
padding: 0; width: 100%; text-align: left; padding: 0; text-align: left;
font: inherit; text-transform: uppercase; letter-spacing: 0.06em; font: inherit; text-transform: uppercase; letter-spacing: 0.06em;
} }
/* Section-level "reject the remaining" — subtle until hovered so it doesn't
compete with the per-row verdicts. */
.fc-sgroup__reject-rest {
flex: 0 0 auto;
background: none; border: none; cursor: pointer;
font-family: 'Inter', sans-serif;
font-size: 10px; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.04em;
color: rgb(var(--v-theme-error));
padding: 2px 5px; border-radius: 4px;
}
.fc-sgroup__reject-rest:hover { background: rgb(var(--v-theme-error), 0.1); }
.fc-sgroup__reject-rest:focus-visible {
outline: 2px solid rgb(var(--v-theme-error)); outline-offset: 1px;
}
</style> </style>
@@ -20,45 +20,39 @@
first, so false positives are easy to spot and reject (reject-to-train first, so false positives are easy to spot and reject (reject-to-train
for these heads). Small set collapsible, open by default. --> for these heads). Small set collapsible, open by default. -->
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.byCategory.system && store.byCategory.system.length" v-if="store.aboveByCategory.system && store.aboveByCategory.system.length"
label="System" :items="store.byCategory.system" label="System" :items="store.aboveByCategory.system"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('system')"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-for="cat in peopleCats" :key="cat" v-for="cat in peopleCats" :key="cat"
v-show="store.byCategory[cat] && store.byCategory[cat].length" v-show="store.aboveByCategory[cat] && store.aboveByCategory[cat].length"
:label="labelFor(cat)" :items="store.byCategory[cat] || []" :label="labelFor(cat)" :items="store.aboveByCategory[cat] || []"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll(cat)"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.byCategory.general && store.byCategory.general.length" v-if="store.aboveByCategory.general && store.aboveByCategory.general.length"
label="General" :items="store.byCategory.general" label="General" :items="store.aboveByCategory.general"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('general')"
/> />
</div> </div>
<v-dialog v-model="aliasDialog" max-width="480">
<AliasPickerDialog
v-if="aliasTarget"
:category="aliasTarget.category"
@confirm="onAliasConfirm" @cancel="aliasDialog = false"
/>
</v-dialog>
</section> </section>
</template> </template>
<script setup> <script setup>
import { toast } from '../../utils/toast.js' import { toast } from '../../utils/toast.js'
import { computed, ref, watch } from 'vue' import { computed, watch } from 'vue'
import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js' import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js'
import { useModalStore } from '../../stores/modal.js' import { useModalStore } from '../../stores/modal.js'
import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue' import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue'
import AliasPickerDialog from './AliasPickerDialog.vue'
const props = defineProps({ const props = defineProps({
imageId: { type: Number, required: true }, imageId: { type: Number, required: true },
@@ -76,6 +70,16 @@ const emit = defineEmits(['accepted', 'dismissed'])
// re-focus the tag input — same return-to-input behaviour as accept. // re-focus the tag input — same return-to-input behaviour as accept.
function onDismiss (s) { store.dismiss(s); emit('dismissed') } function onDismiss (s) { store.dismiss(s); emit('dismissed') }
function onUndismiss (s) { store.undismiss(s); emit('dismissed') } function onUndismiss (s) { store.undismiss(s); emit('dismissed') }
// Section-level "Reject rest": dismiss every still-unhandled suggestion in the
// category, then return focus to the input like a single reject does.
async function onRejectAll (category) {
try {
await store.dismissRemaining(category)
emit('dismissed')
} catch (e) {
toast({ text: `Reject failed: ${e.message}`, type: 'error' })
}
}
const store = useSuggestionsStore() const store = useSuggestionsStore()
const modalStore = useModalStore() const modalStore = useModalStore()
const host = props.host || modalStore const host = props.host || modalStore
@@ -87,13 +91,14 @@ const peopleCats = ['character']
function labelFor(c) { return CATEGORY_LABELS[c] || c } function labelFor(c) { return CATEGORY_LABELS[c] || c }
const isEmpty = computed(() => const isEmpty = computed(() =>
Object.values(store.byCategory).every(list => !list || list.length === 0) Object.values(store.aboveByCategory).every(list => !list || list.length === 0)
) )
watch(() => props.imageId, (id) => { watch(() => props.imageId, (id) => {
if (id == null) return if (id == null) return
store.load(id) // panel: curated, ≥ threshold // One fetch (min=0) backs both surfaces: the panel reads aboveByCategory, the
store.loadAll(id) // dropdown: full prediction set (low-confidence included) // typed tag-input dropdown reads the full set. No second request.
store.load(id)
}, { immediate: true }) }, { immediate: true })
// After a successful accept/alias-accept, refresh the modal's current // After a successful accept/alias-accept, refresh the modal's current
@@ -112,31 +117,6 @@ async function onAccept(s) {
toast({ text: `Accept failed: ${e.message}`, type: 'error' }) toast({ text: `Accept failed: ${e.message}`, type: 'error' })
} }
} }
const aliasDialog = ref(false)
const aliasTarget = ref(null)
function onAlias(s) { aliasTarget.value = s; aliasDialog.value = true }
async function onAliasConfirm(canonicalTagId) {
try {
await store.aliasAccept(aliasTarget.value, canonicalTagId)
aliasDialog.value = false
await host.reloadTags()
emit('accepted')
} catch (e) {
toast({ text: `Alias failed: ${e.message}`, type: 'error' })
}
}
// Undo the model-key→tag mapping behind an aliased suggestion. The store
// reloads suggestions so the prediction reverts to its raw form; the applied
// canonical tag (if any) stays, so no tag-rail reload is needed.
async function onRemoveAlias(s) {
try {
await store.removeAlias(s)
} catch (e) {
toast({ text: `Remove alias failed: ${e.message}`, type: 'error' })
}
}
</script> </script>
<style scoped> <style scoped>
@@ -17,7 +17,14 @@
density="compact" class="fc-tag-autocomplete__list" density="compact" class="fc-tag-autocomplete__list"
> >
<template v-for="(row, idx) in rows" :key="row.key"> <template v-for="(row, idx) in rows" :key="row.key">
<!-- Existing tag match (server autocomplete). --> <!-- One unified row per DB tag (server autocomplete). Every suggestion is
a canonical tag now, so when the model ALSO scored this tag for the
image the row just carries its confidence (🔧 %) in place of the
redundant kind label the coloured leading icon already shows kind.
That means a searched tag that's also a suggestion shows its % on ONE
row, with no dedup and no flicker. Picking it emits pick-existing;
TagPanel.findPending routes a matching suggestion through accept() so
the acceptance is still recorded + it drops from the panel. -->
<v-list-item <v-list-item
v-if="row.type === 'hit'" v-if="row.type === 'hit'"
:active="idx === highlight" @click="onPickRow(row)" :active="idx === highlight" @click="onPickRow(row)"
@@ -32,32 +39,18 @@
<span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span> <span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span>
</v-list-item-title> </v-list-item-title>
<template #append> <template #append>
<span class="text-caption">{{ row.hit.kind }}</span> <span
</template> v-if="row.sugg"
</v-list-item> class="fc-tag-autocomplete__sugg-tag"
:class="{ 'fc-tag-autocomplete__sugg-tag--below': !row.sugg.above_threshold }"
<!-- ML suggestion for THIS image that matches the typed query. Picking :title="row.sugg.above_threshold
it routes through the same accept path as the Suggestions panel, so ? 'The model suggests this tag for this image'
it's recorded + drops out of the panel (operator-asked 2026-06-07). --> : 'The model scored this tag below its suggest threshold'"
<v-list-item >
v-else-if="row.type === 'suggestion'"
:active="idx === highlight" @click="onPickRow(row)"
class="fc-tag-autocomplete__sugg"
>
<template #prepend>
<v-icon size="small" :color="store.colorFor(row.sugg.category)">
{{ iconFor(row.sugg.category) }}
</v-icon>
</template>
<v-list-item-title>
{{ row.sugg.display_name }}
<span v-if="row.sugg.creates_new_tag" class="text-caption">— new</span>
</v-list-item-title>
<template #append>
<span class="fc-tag-autocomplete__sugg-tag">
<v-icon size="x-small">mdi-auto-fix</v-icon> <v-icon size="x-small">mdi-auto-fix</v-icon>
{{ scorePct(row.sugg) }} {{ scorePct(row.sugg) }}
</span> </span>
<span v-else class="text-caption">{{ row.hit.kind }}</span>
</template> </template>
</v-list-item> </v-list-item>
@@ -97,13 +90,13 @@
import { computed, nextTick, onMounted, ref, watch } from 'vue' import { computed, nextTick, onMounted, ref, watch } from 'vue'
import { useTagStore } from '../../stores/tags.js' import { useTagStore } from '../../stores/tags.js'
import { useSuggestionsStore } from '../../stores/suggestions.js' import { useSuggestionsStore } from '../../stores/suggestions.js'
import { useInflightToken } from '../../composables/useInflightToken.js'
import FandomPicker from './FandomPicker.vue' import FandomPicker from './FandomPicker.vue'
const emit = defineEmits(['pick-existing', 'pick-new', 'accept-suggestion', 'cancel']) const emit = defineEmits(['pick-existing', 'pick-new', 'cancel'])
// The host surface's applied tags (host.current?.tags). Both dropdown sections // The host surface's applied tags (host.current?.tags). The dropdown filters
// filter against this so a just-added tag drops out of the search the moment // against this so a just-added tag drops out of the search the moment the chip
// the chip rail updates, not after the next modal refresh (operator-asked // rail updates, not after the next modal refresh (operator-asked 2026-07-03).
// 2026-07-03).
const props = defineProps({ const props = defineProps({
appliedTags: { type: Array, default: () => [] }, appliedTags: { type: Array, default: () => [] },
}) })
@@ -183,17 +176,26 @@ const parsed = computed(() => {
const parsedKind = computed(() => parsed.value.kind) const parsedKind = computed(() => parsed.value.kind)
const parsedName = computed(() => parsed.value.name) const parsedName = computed(() => parsed.value.name)
// Inflight guard: the debounce only clears the TIMER, so once a fetch has fired
// it still races later ones — a slower earlier-prefix response ("s") could land
// after "sex" and overwrite the dropdown with stale, wrong-prefix matches
// (operator-flagged 2026-07-06). Gate each response on a token so only the latest
// query's results are applied.
const acInflight = useInflightToken()
let debounceId = null let debounceId = null
watch(query, () => { watch(query, () => {
highlight.value = 0 highlight.value = 0
if (debounceId) clearTimeout(debounceId) if (debounceId) clearTimeout(debounceId)
acInflight.cancel()
debounceId = setTimeout(async () => { debounceId = setTimeout(async () => {
const q = parsedName.value const q = parsedName.value
if (!q) { hits.value = []; return } if (!q) { hits.value = []; return }
// Autocomplete across ALL kinds. When the user typed a prefix the // Autocomplete across ALL kinds. When the user typed a prefix the
// matches list is naturally narrower because the parsed name is // matches list is naturally narrower because the parsed name is
// shorter; we don't filter server-side by kind. // shorter; we don't filter server-side by kind.
hits.value = await store.autocomplete(q, null, 10) const t = acInflight.claim()
const res = await store.autocomplete(q, null, 10)
if (t.isCurrent()) hits.value = res
}, 200) }, 200)
}) })
@@ -228,9 +230,6 @@ const createLabel = computed(() =>
function scorePct (s) { return `${Math.round(s.score * 100)}%` } function scorePct (s) { return `${Math.round(s.score * 100)}%` }
const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id))) const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id)))
const appliedNames = computed(() =>
new Set(props.appliedTags.map(t => `${t.kind}:${(t.name || '').toLowerCase()}`)),
)
// Server matches minus the image's applied tags. allowCreate/sameNameCharExists // Server matches minus the image's applied tags. allowCreate/sameNameCharExists
// keep reading the RAW hits — they reason about the tag universe (does this tag // keep reading the RAW hits — they reason about the tag universe (does this tag
@@ -239,49 +238,29 @@ const visibleHits = computed(() =>
hits.value.filter(h => !appliedIds.value.has(h.id)), hits.value.filter(h => !appliedIds.value.has(h.id)),
) )
// This image's suggestions that match the typed query, minus any the server // This image's suggestions keyed by canonical tag id, so a matching DB-tag row
// autocomplete already returned (same name+kind) so a tag never shows twice. // can show the model's confidence inline. Every suggestion is a canonical tag
// Sources the FULL prediction set (allByCategory, down to the store floor) — NOT // now (tagging-v2), so the id is the join key — no name/kind matching, no dedup,
// the threshold-filtered panel list — so a low-confidence action/feature the // no flicker. Rejected suggestions are excluded: a dismissed tag shouldn't
// model saw can be typed and accepted in canonical formatting instead of being // advertise a score in the type-to-add dropdown (un-reject lives in the panel).
// hand-entered as a custom tag (operator-asked 2026-06-09). The typed query is const suggByTagId = computed(() => {
// the only filter; the threshold no longer hides anything here. const m = new Map()
const suggestionHits = computed(() => { for (const list of Object.values(suggestions.byCategory)) {
const q = parsedName.value.toLowerCase()
if (!q) return []
const seen = new Set(visibleHits.value.map(h => `${h.kind}:${h.name.toLowerCase()}`))
const out = []
for (const list of Object.values(suggestions.allByCategory)) {
for (const s of list || []) { for (const s of list || []) {
// Rejected suggestions now stay in allByCategory (flagged) so the panel
// can show + un-reject them; keep them OUT of the type-to-add dropdown,
// whose job is finding a tag to ADD (un-reject lives in the panel).
if (s.rejected) continue if (s.rejected) continue
// Already on the image (matched by canonical id or name+kind): nothing if (s.canonical_tag_id != null) m.set(s.canonical_tag_id, s)
// to add. Covers the window between an add and the next suggestions
// fetch, where the stale row would otherwise still be pickable.
if (s.canonical_tag_id != null && appliedIds.value.has(s.canonical_tag_id)) continue
const key = `${s.category}:${s.display_name.toLowerCase()}`
if (appliedNames.value.has(key)) continue
if (!s.display_name.toLowerCase().includes(q)) continue
if (seen.has(key)) continue
seen.add(key)
out.push(s)
} }
} }
// Best matches first; cap generously so a specific typed query surfaces its return m
// matches even when many predictions exist, while the list stays scrollable.
out.sort((a, b) => b.score - a.score)
return out.slice(0, 20)
}) })
// One ordered list backing both the rendered dropdown and keyboard nav, so the // One ordered list backing both the rendered dropdown and keyboard nav, so the
// highlight index maps 1:1 to a row regardless of which section it's in. // highlight index maps 1:1 to a row. Each hit is annotated with its suggestion
// (score) when the model scored that tag for this image.
const rows = computed(() => { const rows = computed(() => {
const r = visibleHits.value.map(h => ({ type: 'hit', key: `h${h.id}`, hit: h })) const r = visibleHits.value.map(h => ({
for (const s of suggestionHits.value) { type: 'hit', key: `h${h.id}`, hit: h, sugg: suggByTagId.value.get(h.id) || null,
r.push({ type: 'suggestion', key: `s${s.category}:${s.display_name}`, sugg: s }) }))
}
if (allowCreate.value) r.push({ type: 'create', key: 'create' }) if (allowCreate.value) r.push({ type: 'create', key: 'create' })
return r return r
}) })
@@ -303,16 +282,9 @@ function moveHighlight (delta) {
// Dispatch a chosen dropdown row by its type. // Dispatch a chosen dropdown row by its type.
function onPickRow (row) { function onPickRow (row) {
if (row.type === 'hit') { emit('pick-existing', row.hit); reset() } if (row.type === 'hit') { emit('pick-existing', row.hit); reset() }
else if (row.type === 'suggestion') { onPickSuggestion(row.sugg) }
else { onCreate() } else { onCreate() }
} }
// Picking an ML suggestion hands it to the parent, which runs the same accept
// flow as the Suggestions panel (creates the tag if raw, records acceptance,
// drops it from the panel). reset() clears the query; the panel-drop makes it
// fall out of suggestionHits reactively.
function onPickSuggestion (s) { emit('accept-suggestion', s); reset() }
function onCreate () { function onCreate () {
const name = parsedName.value const name = parsedName.value
const kind = parsedKind.value const kind = parsedKind.value
@@ -379,14 +351,16 @@ function reset () { query.value = ''; hits.value = []; highlight.value = 0 }
border: 1px solid rgb(var(--v-theme-surface-light)); border: 1px solid rgb(var(--v-theme-surface-light));
border-radius: 6px; border-radius: 6px;
} }
/* Mark ML-suggestion rows so they read as distinct from typed/known matches. */ /* The model-confidence badge on a row whose tag the model also scored. Accent
.fc-tag-autocomplete__sugg { when above the head's suggest threshold; muted when below (a low-confidence
border-left: 2px solid rgb(var(--v-theme-accent), 0.5); match you can still type + pick). */
}
.fc-tag-autocomplete__sugg-tag { .fc-tag-autocomplete__sugg-tag {
display: inline-flex; align-items: center; gap: 2px; display: inline-flex; align-items: center; gap: 2px;
font-size: 11px; font-size: 11px;
font-family: 'JetBrains Mono', monospace; font-family: 'JetBrains Mono', monospace;
color: rgb(var(--v-theme-accent)); color: rgb(var(--v-theme-accent));
} }
.fc-tag-autocomplete__sugg-tag--below {
color: rgb(var(--v-theme-on-surface-variant));
}
</style> </style>
+79 -5
View File
@@ -1,7 +1,7 @@
<template> <template>
<span class="fc-tag-chip" @mouseenter="onEnter" @mouseleave="onLeave"> <span class="fc-tag-chip" @mouseenter="onEnter" @mouseleave="onLeave">
<v-chip <v-chip
size="small" closable size="default" :closable="!unconfirmedAuto"
:color="store.colorFor(tag.kind)" variant="tonal" :color="store.colorFor(tag.kind)" variant="tonal"
class="fc-tag-chip__nav" class="fc-tag-chip__nav"
role="link" role="link"
@@ -9,14 +9,32 @@
@click="$emit('navigate', tag)" @click="$emit('navigate', tag)"
@click:close="$emit('remove', tag.id)" @click:close="$emit('remove', tag.id)"
> >
<v-icon start size="x-small">{{ iconFor(tag.kind) }}</v-icon> <v-icon start size="small">{{ iconFor(tag.kind) }}</v-icon>
{{ tag.name }}<v-icon <span class="fc-tag-chip__name">{{ tag.name }}</span><v-icon
v-if="tag.is_system" end size="x-small" class="fc-tag-chip__system" v-if="tag.is_system" end size="x-small" class="fc-tag-chip__system"
title="System tag — tagged items are excluded from training other concepts" title="System tag — tagged items are excluded from training other concepts"
>mdi-shield-outline</v-icon><span >mdi-shield-outline</v-icon><span
v-if="tag.fandom_id" class="fc-tag-chip__fandom" v-if="tag.fandom_id" class="fc-tag-chip__fandom"
:title="tag.fandom_name ? `Fandom: ${tag.fandom_name}` : 'Has a fandom'" :title="tag.fandom_name ? `Fandom: ${tag.fandom_name}` : 'Has a fandom'"
><template v-if="fandomLabel">&nbsp;{{ fandomLabel }}</template></span> ><template v-if="fandomLabel">&nbsp;{{ fandomLabel }}</template></span>
<!-- Provisional auto-tag: an in-pill yes/no pair REPLACES the (the pair
itself signals "auto" no separate label, operator 2026-07-07). Yes
confirms it (trains + shields from retraction); No removes it (records
a negative). Both hand focus back to the tag input via TagPanel. -->
<span v-if="unconfirmedAuto" class="fc-tag-chip__verdict">
<button
type="button" class="fc-tag-chip__yes"
:title="`Yes — keep “${tag.name}” (trains the model, won't be retracted)`"
:aria-label="`Confirm ${tag.name}`"
@click.stop="$emit('confirm', tag)"
><v-icon size="15">mdi-check</v-icon></button>
<button
type="button" class="fc-tag-chip__no"
:title="`No — remove “${tag.name}”`"
:aria-label="`Reject ${tag.name}`"
@click.stop="$emit('remove', tag.id)"
><v-icon size="15">mdi-close</v-icon></button>
</span>
</v-chip> </v-chip>
<!-- Modal-safe kebab is baked into KebabMenu (this chip lives in the <!-- Modal-safe kebab is baked into KebabMenu (this chip lives in the
teleported image modal #711). System tags hide it entirely: rename teleported image modal #711). System tags hide it entirely: rename
@@ -43,6 +61,8 @@ import { useTagStore } from '../../stores/tags.js'
import { useApi } from '../../composables/useApi.js' import { useApi } from '../../composables/useApi.js'
import KebabMenu from '../common/KebabMenu.vue' import KebabMenu from '../common/KebabMenu.vue'
const AUTO_SOURCES = ['head_auto', 'ccip_auto', 'ml_auto']
const props = defineProps({ const props = defineProps({
tag: { type: Object, required: true }, tag: { type: Object, required: true },
// When set (the tagging panels), hovering the chip asks the backend which crop // When set (the tagging panels), hovering the chip asks the backend which crop
@@ -51,11 +71,19 @@ const props = defineProps({
// the hover is inert (no injected target, or no image to ground against). // the hover is inert (no injected target, or no image to ground against).
imageId: { type: Number, default: null }, imageId: { type: Number, default: null },
}) })
defineEmits(['remove', 'rename', 'set-fandom', 'navigate']) defineEmits(['remove', 'rename', 'set-fandom', 'navigate', 'confirm'])
const store = useTagStore() const store = useTagStore()
const api = useApi() const api = useApi()
// An auto-applied tag the operator hasn't confirmed yet — provisional (milestone
// 139): it doesn't train the model and the retraction sweep can drop it. Shows an
// in-pill yes/no pair in place of the ✕. `source`/`confirmed` come from the
// applied-tags payload (list_for_image / get_image_with_tags).
const unconfirmedAuto = computed(() =>
AUTO_SOURCES.includes(props.tag.source) && !props.tag.confirmed
)
// #1206 Step 4: applied-tag grounding. `fcSuggestionHover` is provided by the // #1206 Step 4: applied-tag grounding. `fcSuggestionHover` is provided by the
// image viewer / Explore host (a no-op elsewhere). Applied tags aren't scored // image viewer / Explore host (a no-op elsewhere). Applied tags aren't scored
// live, so we fetch the winning region on demand and cache it per (image, tag). // live, so we fetch the winning region on demand and cache it per (image, tag).
@@ -108,7 +136,30 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
</script> </script>
<style scoped> <style scoped>
.fc-tag-chip { display: inline-flex; align-items: center; gap: 1px; } /* A chip must never exceed the rail — a long character+fandom chip used to
overflow the right edge and clip its trailing control (operator-flagged
2026-07-06). The name is the elastic part: it ellipsis-truncates so the ✕ (or
the auto-tag yes/no pair) and fandom stay reachable. Full name is on the
chip's hover title. */
.fc-tag-chip { display: inline-flex; align-items: center; gap: 1px; max-width: 100%; min-width: 0; }
.fc-tag-chip__nav { max-width: 100%; min-width: 0; }
/* Tonal chips (esp. character = info) wash out against the dark rail — the fill
is intentionally faint. Give every tag chip a thin border in its OWN kind
colour (currentColor = the tonal chip's themed foreground) so the edge reads
clearly without touching the fill (operator-asked 2026-07-08). Theme-aware:
currentColor tracks the kind colour in either light or dark. */
.fc-tag-chip__nav { border: thin solid color-mix(in srgb, currentColor 55%, transparent); }
.fc-tag-chip__nav :deep(.v-chip__content) { min-width: 0; overflow: hidden; }
/* The bigger size=default chip widens Vuetify's negative start-margin on the
leading kind-icon, pulling it LEFT of the content box above — where that
overflow:hidden then clips the icon's left edge as a vertical slice
(operator-flagged 2026-07-08). Zero the pull so the icon sits inside the clip
box; the chip's own 12px padding keeps a comfortable left inset. */
.fc-tag-chip__nav :deep(.v-icon--start) { margin-inline-start: 0; }
.fc-tag-chip__name {
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
min-width: 0; flex: 0 1 auto;
}
/* The chip body navigates to the filtered gallery (#5); signal it's clickable. /* The chip body navigates to the filtered gallery (#5); signal it's clickable.
The close ✕ (remove) and the sibling kebab stay as the explicit controls. */ The close ✕ (remove) and the sibling kebab stay as the explicit controls. */
.fc-tag-chip__nav { cursor: pointer; } .fc-tag-chip__nav { cursor: pointer; }
@@ -120,4 +171,27 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
.fc-tag-chip__kebab { opacity: 0.7; } .fc-tag-chip__kebab { opacity: 0.7; }
.fc-tag-chip:hover .fc-tag-chip__kebab { opacity: 1; } .fc-tag-chip:hover .fc-tag-chip__kebab { opacity: 1; }
.fc-tag-chip__fandom { opacity: 0.7; font-size: 0.85em; } .fc-tag-chip__fandom { opacity: 0.7; font-size: 0.85em; }
/* Provisional auto-tag: a green ✓ / red ✗ pair in place of the ✕. The yes/no is
obvious enough on its own, so there's no "auto" label (operator-asked
2026-07-07). flex:0 0 auto keeps it visible while the name ellipsis-truncates. */
.fc-tag-chip__verdict {
flex: 0 0 auto; display: inline-flex; align-items: center; gap: 3px;
margin-left: 5px;
}
/* Solid-filled (white glyph on a full success/error circle) so accept/reject
stay well-defined even on a muted tonal chip — e.g. character (info) — instead
of a faint icon that only lit up on hover. Mirrors the Suggestions rail's
verdict buttons so the affordance reads identically (operator-asked 2026-07-08). */
.fc-tag-chip__yes, .fc-tag-chip__no {
display: inline-flex; align-items: center; justify-content: center;
width: 22px; height: 22px; padding: 0; border: none; border-radius: 50%;
color: #fff; cursor: pointer; opacity: 0.92;
transition: transform 0.1s, opacity 0.1s;
}
.fc-tag-chip__yes { background: rgb(var(--v-theme-success)); }
.fc-tag-chip__no { background: rgb(var(--v-theme-error)); }
.fc-tag-chip__yes:hover, .fc-tag-chip__no:hover { opacity: 1; transform: scale(1.1); }
.fc-tag-chip__yes:focus-visible, .fc-tag-chip__no:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 2px;
}
</style> </style>
+19 -15
View File
@@ -6,7 +6,7 @@
v-for="tag in host.current?.tags || []" v-for="tag in host.current?.tags || []"
:key="tag.id" :tag="tag" :image-id="host.currentImageId" :key="tag.id" :tag="tag" :image-id="host.currentImageId"
@remove="onRemove" @rename="openRename" @set-fandom="openSetFandom" @remove="onRemove" @rename="openRename" @set-fandom="openSetFandom"
@navigate="onNavigate" @navigate="onNavigate" @confirm="onConfirm"
/> />
<span v-if="!host.current?.tags?.length" class="text-caption">No tags yet.</span> <span v-if="!host.current?.tags?.length" class="text-caption">No tags yet.</span>
</div> </div>
@@ -17,7 +17,6 @@
ref="tagInputRef" ref="tagInputRef"
:applied-tags="host.current?.tags || []" :applied-tags="host.current?.tags || []"
@pick-existing="onPickExisting" @pick-new="onPickNew" @pick-existing="onPickExisting" @pick-new="onPickNew"
@accept-suggestion="onAcceptSuggestion"
/> />
<v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable> <v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable>
@@ -65,6 +64,7 @@
<script setup> <script setup>
import { ref } from 'vue' import { ref } from 'vue'
import { useRouter } from 'vue-router' import { useRouter } from 'vue-router'
import { useApi } from '../../composables/useApi.js'
import { useModalStore } from '../../stores/modal.js' import { useModalStore } from '../../stores/modal.js'
import { useSuggestionsStore } from '../../stores/suggestions.js' import { useSuggestionsStore } from '../../stores/suggestions.js'
import TagChip from './TagChip.vue' import TagChip from './TagChip.vue'
@@ -83,6 +83,7 @@ const modalStore = useModalStore()
const host = props.host || modalStore const host = props.host || modalStore
const suggestions = useSuggestionsStore() const suggestions = useSuggestionsStore()
const router = useRouter() const router = useRouter()
const api = useApi()
const errorMsg = ref(null) const errorMsg = ref(null)
const tagInputRef = ref(null) const tagInputRef = ref(null)
@@ -106,6 +107,22 @@ defineExpose({ focusTagInput })
// Every tag mutation hands focus back to the input so the operator can keep // Every tag mutation hands focus back to the input so the operator can keep
// typing the next tag without re-clicking — matches the accept-suggestion flow // typing the next tag without re-clicking — matches the accept-suggestion flow
// (operator-asked 2026-06-26; the Explore workspace leans on this hard). // (operator-asked 2026-06-26; the Explore workspace leans on this hard).
// Confirm/keep an auto-applied tag (milestone 139): records the affirmation so
// the tag becomes a training positive AND is shielded from the retraction sweep,
// then reloads so the chip drops its "auto" badge + Keep button.
async function onConfirm(tag) {
errorMsg.value = null
try {
await api.post(`/api/images/${host.currentImageId}/tags/${tag.id}/confirm`)
await host.reloadTags()
// Return the cursor to the tag input — its resting position in these views,
// matching every other chip action (remove/accept) so the keyboard flow
// never leaves the field (operator-asked 2026-07-07).
focusTagInput()
}
catch (e) { errorMsg.value = e.message }
}
async function onRemove(tagId) { async function onRemove(tagId) {
errorMsg.value = null errorMsg.value = null
try { try {
@@ -116,7 +133,6 @@ async function onRemove(tagId) {
// until the next modal open (operator-asked 2026-07-03). // until the next modal open (operator-asked 2026-07-03).
if (host.currentImageId != null) { if (host.currentImageId != null) {
suggestions.load(host.currentImageId) suggestions.load(host.currentImageId)
suggestions.loadAll(host.currentImageId)
} }
focusTagInput() focusTagInput()
} }
@@ -160,18 +176,6 @@ async function onPickNew(payload) {
} }
catch (e) { errorMsg.value = e.message } catch (e) { errorMsg.value = e.message }
} }
// A suggestion picked from the autocomplete dropdown runs the SAME path as the
// Suggestions panel's Accept: the store creates the tag if it's raw, records the
// acceptance, and drops it from the panel; then we refresh the chip rail.
async function onAcceptSuggestion(s) {
errorMsg.value = null
try {
await suggestions.accept(s)
await host.reloadTags()
focusTagInput()
} catch (e) { errorMsg.value = e.message }
}
const renameDialog = ref(false) const renameDialog = ref(false)
const renameTarget = ref(null) const renameTarget = ref(null)
+60 -5
View File
@@ -58,16 +58,31 @@
</div> </div>
<div class="fc-post-card__text"> <div class="fc-post-card__text">
<h3 v-if="plainTitle" class="fc-post-card__title">{{ plainTitle }}</h3> <h3 v-if="displayTitle" class="fc-post-card__title">{{ displayTitle }}</h3>
<h3 v-else class="fc-post-card__title fc-post-card__title--missing"> <h3 v-else class="fc-post-card__title fc-post-card__title--missing">
Post {{ post.external_post_id }} Post {{ post.external_post_id }}
</h3> </h3>
<!-- Translation toggle (#143): English shown by default; reveal the
original, labelled with its detected language. -->
<button
v-if="hasTranslation" type="button" class="fc-post-card__lang"
:title="showOriginal ? 'Show the English translation' : `Show the original${sourceLangLabel} text`"
@click.stop="showOriginal = !showOriginal"
>{{ showOriginal ? 'Show translation' : `Show original${sourceLangLabel}` }}</button>
<!-- Per-post translation override (#155): force a skipped translation on,
or keep the original for a confidently mis-flagged title. Only where
there's text to translate. -->
<PostTranslationControl
v-if="post.post_title || post.description_plain"
:post="post"
/>
<!-- Faithful (semantic) body render once expanded: backend-sanitized <!-- Faithful (semantic) body render once expanded: backend-sanitized
HTML (headings, lists, links, inline images). Collapsed and the HTML (headings, lists, links, inline images). Collapsed and the
no-detail fallback stay plain text. --> no-detail fallback stay plain text. -->
<div <div
v-if="hasDescription && descExpanded && descHtml" v-if="hasDescription && showHtmlBody"
class="fc-post-card__desc fc-post-card__desc--html" class="fc-post-card__desc fc-post-card__desc--html"
v-html="descHtml" v-html="descHtml"
/> />
@@ -75,7 +90,7 @@
v-else-if="hasDescription" ref="descEl" v-else-if="hasDescription" ref="descEl"
class="fc-post-card__desc" class="fc-post-card__desc"
:class="{ 'fc-post-card__desc--clamped': !descExpanded }" :class="{ 'fc-post-card__desc--clamped': !descExpanded }"
>{{ descText }}</p> >{{ descTextDisplay }}</p>
<p v-else class="fc-post-card__desc fc-post-card__desc--missing"> <p v-else class="fc-post-card__desc fc-post-card__desc--missing">
(no description) (no description)
</p> </p>
@@ -129,6 +144,7 @@ import { useModalStore } from '../../stores/modal.js'
import { usePostsStore } from '../../stores/posts.js' import { usePostsStore } from '../../stores/posts.js'
import { toPlainText } from '../../utils/htmlSanitize.js' import { toPlainText } from '../../utils/htmlSanitize.js'
import PostSeriesMenu from './PostSeriesMenu.vue' import PostSeriesMenu from './PostSeriesMenu.vue'
import PostTranslationControl from './PostTranslationControl.vue'
const props = defineProps({ const props = defineProps({
post: { type: Object, required: true }, post: { type: Object, required: true },
@@ -200,13 +216,13 @@ async function fullImageIds () {
} }
async function openModal (imageId) { async function openModal (imageId) {
modal.open(imageId, { postImageIds: await fullImageIds() }) modal.open(imageId, { playlistIds: await fullImageIds() })
} }
async function openModalAtMore () { async function openModalAtMore () {
const ids = await fullImageIds() const ids = await fullImageIds()
const first = ids[visibleCount.value] ?? ids[0] const first = ids[visibleCount.value] ?? ids[0]
if (first != null) modal.open(first, { postImageIds: ids }) if (first != null) modal.open(first, { playlistIds: ids })
} }
// --- description "Show more" (text-only, in place, only when truncated) ---- // --- description "Show more" (text-only, in place, only when truncated) ----
@@ -231,6 +247,32 @@ const canExpand = computed(
() => props.post.description_truncated === true || cssOverflow.value, () => props.post.description_truncated === true || cssOverflow.value,
) )
// --- translation-forward (#143): the English is shown by default when a
// translation exists; a per-card toggle reveals the original. Translations are
// plain text, so showTranslated also decides HTML-vs-plain body rendering.
const showOriginal = ref(false)
const hasTranslation = computed(
() => !!(props.post.post_title_translated || props.post.description_translated),
)
const showTranslated = computed(() => hasTranslation.value && !showOriginal.value)
const sourceLangLabel = computed(() =>
props.post.translated_source_lang ? ` (${props.post.translated_source_lang})` : '',
)
const displayTitle = computed(() =>
showTranslated.value && props.post.post_title_translated
? toPlainText(props.post.post_title_translated)
: plainTitle.value,
)
const descTextDisplay = computed(() => {
if (!showTranslated.value) return descText.value
return descExpanded.value
? (detail.value?.description_translated_full || props.post.description_translated)
: props.post.description_translated
})
const showHtmlBody = computed(
() => !showTranslated.value && descExpanded.value && !!descHtml.value,
)
function measureOverflow () { function measureOverflow () {
const el = descEl.value const el = descEl.value
cssOverflow.value = !!el && el.scrollHeight > el.clientHeight + 1 cssOverflow.value = !!el && el.scrollHeight > el.clientHeight + 1
@@ -467,6 +509,19 @@ function formatBytes (n) {
font-size: 0.85rem; font-weight: 600; font-size: 0.85rem; font-weight: 600;
} }
.fc-post-card__more:hover { text-decoration: underline; } .fc-post-card__more:hover { text-decoration: underline; }
/* Translation toggle (#143) — quiet secondary affordance under the title. */
.fc-post-card__lang {
display: block; margin: 2px 0 6px; padding: 0;
background: none; border: 0; cursor: pointer;
color: rgb(var(--v-theme-on-surface-variant));
font-size: 0.75rem; font-weight: 600;
}
.fc-post-card__lang:hover {
text-decoration: underline; color: rgb(var(--v-theme-accent));
}
.fc-post-card__lang:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
}
.fc-post-card__atts { .fc-post-card__atts {
display: flex; flex-wrap: wrap; gap: 8px; display: flex; flex-wrap: wrap; gap: 8px;
@@ -0,0 +1,102 @@
<template>
<!-- Sticky per-post translation override (#155). Quiet inline control under the
post title: force a skipped legit-foreign title on, or knock a wrongly
mis-translated one back to the original. The choice sticks through a
Re-translate-all. -->
<div class="fc-post-tx">
<span class="fc-post-tx__label">Translation:</span>
<v-menu :disabled="busy">
<template #activator="{ props: menuProps }">
<button
type="button" class="fc-post-tx__btn" v-bind="menuProps" :disabled="busy"
:aria-label="`Translation handling: ${currentLabel}`"
>
{{ currentLabel }}
<v-icon size="14">mdi-menu-down</v-icon>
</button>
</template>
<v-list density="compact" min-width="240" class="fc-post-tx__list">
<v-list-item
v-for="opt in OPTIONS" :key="opt.value"
:active="opt.value === current" @click="choose(opt.value)"
>
<template #prepend>
<v-icon size="small">{{ opt.icon }}</v-icon>
</template>
<v-list-item-title>{{ opt.label }}</v-list-item-title>
<v-list-item-subtitle>{{ opt.help }}</v-list-item-subtitle>
</v-list-item>
</v-list>
</v-menu>
<v-progress-circular v-if="busy" indeterminate size="13" width="2" />
</div>
</template>
<script setup>
import { computed, ref } from 'vue'
import { usePostsStore } from '../../stores/posts.js'
import { toast } from '../../utils/toast.js'
const props = defineProps({ post: { type: Object, required: true } })
const posts = usePostsStore()
const busy = ref(false)
const OPTIONS = [
{ value: 'auto', label: 'Auto', icon: 'mdi-cog-outline',
help: 'Translate only when confident enough' },
{ value: 'force', label: 'Force translate', icon: 'mdi-translate',
help: 'Always translate, even at low confidence' },
{ value: 'original', label: 'Keep original', icon: 'mdi-translate-off',
help: 'Never translate — keep the original text' },
]
const current = computed(() => props.post.translation_override || 'auto')
const currentLabel = computed(
() => (OPTIONS.find((o) => o.value === current.value) || OPTIONS[0]).label,
)
async function choose (value) {
if (value === current.value || busy.value) return
busy.value = true
try {
const res = await posts.applyTranslationOverride(props.post.id, value)
if (res.applied === 'queued') {
toast({
text: 'Saved — this post will translate on the next sweep (service offline).',
type: 'info',
})
}
} catch (e) {
toast({ text: `Couldn't update translation: ${e.message}`, type: 'error' })
} finally {
busy.value = false
}
}
</script>
<style scoped>
.fc-post-tx {
display: inline-flex; align-items: center; gap: 6px;
margin: 2px 0 6px;
}
.fc-post-tx__label {
font-size: 0.72rem;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn {
display: inline-flex; align-items: center; gap: 1px;
padding: 0; background: none; border: 0; cursor: pointer;
font-size: 0.72rem; font-weight: 700;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn:hover { color: rgb(var(--v-theme-accent)); }
.fc-post-tx__btn:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
border-radius: 2px;
}
.fc-post-tx__btn:disabled { cursor: default; opacity: 0.6; }
.fc-post-tx__list :deep(.v-list-item-subtitle) {
font-size: 0.7rem;
}
</style>
@@ -159,6 +159,42 @@
</div> </div>
</div> </div>
<!-- Presentation chrome auto-hide (#141) -->
<div class="fc-auto mt-6">
<div class="d-flex align-center mb-1" style="gap: 10px;">
<v-icon size="18" color="accent">mdi-image-off-outline</v-icon>
<span class="fc-section-h">Hide presentation chrome</span>
<v-switch
v-model="presentationEnabled" :loading="settingBusy" hide-details
density="compact" color="success" class="ml-auto"
@update:model-value="onTogglePresentation"
/>
</div>
<p class="fc-muted text-body-2 mb-3">
Auto-hide banners and editor screenshots from the gallery once a head has
learned them ( {{ minPositives }} examples) and clears
{{ Math.round((presentationThresholdInput || 0) * 100) }}% confidence.
<code>wip</code> is never auto-hidden. If a hidden image also looks like
real content ( {{ Math.round((presentationConflictInput || 0) * 100) }}%
on a content tag), it's flagged in the Hidden view instead of buried.
Every auto-hide is reversible.
</p>
<div class="d-flex mb-3" style="gap: 12px;">
<v-text-field
v-model.number="presentationThresholdInput" label="Hide confidence"
type="number" min="0.5" max="0.999" step="0.01" density="compact"
hide-details style="max-width: 200px;" :disabled="settingBusy"
@change="onSavePresentation"
/>
<v-text-field
v-model.number="presentationConflictInput" label="Flag if content ≥"
type="number" min="0" max="1" step="0.05" density="compact"
hide-details style="max-width: 200px;" :disabled="settingBusy"
@change="onSavePresentation"
/>
</div>
</div>
<!-- Performance / tuning --> <!-- Performance / tuning -->
<div v-if="metricsConcepts.length" class="mt-5"> <div v-if="metricsConcepts.length" class="mt-5">
<div class="fc-section-h mb-1">How auto-apply is landing</div> <div class="fc-section-h mb-1">How auto-apply is landing</div>
@@ -218,6 +254,11 @@ const autoStatus = ref(null)
const metricsData = ref(null) const metricsData = ref(null)
let autoTimer = null let autoTimer = null
// --- Presentation chrome auto-hide state (#141) ---
const presentationEnabled = ref(true)
const presentationThresholdInput = ref(0.90)
const presentationConflictInput = ref(0.50)
const autoRunning = computed(() => autoStatus.value?.running_id != null) const autoRunning = computed(() => autoStatus.value?.running_id != null)
const lastSweep = computed(() => const lastSweep = computed(() =>
(autoStatus.value?.runs || []).find(r => r.status !== 'running') || null) (autoStatus.value?.runs || []).find(r => r.status !== 'running') || null)
@@ -248,6 +289,9 @@ onMounted(async () => {
autoEnabled.value = !!s.head_auto_apply_enabled autoEnabled.value = !!s.head_auto_apply_enabled
autoPrecisionInput.value = s.head_auto_apply_precision ?? 0.97 autoPrecisionInput.value = s.head_auto_apply_precision ?? 0.97
autoMinPosInput.value = s.head_auto_apply_min_positives ?? 30 autoMinPosInput.value = s.head_auto_apply_min_positives ?? 30
presentationEnabled.value = s.presentation_auto_apply_enabled ?? true
presentationThresholdInput.value = s.presentation_auto_apply_threshold ?? 0.90
presentationConflictInput.value = s.presentation_conflict_threshold ?? 0.50
} catch { /* non-fatal */ } } catch { /* non-fatal */ }
await refresh() await refresh()
if (running.value) startPoll() if (running.value) startPoll()
@@ -332,6 +376,32 @@ async function onSaveSettings() {
settingBusy.value = false settingBusy.value = false
} }
} }
async function onTogglePresentation(val) {
settingBusy.value = true
try {
await mlSettings.patchSettings({ presentation_auto_apply_enabled: !!val })
toast({ text: val ? 'Chrome auto-hide on' : 'Chrome auto-hide off', type: 'success' })
} catch (e) {
presentationEnabled.value = !val // revert the switch
toast({ text: `Could not update: ${e.message}`, type: 'error' })
} finally {
settingBusy.value = false
}
}
async function onSavePresentation() {
settingBusy.value = true
try {
await mlSettings.patchSettings({
presentation_auto_apply_threshold: Number(presentationThresholdInput.value),
presentation_conflict_threshold: Number(presentationConflictInput.value),
})
} catch (e) {
toast({ text: `Could not save: ${e.message}`, type: 'error' })
} finally {
settingBusy.value = false
}
}
function onPreview() { startSweep(true) } function onPreview() { startSweep(true) }
function onApplyNow() { startSweep(false) } function onApplyNow() { startSweep(false) }
async function startSweep(dryRun) { async function startSweep(dryRun) {
@@ -13,6 +13,7 @@
</p> </p>
<div class="fc-tile-stack"> <div class="fc-tile-stack">
<ImportFiltersForm /> <ImportFiltersForm />
<TranslationCard />
</div> </div>
</section> </section>
@@ -69,6 +70,7 @@
import { onMounted, onUnmounted } from 'vue' import { onMounted, onUnmounted } from 'vue'
import ImportFiltersForm from './ImportFiltersForm.vue' import ImportFiltersForm from './ImportFiltersForm.vue'
import TranslationCard from './TranslationCard.vue'
import MLBackfillCard from './MLBackfillCard.vue' import MLBackfillCard from './MLBackfillCard.vue'
import ThumbnailBackfillCard from './ThumbnailBackfillCard.vue' import ThumbnailBackfillCard from './ThumbnailBackfillCard.vue'
import ArchiveReextractCard from './ArchiveReextractCard.vue' import ArchiveReextractCard from './ArchiveReextractCard.vue'
@@ -0,0 +1,337 @@
<template>
<MaintenanceTile
icon="mdi-translate"
title="Post translation (Interpreter)"
blurb="Translate non-English post titles + descriptions to English via your self-hosted Interpreter service, so archived posts read naturally."
>
<div class="d-flex align-center mb-3" style="gap: 10px;">
<span class="fc-section-h">Enabled</span>
<v-switch
v-model="enabled" :loading="busy" hide-details density="compact"
color="success" class="ml-auto" @update:model-value="onToggle"
/>
</div>
<p class="fc-muted text-body-2 mb-3">
Point this at your Interpreter service. It stays off until a URL is set, and
only translates posts whose text isn't already in the target language — the
English is shown by default on post cards, with a toggle back to the original.
</p>
<v-text-field
v-model="baseUrl" label="Interpreter base URL"
placeholder="http://interpreter.your-domain/" density="compact"
hide-details class="mb-3" :disabled="busy" @change="onSave"
/>
<v-text-field
v-model="targetLang" label="Target language" density="compact"
hide-details style="max-width: 160px;" class="mb-3" :disabled="busy"
@change="onSave"
/>
<!-- Acceptance floor (#155): latin-script translations below this Interpreter
confidence are kept as the original. Per-post overrides handle exceptions. -->
<v-text-field
v-model.number="minConfidence" label="Acceptance confidence"
type="number" min="0" max="1" step="0.01" density="compact"
hide-details style="max-width: 200px;" class="mb-1" :disabled="busy"
@change="onSaveConfidence"
/>
<div class="fc-muted text-caption mb-3">
Latin-script titles Interpreter is less than this sure about (01) are kept
as the original; CJK is always translated. Default 0.90 — raise it to reject
more mis-detections, lower it to translate more. Per-post controls in the
posts feed override this either way.
</div>
<div class="d-flex align-center flex-wrap mb-3" style="gap: 8px;">
<v-icon size="12" :color="statusColor">mdi-circle</v-icon>
<span class="fc-muted text-body-2">{{ statusText }}</span>
<v-btn
size="x-small" variant="tonal" rounded="pill" class="ml-2"
:loading="testing" :disabled="!baseUrl" @click="onTest"
>Test connection</v-btn>
<span
v-if="testResult !== null" class="text-caption"
:class="testResult ? 'text-success' : 'text-error'"
>{{ testResult ? ' reachable' : ' unreachable' }}</span>
</div>
<div class="d-flex align-center flex-wrap" style="gap: 10px;">
<v-btn
size="small" variant="flat" color="accent" rounded="pill"
prepend-icon="mdi-translate" :loading="running"
:disabled="!enabled || !baseUrl" @click="onRun"
>Translate now</v-btn>
<v-btn
size="small" variant="tonal" rounded="pill"
prepend-icon="mdi-refresh" :loading="retranslating"
:disabled="!enabled || !baseUrl" @click="confirmAll = true"
>Re-translate all</v-btn>
<span
v-if="status && status.active"
class="fc-muted text-caption d-inline-flex align-center" style="gap: 6px;"
>
<v-progress-circular indeterminate size="12" width="2" color="accent" />
Translating… {{ status.untranslated_count }} remaining
</span>
<span v-else-if="status" class="fc-muted text-caption">
{{ status.untranslated_count }}
post{{ status.untranslated_count === 1 ? '' : 's' }} awaiting translation
</span>
</div>
<p
v-if="status && status.last_run && ['error', 'timeout'].includes(status.last_run.status)"
class="text-caption text-error mt-1 mb-0"
>
Last translation run ended with “{{ status.last_run.status }}”. It resumes on
the next sweep; check the Interpreter connection if it persists.
</p>
<p class="fc-muted text-caption mt-2 mb-0">
Use <strong>Re-translate all</strong> after switching the Interpreter model —
it clears every stored translation and re-runs it through the new model
(unchanged text is served from cache, so it's cheap). To refresh just one
artist, use the Re-translate button on that artist's page.
</p>
<v-divider class="my-3" />
<!-- Diagnostic probe (task #1376): paste text → what Interpreter DETECTS
(language + confidence) and returns, without saving. Used to see why a
short/abbreviation-heavy English title gets mis-detected, and to pick a
detection-confidence / min-length guard from real numbers. -->
<div class="mb-1">
<span class="fc-section-h">Test translation</span>
<p class="fc-muted text-caption mt-1 mb-2">
Paste a title or snippet to see what Interpreter detects and returns —
handy for checking why a short string is mis-detected. Nothing is saved.
</p>
</div>
<v-textarea
v-model="probeText" label="Text to test" rows="2" auto-grow
density="compact" hide-details class="mb-2" :disabled="!baseUrl"
/>
<div class="d-flex align-center flex-wrap mb-2" style="gap: 8px;">
<v-btn
size="small" variant="tonal" rounded="pill" prepend-icon="mdi-magnify"
:loading="probing" :disabled="!baseUrl || !probeText.trim()"
@click="onProbe"
>Test translation</v-btn>
</div>
<v-sheet
v-if="probeResult" rounded class="pa-3 mb-1 text-body-2"
color="rgba(var(--v-theme-accent), 0.06)"
>
<div class="d-flex flex-wrap" style="gap: 4px 16px;">
<span>
<span class="fc-muted">Detected:</span>
<strong>{{ probeResult.detected_lang || '' }}</strong>
</span>
<span v-if="probeResult.detected_confidence != null">
<span class="fc-muted">Confidence:</span>
<strong>{{ fmtConf(probeResult.detected_confidence) }}</strong>
</span>
<span>
<span class="fc-muted">Engine:</span>
{{ probeResult.engine || '' }}<template
v-if="probeResult.engine_version"
> ({{ probeResult.engine_version }})</template>
</span>
<span><span class="fc-muted">Target:</span> {{ probeResult.target }}</span>
</div>
<v-divider class="my-2" />
<div class="fc-muted mb-1">
Result<template v-if="probeResult.detected_lang === probeResult.target">
— already {{ probeResult.target }}, so the sweep leaves it as-is</template>:
</div>
<p class="mb-0" style="white-space: pre-wrap;">{{
probeResult.translated || '(no change)'
}}</p>
</v-sheet>
<v-alert
v-if="err" type="error" variant="tonal" density="compact"
class="mt-3" closable @click:close="err = null"
>{{ err }}</v-alert>
<!-- m146: re-translate-everything is a bigger hammer than 'Translate now'
(it clears + rebuilds all translations), so it asks first. -->
<v-dialog v-model="confirmAll" max-width="440">
<v-card>
<v-card-title>Re-translate everything?</v-card-title>
<v-card-text class="text-body-2">
This clears the stored translation for <strong>every</strong> post and
re-runs them through your current Interpreter model. Text that hasn't
changed comes back from the cache instantly; anything the new model
translates differently is refreshed. It runs in the background until
done.
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn variant="text" @click="confirmAll = false">Cancel</v-btn>
<v-btn
color="accent" :loading="retranslating" @click="onRetranslateAll"
>Re-translate all</v-btn>
</v-card-actions>
</v-card>
</v-dialog>
</MaintenanceTile>
</template>
<script setup>
import { computed, onMounted, onUnmounted, ref } from 'vue'
import { toast } from '../../utils/toast.js'
import { useApi } from '../../composables/useApi.js'
import { useImportStore } from '../../stores/import.js'
import MaintenanceTile from '../common/MaintenanceTile.vue'
const api = useApi()
const store = useImportStore()
const enabled = ref(false)
const baseUrl = ref('')
const targetLang = ref('en')
const minConfidence = ref(0.9)
const busy = ref(false)
const running = ref(false)
const retranslating = ref(false)
const confirmAll = ref(false)
const testing = ref(false)
const testResult = ref(null) // null = not tested this session; true/false = last result
const status = ref(null)
const err = ref(null)
// #1376 "Test translation" probe: paste text → detected lang/confidence + result.
const probeText = ref('')
const probing = ref(false)
const probeResult = ref(null)
// Confidence scale is Interpreter-defined (0100 or 01) — show it as-is, just
// trimmed to 2 decimals, so the operator reads the true number.
function fmtConf (c) {
return c == null ? '' : Math.round(c * 100) / 100
}
const statusColor = computed(() => {
if (!enabled.value || !baseUrl.value) return 'grey'
return status.value?.healthy ? 'success' : 'error'
})
const statusText = computed(() => {
if (!enabled.value) return 'Off'
if (!baseUrl.value) return 'No base URL set'
if (status.value == null) return 'Checking…'
return status.value.healthy ? 'Interpreter reachable' : 'Interpreter unreachable'
})
let pollTimer = null
async function loadStatus() {
try { status.value = await api.get('/api/settings/translation/status') }
catch { status.value = null }
// Poll live while a sweep is running so the remaining count ticks down; stop
// when idle (a fresh trigger restarts it via its own setTimeout(loadStatus)).
clearTimeout(pollTimer)
if (status.value?.active) pollTimer = setTimeout(loadStatus, 3000)
}
onUnmounted(() => clearTimeout(pollTimer))
async function onTest() {
// Ping /v1/health for the CURRENTLY-typed URL (not the saved one), so a new
// URL can be verified before enabling. Also reflects in the status dot.
testing.value = true
err.value = null
testResult.value = null
try {
const r = await api.post('/api/settings/translation/test', {
body: { base_url: baseUrl.value.trim() },
})
testResult.value = !!r.healthy
if (status.value) status.value.healthy = !!r.healthy
} catch (e) {
err.value = e.message
} finally {
testing.value = false
}
}
async function onProbe () {
probing.value = true
err.value = null
probeResult.value = null
try {
probeResult.value = await api.post('/api/settings/translation/probe', {
body: { text: probeText.value.trim() },
})
} catch (e) {
err.value = e.message
} finally {
probing.value = false
}
}
onMounted(async () => {
try {
await store.loadSettings()
const s = store.settings || {}
enabled.value = !!s.translation_enabled
baseUrl.value = s.interpreter_base_url || ''
targetLang.value = s.translation_target_lang || 'en'
minConfidence.value = s.translation_min_confidence ?? 0.9
} catch { /* non-fatal */ }
await loadStatus()
})
async function save(patch) {
busy.value = true
err.value = null
try { await store.patchSettings(patch) }
catch (e) { err.value = e.message }
finally { busy.value = false }
}
async function onToggle(val) {
await save({ translation_enabled: !!val })
toast({ text: val ? 'Translation on' : 'Translation off', type: 'success' })
await loadStatus()
}
async function onSave() {
await save({
interpreter_base_url: baseUrl.value.trim(),
translation_target_lang: (targetLang.value || 'en').trim() || 'en',
})
await loadStatus()
}
async function onSaveConfidence() {
// Clamp to [0, 1] so a stray value never bounces off the API 400.
let v = Number(minConfidence.value)
if (!Number.isFinite(v)) v = 0.9
v = Math.min(1, Math.max(0, v))
minConfidence.value = v
await save({ translation_min_confidence: v })
}
async function onRun() {
running.value = true
err.value = null
try {
await api.post('/api/settings/translation/run')
toast({ text: 'Translation sweep started', type: 'success' })
} catch (e) {
err.value = e.message
} finally {
running.value = false
setTimeout(loadStatus, 1500) // let the sweep make a dent, then refresh
}
}
async function onRetranslateAll() {
retranslating.value = true
err.value = null
try {
await api.post('/api/settings/translation/retranslate', { body: { all: true } })
confirmAll.value = false
toast({ text: 'Re-translating all posts…', type: 'success' })
} catch (e) {
err.value = e.message
} finally {
retranslating.value = false
setTimeout(loadStatus, 1500) // reset spikes the untranslated count — refresh it
}
}
</script>
+3 -1
View File
@@ -44,7 +44,9 @@ export const useExploreStore = defineStore('explore', () => {
// empty and let the view explain why (anchor.has_embedding === false). // empty and let the view explain why (anchor.has_embedding === false).
if (detail.has_embedding) { if (detail.has_embedding) {
const body = await api.get('/api/gallery/similar', { const body = await api.get('/api/gallery/similar', {
params: { similar_to: numId, limit: NEIGHBOR_LIMIT }, // exclude_wip: keep work-in-progress out of the Explore rabbit-hole
// (the gallery's own "similar" button still shows it) — operator 2026-07-08.
params: { similar_to: numId, limit: NEIGHBOR_LIMIT, exclude_wip: 1 },
}) })
if (!t.isCurrent()) return if (!t.isCurrent()) return
neighbors.value = body.images || [] neighbors.value = body.images || []
+8 -1
View File
@@ -30,6 +30,9 @@ export const useGalleryStore = defineStore('gallery', () => {
tag_or: [], tag_exclude: [], tag_or: [], tag_exclude: [],
// Phase-2 faceted refine params. // Phase-2 faceted refine params.
platform: null, untagged: false, no_artist: false, platform: null, untagged: false, no_artist: false,
// Reveal the presentation chrome (banner / editor screenshot) the gallery
// hides by default (milestone 141).
include_hidden: false,
date_from: null, date_to: null, date_from: null, date_to: null,
// Phase-3 visual similarity: when set, the gallery is in "similar mode" — // Phase-3 visual similarity: when set, the gallery is in "similar mode" —
// ranked by cosine distance to this image, bounded top-N, no cursor. // ranked by cosine distance to this image, bounded top-N, no cursor.
@@ -158,6 +161,7 @@ export const useGalleryStore = defineStore('gallery', () => {
if (filter.value.platform) p.platform = filter.value.platform if (filter.value.platform) p.platform = filter.value.platform
if (filter.value.untagged) p.untagged = '1' if (filter.value.untagged) p.untagged = '1'
if (filter.value.no_artist) p.no_artist = '1' if (filter.value.no_artist) p.no_artist = '1'
if (filter.value.include_hidden) p.include_hidden = '1'
if (filter.value.date_from) p.date_from = filter.value.date_from if (filter.value.date_from) p.date_from = filter.value.date_from
if (filter.value.date_to) p.date_to = filter.value.date_to if (filter.value.date_to) p.date_to = filter.value.date_to
return p return p
@@ -196,6 +200,7 @@ export const useGalleryStore = defineStore('gallery', () => {
platform: q.platform || null, platform: q.platform || null,
untagged: _truthy(q.untagged), untagged: _truthy(q.untagged),
no_artist: _truthy(q.no_artist), no_artist: _truthy(q.no_artist),
include_hidden: _truthy(q.include_hidden),
date_from: _parseDate(q.date_from), date_from: _parseDate(q.date_from),
date_to: _parseDate(q.date_to), date_to: _parseDate(q.date_to),
similar_to: _toId(q.similar_to), similar_to: _toId(q.similar_to),
@@ -284,7 +289,8 @@ export function cloneFilter(f) {
tag_exclude: [...(f.tag_exclude || [])], tag_exclude: [...(f.tag_exclude || [])],
artist_id: f.artist_id, media_type: f.media_type, artist_id: f.artist_id, media_type: f.media_type,
sort: f.sort, platform: f.platform, untagged: f.untagged, sort: f.sort, platform: f.platform, untagged: f.untagged,
no_artist: f.no_artist, date_from: f.date_from, date_to: f.date_to, no_artist: f.no_artist, include_hidden: f.include_hidden,
date_from: f.date_from, date_to: f.date_to,
similar_to: f.similar_to, similar_to: f.similar_to,
} }
} }
@@ -301,6 +307,7 @@ export function filterToQuery(f) {
if (f.platform) q.platform = f.platform if (f.platform) q.platform = f.platform
if (f.untagged) q.untagged = '1' if (f.untagged) q.untagged = '1'
if (f.no_artist) q.no_artist = '1' if (f.no_artist) q.no_artist = '1'
if (f.include_hidden) q.include_hidden = '1'
if (f.date_from) q.date_from = f.date_from if (f.date_from) q.date_from = f.date_from
if (f.date_to) q.date_to = f.date_to if (f.date_to) q.date_to = f.date_to
if (f.similar_to) q.similar_to = String(f.similar_to) if (f.similar_to) q.similar_to = String(f.similar_to)
+32 -31
View File
@@ -17,12 +17,13 @@ export const useModalStore = defineStore('modal', () => {
// chip rail. Audit 2026-06-02. // chip rail. Audit 2026-06-02.
const inflight = useInflightToken() const inflight = useInflightToken()
// Post-scoped cycle. When set, prev/next cycles within this array // Scoped playlist. When set, prev/next cycles within THIS ordered id array
// (used by PostCard image clicks — the modal is scoped to that post's // the current gallery filter (GalleryView) or a post's images (PostCard) — so
// images). When null, prev/next falls back to current.value.neighbors // the modal walks exactly what the user was looking at, not a global order.
// (the gallery-store-driven /api/gallery/image/<id> neighbors). // When null, prev/next falls back to current.value.neighbors (the
const postImageIds = ref(null) // /api/gallery/image/<id> global neighbours).
const postImageIndex = ref(0) const playlistIds = ref(null)
const playlistIndex = ref(0)
async function open (id, opts = {}) { async function open (id, opts = {}) {
// Cancel any in-flight tag mutation or reloadTags from the // Cancel any in-flight tag mutation or reloadTags from the
@@ -32,13 +33,13 @@ export const useModalStore = defineStore('modal', () => {
current.value = null // cleared upfront so it stays null on error current.value = null // cleared upfront so it stays null on error
// Update post-scoped state if caller passed it; otherwise clear so // Update post-scoped state if caller passed it; otherwise clear so
// the next open() from gallery context uses neighbors mode. // the next open() from gallery context uses neighbors mode.
if (opts.postImageIds != null) { if (opts.playlistIds != null) {
postImageIds.value = opts.postImageIds playlistIds.value = opts.playlistIds
postImageIndex.value = opts.postImageIds.indexOf(id) playlistIndex.value = opts.playlistIds.indexOf(id)
if (postImageIndex.value < 0) postImageIndex.value = 0 if (playlistIndex.value < 0) playlistIndex.value = 0
} else if (opts.clearPostScope !== false && postImageIds.value != null) { } else if (opts.clearPlaylist !== false && playlistIds.value != null) {
postImageIds.value = null playlistIds.value = null
postImageIndex.value = 0 playlistIndex.value = 0
} }
const t = inflight.claim() const t = inflight.claim()
await run(async () => { await run(async () => {
@@ -53,17 +54,17 @@ export const useModalStore = defineStore('modal', () => {
currentImageId.value = null currentImageId.value = null
current.value = null current.value = null
error.value = null error.value = null
postImageIds.value = null playlistIds.value = null
postImageIndex.value = 0 playlistIndex.value = 0
} }
async function goPrev () { async function goPrev () {
if (postImageIds.value != null) { if (playlistIds.value != null) {
if (postImageIndex.value > 0) { if (playlistIndex.value > 0) {
const newIdx = postImageIndex.value - 1 const newIdx = playlistIndex.value - 1
const newId = postImageIds.value[newIdx] const newId = playlistIds.value[newIdx]
postImageIndex.value = newIdx playlistIndex.value = newIdx
await open(newId, { postImageIds: postImageIds.value }) await open(newId, { playlistIds: playlistIds.value })
} }
return return
} }
@@ -73,12 +74,12 @@ export const useModalStore = defineStore('modal', () => {
} }
async function goNext () { async function goNext () {
if (postImageIds.value != null) { if (playlistIds.value != null) {
if (postImageIndex.value < postImageIds.value.length - 1) { if (playlistIndex.value < playlistIds.value.length - 1) {
const newIdx = postImageIndex.value + 1 const newIdx = playlistIndex.value + 1
const newId = postImageIds.value[newIdx] const newId = playlistIds.value[newIdx]
postImageIndex.value = newIdx playlistIndex.value = newIdx
await open(newId, { postImageIds: postImageIds.value }) await open(newId, { playlistIds: playlistIds.value })
} }
return return
} }
@@ -177,19 +178,19 @@ export const useModalStore = defineStore('modal', () => {
const isOpen = computed(() => currentImageId.value !== null) const isOpen = computed(() => currentImageId.value !== null)
const canPrev = computed(() => { const canPrev = computed(() => {
if (postImageIds.value != null) return postImageIndex.value > 0 if (playlistIds.value != null) return playlistIndex.value > 0
return current.value?.neighbors?.prev_id != null return current.value?.neighbors?.prev_id != null
}) })
const canNext = computed(() => { const canNext = computed(() => {
if (postImageIds.value != null) { if (playlistIds.value != null) {
return postImageIndex.value < (postImageIds.value.length - 1) return playlistIndex.value < (playlistIds.value.length - 1)
} }
return current.value?.neighbors?.next_id != null return current.value?.neighbors?.next_id != null
}) })
return { return {
currentImageId, current, loading, error, currentImageId, current, loading, error,
postImageIds, postImageIndex, playlistIds, playlistIndex,
isOpen, canPrev, canNext, isOpen, canPrev, canNext,
open, close, goPrev, goNext, open, close, goPrev, goNext,
reloadTags, removeTag, addExistingTag, createAndAdd, reloadTags, removeTag, addExistingTag, createAndAdd,
+19 -1
View File
@@ -71,6 +71,24 @@ export const usePostsStore = defineStore('posts', () => {
return await api.get(`/api/posts/${id}`) return await api.get(`/api/posts/${id}`)
} }
// Set a post's sticky translation override (auto/force/original, #155) and
// patch the loaded item in place with the endpoint's result (it translates /
// clears immediately when the service is up), so the card reflects the change
// without a feed reload.
async function applyTranslationOverride(postId, override) {
const res = await api.post(`/api/posts/${postId}/translation-override`, {
body: { override },
})
const item = items.value.find((p) => p.id === postId)
if (item) {
item.translation_override = res.translation_override
item.post_title_translated = res.post_title_translated
item.description_translated = res.description_translated
item.translated_source_lang = res.translated_source_lang
}
return res
}
// Filter overlay for the around/older/newer (in-context anchored) // Filter overlay for the around/older/newer (in-context anchored)
// path. Keep this distinct from `filters.value` (the down-only feed) // path. Keep this distinct from `filters.value` (the down-only feed)
// so a normal-feed filter change doesn't leak into an active anchored // so a normal-feed filter change doesn't leak into an active anchored
@@ -168,7 +186,7 @@ export const usePostsStore = defineStore('posts', () => {
return { return {
items, cursor, loading, done, error, filters, items, cursor, loading, done, error, filters,
cursorOlder, cursorNewer, doneOlder, doneNewer, anchorId, cursorOlder, cursorNewer, doneOlder, doneNewer, anchorId,
loadInitial, loadMore, getPostFull, loadInitial, loadMore, getPostFull, applyTranslationOverride,
loadAround, loadOlder, loadNewer, loadAround, loadOlder, loadNewer,
} }
}) })
+85 -141
View File
@@ -1,6 +1,6 @@
import { defineStore } from 'pinia' import { defineStore } from 'pinia'
import { toast } from '../utils/toast.js' import { toast } from '../utils/toast.js'
import { ref } from 'vue' import { computed, ref } from 'vue'
import { useApi } from '../composables/useApi.js' import { useApi } from '../composables/useApi.js'
import { useAsyncAction } from '../composables/useAsyncAction.js' import { useAsyncAction } from '../composables/useAsyncAction.js'
import { useInflightToken } from '../composables/useInflightToken.js' import { useInflightToken } from '../composables/useInflightToken.js'
@@ -18,182 +18,112 @@ export const CATEGORY_LABELS = {
export const useSuggestionsStore = defineStore('suggestions', () => { export const useSuggestionsStore = defineStore('suggestions', () => {
const api = useApi() const api = useApi()
const byCategory = ref({}) // { category: [suggestion, ...] } — panel (≥ threshold) // ONE source of truth per image: every trained head scored for this image
// The typed tag-input dropdown searches the model's FULL prediction set for // (min=0), each row carrying above_threshold. The Suggestions PANEL renders
// the image (min=0, down to the store floor) so low-confidence actions/ // aboveByCategory; the typed tag-input dropdown reads the full set and
// features can be picked in canonical formatting (operator-asked 2026-06-09). // annotates matching DB-tag rows with their score. A single fetch backs both —
const allByCategory = ref({}) // every suggestion is a canonical tag now (tagging-v2, #114), so there is no
// raw-prediction / create-new / alias-remap path to reconcile.
const byCategory = ref({})
const { loading, error, run } = useAsyncAction({ errorAs: 'message' }) const { loading, error, run } = useAsyncAction({ errorAs: 'message' })
let currentImageId = null let currentImageId = null
const inflightAll = useInflightToken() // Audit 2026-06-02: without an inflight guard a late /suggestions response from
// Audit 2026-06-02: this store had no inflight guard — a late // a prior image could overwrite byCategory while currentImageId points at a new
// /suggestions response from a prior image could overwrite // one, and accept() could apply image A's tag to image B. Both guarded below by
// byCategory while currentImageId pointed at a new one, and // capturing imageId at call-time and gating writes on the token.
// accept() dereferenced currentImageId AFTER an awaited POST so
// the subsequent /suggestions/accept could apply A's chosen tag
// to image B (and push it to the allowlist). Both fixed below
// by capturing imageId at call-time and gating writes on the token.
const inflight = useInflightToken() const inflight = useInflightToken()
// The curated above-threshold subset the Suggestions panel shows. A rejected
// row that still scores above its cut stays (flagged) so the rejection is
// visible + reversible; below-threshold rows live only in the dropdown.
const aboveByCategory = computed(() => {
const out = {}
for (const [cat, list] of Object.entries(byCategory.value)) {
const kept = (list || []).filter(s => s.above_threshold)
if (kept.length) out[cat] = kept
}
return out
})
async function load(imageId) { async function load(imageId) {
// Cancel any in-flight load from the previous image so its late // Cancel any in-flight load from the previous image so its late response
// response can't overwrite this image's byCategory. // can't overwrite this image's list.
inflight.cancel() inflight.cancel()
currentImageId = imageId currentImageId = imageId
byCategory.value = {} // cleared upfront so it stays empty on error byCategory.value = {} // cleared upfront so it stays empty on error
const t = inflight.claim() const t = inflight.claim()
await run(async () => { await run(async () => {
const body = await api.get(`/api/images/${imageId}/suggestions`) // min=0 → every head, each flagged above_threshold; the panel + the typed
// dropdown are both derived from this one payload.
const body = await api.get(`/api/images/${imageId}/suggestions`, {
params: { min: 0 },
})
if (!t.isCurrent()) return if (!t.isCurrent()) return
byCategory.value = body.by_category || {} byCategory.value = body.by_category || {}
}) })
} }
// Load the full prediction set for the dropdown (separate inflight guard so a // A stable identity for a suggestion across the panel + dropdown surfaces:
// late response from a prior image can't overwrite the current one's list). // its canonical tag id (every suggestion has one now).
async function loadAll(imageId) {
inflightAll.cancel()
allByCategory.value = {}
const t = inflightAll.claim()
try {
const body = await api.get(`/api/images/${imageId}/suggestions`, {
params: { min: 0 },
})
if (!t.isCurrent()) return
allByCategory.value = body.by_category || {}
} catch {
// Dropdown is best-effort — the panel surfaces load errors.
}
}
// A stable identity for a suggestion across the two lists (panel vs dropdown
// are separate fetches, so object identity differs): tag id when known, else
// the raw display key.
function _keyOf(s) { function _keyOf(s) {
return s.canonical_tag_id != null return `id:${s.canonical_tag_id}`
? `id:${s.canonical_tag_id}`
: `raw:${s.category}:${(s.display_name || '').toLowerCase()}`
} }
// Drop an accepted/dismissed suggestion from BOTH the panel list and the // Drop an accepted suggestion from the list so it can't reappear.
// dropdown's full list so it can't reappear in either surface.
function _dropEverywhere(suggestion) { function _dropEverywhere(suggestion) {
const key = _keyOf(suggestion) const key = _keyOf(suggestion)
const cat = suggestion.category const list = byCategory.value[suggestion.category]
for (const map of [byCategory.value, allByCategory.value]) { if (list) byCategory.value[suggestion.category] = list.filter(s => _keyOf(s) !== key)
const list = map[cat]
if (list) map[cat] = list.filter(s => _keyOf(s) !== key)
}
} }
// Flip the `rejected` flag on the matching suggestion in BOTH lists in place, // Flip the `rejected` flag on the matching suggestion in place, so a
// so a reject/un-reject shows immediately without dropping the row (visible, // reject/un-reject shows immediately without dropping the row (visible,
// reversible rejection — misclick recovery, operator-asked 2026-06-27). // reversible rejection — misclick recovery, operator-asked 2026-06-27).
function _setRejectedEverywhere(suggestion, value) { function _setRejectedEverywhere(suggestion, value) {
const key = _keyOf(suggestion) const key = _keyOf(suggestion)
const cat = suggestion.category for (const s of byCategory.value[suggestion.category] || []) {
for (const map of [byCategory.value, allByCategory.value]) { if (_keyOf(s) === key) s.rejected = value
for (const s of map[cat] || []) {
if (_keyOf(s) === key) s.rejected = value
}
} }
} }
// opts.tagId: the tag's known id, when the caller already created/resolved // opts.tagId: the tag's known id when the caller already resolved it (TagPanel's
// it (TagPanel's manual-add-matches-suggestion path). Passed separately — // manual-add-matches-suggestion path). Passed separately — NOT spread onto the
// NOT spread onto the suggestion — because _dropEverywhere keys off the // suggestion — because _dropEverywhere keys off the suggestion object.
// suggestion object, and mutating canonical_tag_id on a raw suggestion
// would stop it matching its own row in the lists.
async function accept(suggestion, { tagId: knownTagId = null } = {}) { async function accept(suggestion, { tagId: knownTagId = null } = {}) {
// Capture imageId so a mid-flight prev/next can't reroute the // Capture imageId so a mid-flight prev/next can't reroute the accept POST to
// accept POST to a different image AND push the tag to that // a different image.
// image's allowlist.
const imageId = currentImageId const imageId = currentImageId
if (imageId == null) return if (imageId == null) return
// Raw tags (creates_new_tag) have no canonical_tag_id; the backend's const tagId = knownTagId ?? suggestion.canonical_tag_id
// accept endpoint needs a tag_id, so for raw tags we create the tag
// first via the existing /api/tags endpoint, then accept by id.
let tagId = knownTagId ?? suggestion.canonical_tag_id
if (tagId == null) {
const created = await api.post('/api/tags', {
body: { name: suggestion.display_name, kind: suggestion.category }
})
tagId = created.id
}
await api.post(`/api/images/${imageId}/suggestions/accept`, { await api.post(`/api/images/${imageId}/suggestions/accept`, {
body: { tag_id: tagId } body: { tag_id: tagId }
}) })
// Only drop from THIS image's category list — if the user navigated, // Only drop from THIS image's list — if the user navigated, the new image
// the new image has its own suggestions and this drop would corrupt them. // has its own suggestions and this drop would corrupt them.
if (currentImageId === imageId) { if (currentImageId === imageId) {
_dropEverywhere(suggestion) _dropEverywhere(suggestion)
} }
_acceptToast('Tagged', suggestion.display_name) _acceptToast('Tagged', suggestion.display_name)
} }
// One non-blocking toast for accept/alias. The accepted tag is applied to this // One non-blocking toast for accept. The accepted tag is applied to this image
// image and feeds head training; head auto-apply handles propagation (earned), // and feeds head training; head auto-apply handles propagation (earned).
// so there's no instant fan-out to project.
function _acceptToast(verb, displayName) { function _acceptToast(verb, displayName) {
toast({ text: `${verb}: ${displayName}`, type: 'success' }) toast({ text: `${verb}: ${displayName}`, type: 'success' })
} }
async function aliasAccept(suggestion, canonicalTagId) { // Find a live (non-rejected) suggestion matching a manually-picked tag — by
const imageId = currentImageId // canonical id when known, else by (kind, name). Manually adding a tag the
if (imageId == null) return // model also suggested must be indistinguishable from accepting the suggestion
// The alias MUST be stored under the raw model key — resolution looks up the // (recorded + dropped from the panel), so TagPanel routes matches through
// raw prediction key, not the normalized display name. Sending display_name // accept() (operator-asked 2026-07-03).
// (the old bug) stored an alias that never resolved, so the prediction kept
// reappearing unaliased. raw_name is null only for centroid hits, which
// can't be aliased (the UI hides the action for them).
const aliasString = suggestion.raw_name ?? suggestion.display_name
await api.post(`/api/images/${imageId}/suggestions/alias`, {
body: {
alias_string: aliasString,
alias_category: suggestion.category,
canonical_tag_id: canonicalTagId
}
})
if (currentImageId === imageId) {
_dropEverywhere(suggestion)
}
_acceptToast('Aliased & tagged', suggestion.display_name)
}
// Remove the alias behind an aliased suggestion (the raw prediction reverts to
// its unaliased form on reload). The canonical tag stays applied if it was
// accepted — this only undoes the model-key→tag mapping.
async function removeAlias(suggestion) {
const imageId = currentImageId
if (imageId == null || suggestion.raw_name == null) return
await api.delete(
`/api/aliases/${encodeURIComponent(suggestion.raw_name)}/${encodeURIComponent(suggestion.category)}`
)
if (currentImageId === imageId) {
await load(imageId)
await loadAll(imageId)
}
toast({ text: `Alias removed: ${suggestion.display_name}`, type: 'success' })
}
// Find a live (non-rejected) pending suggestion matching a manually-picked
// tag — by canonical id when known, else by (kind, name). Manually adding a
// tag the model also suggested must be indistinguishable from accepting the
// suggestion (recorded + dropped from the panel), so TagPanel routes matches
// through accept() (operator-asked 2026-07-03). Searches the full dropdown
// set first, then the panel list (loadAll is best-effort and can be empty).
function findPending(kind, name, tagId = null) { function findPending(kind, name, tagId = null) {
const lname = (name || '').toLowerCase() const lname = (name || '').toLowerCase()
for (const map of [allByCategory.value, byCategory.value]) { for (const list of Object.values(byCategory.value)) {
for (const list of Object.values(map)) { for (const s of list || []) {
for (const s of list || []) { if (s.rejected) continue
if (s.rejected) continue if (tagId != null && s.canonical_tag_id === tagId) return s
if (tagId != null && s.canonical_tag_id === tagId) return s if (s.category === kind && (s.display_name || '').toLowerCase() === lname) return s
if (
s.category === kind &&
(s.display_name || '').toLowerCase() === lname
) return s
}
} }
} }
return null return null
@@ -202,14 +132,8 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
async function dismiss(suggestion) { async function dismiss(suggestion) {
const imageId = currentImageId const imageId = currentImageId
if (imageId == null) return if (imageId == null) return
// Dismiss needs a tag_id. Raw tags (creates_new_tag) have none, so there's
// nothing to persist a rejection against — drop them client-side as before.
// Canonical tags persist a rejection and STAY in the list flagged rejected, // Canonical tags persist a rejection and STAY in the list flagged rejected,
// so the operator can see it and one-click un-reject (misclick recovery). // so the operator can see it and one-click un-reject (misclick recovery).
if (suggestion.canonical_tag_id == null) {
if (currentImageId === imageId) _dropEverywhere(suggestion)
return
}
await api.post(`/api/images/${imageId}/suggestions/dismiss`, { await api.post(`/api/images/${imageId}/suggestions/dismiss`, {
body: { tag_id: suggestion.canonical_tag_id } body: { tag_id: suggestion.canonical_tag_id }
}) })
@@ -218,10 +142,31 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
} }
} }
// Reject every still-unhandled VISIBLE (above-threshold) suggestion in a
// category in one go ("confirm the good ones, reject the rest" — operator-asked
// 2026-07-06). Only touches what the panel shows, not the low-confidence tail
// the dropdown carries. Dispatched in parallel so a big section clears fast.
async function dismissRemaining(category) {
const imageId = currentImageId
if (imageId == null) return
const targets = (byCategory.value[category] || []).filter(
(s) => s.above_threshold && !s.rejected
)
if (!targets.length) return
await Promise.all(targets.map((s) =>
api.post(`/api/images/${imageId}/suggestions/dismiss`, {
body: { tag_id: s.canonical_tag_id },
})
))
if (currentImageId === imageId) {
targets.forEach((s) => _setRejectedEverywhere(s, true))
}
}
// Undo a per-image dismissal — the suggestion reverts to a live row. // Undo a per-image dismissal — the suggestion reverts to a live row.
async function undismiss(suggestion) { async function undismiss(suggestion) {
const imageId = currentImageId const imageId = currentImageId
if (imageId == null || suggestion.canonical_tag_id == null) return if (imageId == null) return
await api.post(`/api/images/${imageId}/suggestions/undismiss`, { await api.post(`/api/images/${imageId}/suggestions/undismiss`, {
body: { tag_id: suggestion.canonical_tag_id } body: { tag_id: suggestion.canonical_tag_id }
}) })
@@ -231,8 +176,7 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
} }
return { return {
byCategory, allByCategory, loading, error, byCategory, aboveByCategory, loading, error,
load, loadAll, accept, aliasAccept, removeAlias, dismiss, undismiss, load, accept, dismiss, dismissRemaining, undismiss, findPending
findPending
} }
}) })
+6 -1
View File
@@ -13,6 +13,7 @@
<div class="fc-gallery-layout__main"> <div class="fc-gallery-layout__main">
<PostInfoHeader /> <PostInfoHeader />
<GalleryFilterBar v-if="store.filter.post_id == null" /> <GalleryFilterBar v-if="store.filter.post_id == null" />
<HiddenReviewStrip v-if="store.filter.post_id == null" />
<EmptyState v-if="store.isEmpty" /> <EmptyState v-if="store.isEmpty" />
<GalleryGrid v-else @open="openImage" /> <GalleryGrid v-else @open="openImage" />
</div> </div>
@@ -33,6 +34,7 @@ import { useGalleryStore } from '../stores/gallery.js'
import { useModalStore } from '../stores/modal.js' import { useModalStore } from '../stores/modal.js'
import GalleryGrid from '../components/gallery/GalleryGrid.vue' import GalleryGrid from '../components/gallery/GalleryGrid.vue'
import GalleryFilterBar from '../components/gallery/GalleryFilterBar.vue' import GalleryFilterBar from '../components/gallery/GalleryFilterBar.vue'
import HiddenReviewStrip from '../components/gallery/HiddenReviewStrip.vue'
import TimelineSidebar from '../components/gallery/TimelineSidebar.vue' import TimelineSidebar from '../components/gallery/TimelineSidebar.vue'
import EmptyState from '../components/gallery/EmptyState.vue' import EmptyState from '../components/gallery/EmptyState.vue'
import PostInfoHeader from '../components/gallery/PostInfoHeader.vue' import PostInfoHeader from '../components/gallery/PostInfoHeader.vue'
@@ -54,7 +56,10 @@ watch(() => route.query, (q) => {
}) })
function openImage(id) { function openImage(id) {
modal.open(id) // Walk the current gallery filter in the modal (#1322) — prev/next moves
// through exactly the filtered set the operator is viewing, not global
// neighbours. Snapshot of the currently-loaded, filtered, ordered ids.
modal.open(id, { playlistIds: store.images.map((i) => i.id) })
} }
</script> </script>
+1 -1
View File
@@ -80,6 +80,6 @@ describe('PostCard', () => {
const w = mountComponent(PostCard, { props: { post }, pinia }) const w = mountComponent(PostCard, { props: { post }, pinia })
await w.find('.fc-post-card__hero').trigger('click') await w.find('.fc-post-card__hero').trigger('click')
await flushPromises() await flushPromises()
expect(openSpy).toHaveBeenCalledWith(10, { postImageIds: [10, 11] }) expect(openSpy).toHaveBeenCalledWith(10, { playlistIds: [10, 11] })
}) })
}) })
+36 -27
View File
@@ -16,25 +16,44 @@ function stubFetch(handler) {
}) })
} }
// Every suggestion is a canonical DB tag now (tagging-v2): a real id, flagged
// above/below its head's suggest threshold. No raw / creates-new / alias cases.
const sugg = (over = {}) => ({ const sugg = (over = {}) => ({
canonical_tag_id: 7, canonical_tag_id: 7,
display_name: 'Ichigo', display_name: 'Ichigo',
category: 'character', category: 'character',
score: 0.9, score: 0.9,
source: 'head', source: 'head',
creates_new_tag: false, above_threshold: true,
rejected: false, rejected: false,
...over, ...over,
}) })
// Seed the store through its own load/loadAll so currentImageId is set the // Seed the store through its single load() so currentImageId is set the way the
// way the panel sets it (accept() derives the image from it). // panel sets it (accept() derives the image from it).
async function seed(store, byCategory) { async function seed(store, byCategory) {
stubFetch(() => ({ status: 200, body: { by_category: byCategory } })) stubFetch(() => ({ status: 200, body: { by_category: byCategory } }))
await store.load(1) await store.load(1)
await store.loadAll(1)
} }
describe('suggestions store: load derives aboveByCategory', () => {
beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks())
it('one fetch backs both surfaces: byCategory is the full set, aboveByCategory the panel subset', async () => {
const s = useSuggestionsStore()
await seed(s, {
general: [
sugg({ canonical_tag_id: 1, display_name: 'sword', category: 'general' }),
sugg({ canonical_tag_id: 2, display_name: 'faint', category: 'general', above_threshold: false }),
],
})
expect(s.byCategory.general).toHaveLength(2) // dropdown sees all
expect(s.aboveByCategory.general).toHaveLength(1) // panel sees above-threshold only
expect(s.aboveByCategory.general[0].display_name).toBe('sword')
})
})
describe('suggestions store: findPending (manual add == accept, 2026-07-03)', () => { describe('suggestions store: findPending (manual add == accept, 2026-07-03)', () => {
beforeEach(() => setActivePinia(createPinia())) beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks()) afterEach(() => vi.restoreAllMocks())
@@ -45,15 +64,12 @@ describe('suggestions store: findPending (manual add == accept, 2026-07-03)', ()
expect(s.findPending('general', 'unrelated', 7)?.display_name).toBe('Ichigo') expect(s.findPending('general', 'unrelated', 7)?.display_name).toBe('Ichigo')
}) })
it('matches raw suggestions by kind + name, case-insensitively', async () => { it('matches by kind + name, case-insensitively', async () => {
const s = useSuggestionsStore() const s = useSuggestionsStore()
await seed(s, { await seed(s, {
general: [sugg({ general: [sugg({ canonical_tag_id: 12, display_name: 'Holding Sword', category: 'general' })],
canonical_tag_id: null, display_name: 'Holding Sword',
category: 'general', creates_new_tag: true,
})],
}) })
expect(s.findPending('general', 'holding sword')).toBeTruthy() expect(s.findPending('general', 'holding sword')?.canonical_tag_id).toBe(12)
// Kind must match: same name under another kind is a different tag. // Kind must match: same name under another kind is a different tag.
expect(s.findPending('character', 'holding sword')).toBeNull() expect(s.findPending('character', 'holding sword')).toBeNull()
}) })
@@ -66,37 +82,31 @@ describe('suggestions store: findPending (manual add == accept, 2026-07-03)', ()
}) })
}) })
describe('suggestions store: accept with a known tag id', () => { describe('suggestions store: accept', () => {
beforeEach(() => setActivePinia(createPinia())) beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks()) afterEach(() => vi.restoreAllMocks())
it('POSTs only the accept endpoint (no tag create) and drops the row', async () => { it('POSTs only the accept endpoint (canonical id) and drops the row', async () => {
const s = useSuggestionsStore() const s = useSuggestionsStore()
await seed(s, { character: [sugg()] }) await seed(s, { character: [sugg()] })
const calls = [] const calls = []
stubFetch((url, init) => { stubFetch((url, init) => {
calls.push({ url, method: init?.method }) calls.push({ url, method: init?.method, body: init?.body })
return { status: 200, body: { accepted: true, tag_id: 7 } } return { status: 200, body: { accepted: true, tag_id: 7 } }
}) })
await s.accept(s.byCategory.character[0], { tagId: 7 }) await s.accept(s.byCategory.character[0])
expect(calls).toHaveLength(1) expect(calls).toHaveLength(1)
expect(calls[0].url).toContain('/api/images/1/suggestions/accept') expect(calls[0].url).toContain('/api/images/1/suggestions/accept')
expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 7 })
expect(s.byCategory.character).toHaveLength(0) expect(s.byCategory.character).toHaveLength(0)
expect(s.allByCategory.character).toHaveLength(0)
}) })
it('a RAW suggestion accepted with a known tagId skips create AND still drops its row', async () => { it('honors a known tagId and still drops the row by the suggestion identity', async () => {
// TagPanel's manual-create path: the tag was just created by // TagPanel's manual-add-matches-suggestion path passes the resolved tag id;
// host.createAndAdd, so accept must not create again — and the drop must // accept sends exactly it and drops the original suggestion row (which keys
// key off the original raw suggestion object, not the resolved id // off the suggestion object, not the passed id).
// (regression: spreading canonical_tag_id onto the suggestion changed its
// identity key and left the panel row behind).
const s = useSuggestionsStore() const s = useSuggestionsStore()
const raw = sugg({ await seed(s, { general: [sugg({ canonical_tag_id: 12, display_name: 'Holding Sword', category: 'general' })] })
canonical_tag_id: null, display_name: 'Holding Sword',
category: 'general', creates_new_tag: true,
})
await seed(s, { general: [raw] })
const calls = [] const calls = []
stubFetch((url, init) => { stubFetch((url, init) => {
calls.push({ url, body: init?.body }) calls.push({ url, body: init?.body })
@@ -107,6 +117,5 @@ describe('suggestions store: accept with a known tag id', () => {
expect(calls[0].url).toContain('/api/images/1/suggestions/accept') expect(calls[0].url).toContain('/api/images/1/suggestions/accept')
expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 42 }) expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 42 })
expect(s.byCategory.general).toHaveLength(0) expect(s.byCategory.general).toHaveLength(0)
expect(s.allByCategory.general).toHaveLength(0)
}) })
}) })
+29
View File
@@ -81,6 +81,35 @@ async def test_detector_settings_defaults_patch_and_validation(client):
"/api/ml/settings", json={"detector_max_regions": 0})).status_code == 400 "/api/ml/settings", json={"detector_max_regions": 0})).status_code == 400
@pytest.mark.asyncio
async def test_presentation_settings_defaults_patch_and_validation(client):
# #141: presentation-chrome auto-hide knobs (banner/editor auto-apply +
# the "also looks like content" conflict threshold) are exposed + editable.
body = await (await client.get("/api/ml/settings")).get_json()
assert body["presentation_auto_apply_enabled"] is True
assert body["presentation_auto_apply_threshold"] == 0.90
assert body["presentation_conflict_threshold"] == 0.50
ok = await client.patch("/api/ml/settings", json={
"presentation_auto_apply_enabled": False,
"presentation_auto_apply_threshold": 0.95,
"presentation_conflict_threshold": 0.4,
})
assert ok.status_code == 200
out = await ok.get_json()
assert out["presentation_auto_apply_enabled"] is False
assert out["presentation_auto_apply_threshold"] == 0.95
assert out["presentation_conflict_threshold"] == 0.4
# Auto-apply threshold below the floor + conflict cut above 1 are rejected.
assert (await client.patch(
"/api/ml/settings",
json={"presentation_auto_apply_threshold": 0.4})).status_code == 400
assert (await client.patch(
"/api/ml/settings",
json={"presentation_conflict_threshold": 1.5})).status_code == 400
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_embedder_models_list(client): async def test_embedder_models_list(client):
# #1203: the dropdown reads the supported-model list from the server. # #1203: the dropdown reads the supported-model list from the server.
+62
View File
@@ -184,6 +184,68 @@ async def test_rejects_bad_direction(client):
assert (await resp.get_json())["error"] == "invalid_direction" assert (await resp.get_json())["error"] == "invalid_direction"
@pytest.mark.asyncio
async def test_translation_override_rejects_bad_value(client, seeded_post):
_, _, post = seeded_post
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "nope"}
)
assert resp.status_code == 400
assert (await resp.get_json())["error"] == "invalid_override"
@pytest.mark.asyncio
async def test_translation_override_404_for_unknown(client):
resp = await client.post(
"/api/posts/999999/translation-override", json={"override": "original"}
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_translation_override_original_clears_and_keeps(client, db, seeded_post):
# 'keep original' clears a stored translation immediately (no Interpreter) and
# marks the post handled (== target). Asserts on the response, which reflects
# the endpoint's own committed session.
_, _, post = seeded_post
post.post_title_translated = "STALE"
post.description_translated = "STALE"
post.translated_source_lang = "de"
await db.commit()
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "original"}
)
assert resp.status_code == 200
body = await resp.get_json()
assert body["translation_override"] == "original"
assert body["applied"] == "cleared"
assert body["post_title_translated"] is None
assert body["translated_source_lang"] == "en"
@pytest.mark.asyncio
async def test_translation_override_force_queues_when_disabled(client, seeded_post):
# Translation disabled (default) → can't translate inline; the override is
# saved and the post is queued (columns NULLed) for the next sweep.
_, _, post = seeded_post
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "force"}
)
assert resp.status_code == 200
body = await resp.get_json()
assert body["translation_override"] == "force"
assert body["applied"] == "queued"
assert body["translated_source_lang"] is None
@pytest.mark.asyncio
async def test_list_exposes_translation_override(client, seeded_post):
resp = await client.get("/api/posts")
assert resp.status_code == 200
body = await resp.get_json()
assert body["items"][0]["translation_override"] == "auto" # default
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_detail_returns_uncapped_thumbnails(client, db): async def test_detail_returns_uncapped_thumbnails(client, db):
"""Feed query caps thumbnails at 6 for previews; detail endpoint """Feed query caps thumbnails at 6 for previews; detail endpoint
+220
View File
@@ -28,6 +28,226 @@ async def test_patch_import_settings(client):
assert body["transparency_threshold"] == 0.7 assert body["transparency_threshold"] == 0.7
@pytest.mark.asyncio
async def test_translation_settings_defaults_and_patch(client):
# #143: translation is OFF by default with NO default host — the operator
# points it at their own Interpreter proxy and enables it.
body = await (await client.get("/api/settings/import")).get_json()
assert body["translation_enabled"] is False
assert body["interpreter_base_url"] == ""
assert body["translation_target_lang"] == "en"
ok = await client.patch("/api/settings/import", json={
"translation_enabled": True,
"interpreter_base_url": "http://interpreter.lan",
})
assert ok.status_code == 200
out = await ok.get_json()
assert out["translation_enabled"] is True
assert out["interpreter_base_url"] == "http://interpreter.lan"
# Type validation.
assert (await client.patch(
"/api/settings/import", json={"translation_enabled": "yes"})).status_code == 400
assert (await client.patch(
"/api/settings/import", json={"interpreter_base_url": 123})).status_code == 400
@pytest.mark.asyncio
async def test_translation_min_confidence_default_and_patch(client):
# #155: the latin-script acceptance floor is an operator-tunable setting,
# default 0.9 (stricter than the old hardcoded 0.8).
body = await (await client.get("/api/settings/import")).get_json()
assert body["translation_min_confidence"] == 0.9
ok = await client.patch(
"/api/settings/import", json={"translation_min_confidence": 0.95}
)
assert ok.status_code == 200
assert (await ok.get_json())["translation_min_confidence"] == 0.95
# Out-of-range + wrong-type are rejected.
assert (await client.patch(
"/api/settings/import", json={"translation_min_confidence": 1.5})).status_code == 400
assert (await client.patch(
"/api/settings/import", json={"translation_min_confidence": "high"})).status_code == 400
@pytest.mark.asyncio
async def test_translation_status_defaults(client):
# Off + no URL → no network health call, healthy False (#143).
body = await (await client.get("/api/settings/translation/status")).get_json()
assert body["enabled"] is False
assert body["base_url_set"] is False
assert body["healthy"] is False
assert "untranslated_count" in body
assert body["active"] is False # no sweep running
assert body["last_run"] is None
@pytest.mark.asyncio
async def test_translation_status_reports_active_and_last_run(client, db):
# A running translate/retranslate TaskRun → active True; the most recent
# finished one → last_run (task basename + status).
from datetime import UTC, datetime
from backend.app.models import TaskRun
db.add(TaskRun(
celery_task_id="r-run", queue="maintenance_long",
task_name="backend.app.tasks.translation.retranslate_posts",
started_at=datetime.now(UTC), status="running",
))
db.add(TaskRun(
celery_task_id="r-done", queue="maintenance_long",
task_name="backend.app.tasks.translation.translate_posts",
started_at=datetime.now(UTC), finished_at=datetime.now(UTC),
status="success",
))
await db.commit()
body = await (await client.get("/api/settings/translation/status")).get_json()
assert body["active"] is True
assert body["last_run"]["task"] == "translate_posts"
assert body["last_run"]["status"] == "success"
@pytest.mark.asyncio
async def test_translation_run_requires_config(client):
resp = await client.post("/api/settings/translation/run")
assert resp.status_code == 400 # disabled / no base URL
@pytest.mark.asyncio
async def test_translation_test_connection(client, monkeypatch):
# Tests a GIVEN url (not the saved one) without persisting anything.
monkeypatch.setattr("backend.app.api.settings.ic.health", lambda *a, **k: True)
resp = await client.post(
"/api/settings/translation/test", json={"base_url": "http://i.lan"})
assert resp.status_code == 200
assert (await resp.get_json())["healthy"] is True
# Empty URL → False, no health call.
empty = await client.post(
"/api/settings/translation/test", json={"base_url": ""})
assert (await empty.get_json())["healthy"] is False
@pytest.mark.asyncio
async def test_translation_run_enqueues_when_configured(client, monkeypatch):
sink = {}
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.delay",
lambda *a, **k: sink.update(k) or type("R", (), {"id": "t1"})(),
)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post("/api/settings/translation/run")
assert resp.status_code == 202
assert (await resp.get_json())["celery_task_id"] == "t1"
assert sink["drain"] is True # "Translate now" chases the whole backlog
@pytest.mark.asyncio
async def test_translation_probe_requires_text(client):
resp = await client.post(
"/api/settings/translation/probe", json={"text": " "})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_requires_base_url(client):
# Text given but no Interpreter URL saved → nothing to probe against.
resp = await client.post(
"/api/settings/translation/probe", json={"text": "hello"})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_returns_detection(client, monkeypatch):
# The diagnostic surfaces detected language + confidence + result, unsaved.
monkeypatch.setattr(
"backend.app.api.settings.ic.translate",
lambda texts, **k: {
"translations": ["Work in Progress"],
"detected_lang": "de", "detected_confidence": 71.5,
"engine": "llm", "engine_version": "v1",
},
)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post(
"/api/settings/translation/probe", json={"text": "WIP"})
assert resp.status_code == 200
body = await resp.get_json()
assert body["detected_lang"] == "de"
assert body["detected_confidence"] == 71.5
assert body["translated"] == "Work in Progress"
assert body["target"] == "en"
def _fake_retranslate(monkeypatch, sink):
# Record the kwargs the endpoint enqueues with, return a fake AsyncResult.
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.delay",
lambda *a, **k: sink.update(k) or type("R", (), {"id": "r1"})(),
)
@pytest.mark.asyncio
async def test_translation_retranslate_requires_config(client):
# Disabled → 400 even with an explicit scope (no reset is enqueued).
resp = await client.post(
"/api/settings/translation/retranslate", json={"all": True})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_retranslate_requires_target(client, monkeypatch):
# 'all' must be explicit: an empty body is a 400, so a stray call can't
# wipe every translation.
_fake_retranslate(monkeypatch, {})
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post("/api/settings/translation/retranslate")
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_retranslate_artist_scope(client, monkeypatch):
sink = {}
_fake_retranslate(monkeypatch, sink)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post(
"/api/settings/translation/retranslate", json={"artist_id": 7})
assert resp.status_code == 202
assert (await resp.get_json())["celery_task_id"] == "r1"
assert sink["artist_ids"] == [7]
@pytest.mark.asyncio
async def test_translation_retranslate_all_scope(client, monkeypatch):
sink = {}
_fake_retranslate(monkeypatch, sink)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post(
"/api/settings/translation/retranslate", json={"all": True})
assert resp.status_code == 202
assert sink["artist_ids"] is None
@pytest.mark.asyncio
async def test_translation_retranslate_bad_artist_id(client):
resp = await client.post(
"/api/settings/translation/retranslate", json={"artist_id": "abc"})
assert resp.status_code == 400
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_patch_rejects_non_object(client): async def test_patch_rejects_non_object(client):
resp = await client.patch("/api/settings/import", json=[1, 2, 3]) resp = await client.patch("/api/settings/import", json=[1, 2, 3])
+1 -9
View File
@@ -52,6 +52,7 @@ async def test_get_suggestions(client, db):
general = body["by_category"].get("general", []) general = body["by_category"].get("general", [])
s2 = next(x for x in general if x["canonical_tag_id"] == tag.id) s2 = next(x for x in general if x["canonical_tag_id"] == tag.id)
assert s2["source"] == "head" assert s2["source"] == "head"
assert s2["above_threshold"] is True # ~0.73 clears the 0.5 suggest cut
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -114,12 +115,3 @@ async def test_undismiss_reverses_rejection(client, db):
f"/api/images/{img.id}/suggestions/undismiss", json={"tag_id": tag.id} f"/api/images/{img.id}/suggestions/undismiss", json={"tag_id": tag.id}
) )
assert resp2.status_code == 204 assert resp2.status_code == 204
@pytest.mark.asyncio
async def test_alias_requires_fields(client, db):
img = await _img(db)
resp = await client.post(
f"/api/images/{img.id}/suggestions/alias", json={"alias_string": "x"}
)
assert resp.status_code == 400
+32
View File
@@ -141,6 +141,38 @@ async def test_applied_tag_grounding_returns_winning_region(client, db):
assert body["grounding"]["kind"] == "concept" assert body["grounding"]["kind"] == "concept"
@pytest.mark.asyncio
async def test_applied_tags_expose_source_and_confirmed(client, db):
# The chip UI needs each applied tag's source (auto vs manual) + confirmed
# state to badge auto-tags and offer Keep/confirm (milestone 139).
from backend.app.models import TagPositiveConfirmation
from backend.app.models.tag import image_tag
img = ImageRecord(
path="/images/srcflag.jpg", sha256="sf" * 32, size_bytes=1,
mime="image/jpeg", width=1, height=1, origin="imported_filesystem",
integrity_status="unknown",
)
db.add(img)
await db.flush()
auto = await TagService(db).find_or_create("autotag", TagKind.general)
manual = await TagService(db).find_or_create("manualtag", TagKind.general)
await db.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=auto.id, source="head_auto"))
await db.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=manual.id, source="manual"))
db.add(TagPositiveConfirmation(image_record_id=img.id, tag_id=manual.id))
await db.commit()
resp = await client.get(f"/api/images/{img.id}/tags")
assert resp.status_code == 200
by_id = {t["id"]: t for t in await resp.get_json()}
assert by_id[auto.id]["source"] == "head_auto"
assert by_id[auto.id]["confirmed"] is False
assert by_id[manual.id]["source"] == "manual"
assert by_id[manual.id]["confirmed"] is True # has a confirmation row
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_applied_tag_grounding_no_head(client, db): async def test_applied_tag_grounding_no_head(client, db):
# A tag with no head can't be localized → has_head False, grounding null; the # A tag with no head can't be localized → has_head False, grounding null; the
+22
View File
@@ -12,6 +12,7 @@ from backend.app.models import (
MLSettings, MLSettings,
Tag, Tag,
TagKind, TagKind,
TagPositiveConfirmation,
) )
from backend.app.models.tag import image_tag from backend.app.models.tag import image_tag
from backend.app.services.ml.character_prototypes import ( from backend.app.services.ml.character_prototypes import (
@@ -141,6 +142,27 @@ def test_multi_character_image_not_referenced(db_sync):
assert _proto_count(db_sync, daphne.id) == 0 assert _proto_count(db_sync, daphne.id) == 0
def test_unconfirmed_auto_char_tag_not_referenced(db_sync):
# A single-character image whose ONLY character tag was AUTO-applied
# (unconfirmed) must NOT seed a prototype — else CCIP self-trains on its own
# guess (milestone 139). Confirming it (which trips the global gate) makes it
# a reference.
raven = _char(db_sync, "Raven")
img = _img(db_sync, "z" * 64)
_figure(db_sync, img.id)
db_sync.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=raven.id, source="ccip_auto",
))
db_sync.commit()
refresh_character_prototypes(db_sync)
assert _proto_count(db_sync, raven.id) == 0
db_sync.add(TagPositiveConfirmation(image_record_id=img.id, tag_id=raven.id))
db_sync.commit()
refresh_character_prototypes(db_sync)
assert _proto_count(db_sync, raven.id) == 1
def test_lost_references_are_removed(db_sync): def test_lost_references_are_removed(db_sync):
raven = _char(db_sync, "Raven") raven = _char(db_sync, "Raven")
img = _img(db_sync, "e" * 64) img = _img(db_sync, "e" * 64)
+42
View File
@@ -1,6 +1,7 @@
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
import pytest import pytest
from sqlalchemy import select
from backend.app.models import ImageRecord, Tag, TagKind from backend.app.models import ImageRecord, Tag, TagKind
from backend.app.models.tag import image_tag from backend.app.models.tag import image_tag
@@ -73,6 +74,47 @@ async def test_scroll_returns_newest_first(db):
assert page.images[0].created_at > page.images[-1].created_at assert page.images[0].created_at > page.images[-1].created_at
async def _system_tag(db, name):
return (await db.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == name)
)).scalar_one()
@pytest.mark.asyncio
async def test_scroll_hides_presentation_chrome_by_default(db):
# banner / editor screenshot (presentation system tags) are hidden from the
# default gallery; wip (also a system tag) is NOT — it's real, in-progress
# art (milestone 141). Seeded system tags survive the harness TRUNCATE.
imgs = await _seed_images(db, 3, sha_prefix="p")
banner = await _system_tag(db, "banner")
wip = await _system_tag(db, "wip")
await db.execute(image_tag.insert().values(
image_record_id=imgs[0].id, tag_id=banner.id, source="manual"))
await db.execute(image_tag.insert().values(
image_record_id=imgs[1].id, tag_id=wip.id, source="manual"))
await db.flush()
svc = GalleryService(db)
# Default: the banner image is hidden; the wip image + the plain image stay.
default_ids = {i.id for i in (await svc.scroll(cursor=None, limit=10)).images}
assert imgs[0].id not in default_ids # banner hidden
assert imgs[1].id in default_ids # wip visible
assert imgs[2].id in default_ids # plain visible
# include_hidden surfaces the banner image (the Hidden view).
shown = {i.id for i in (
await svc.scroll(cursor=None, limit=10, include_hidden=True)
).images}
assert imgs[0].id in shown
# Filtering explicitly FOR banner shows it (a view exclusively for a
# presentation tag suppresses the implicit hide).
only_banner = {i.id for i in (
await svc.scroll(cursor=None, limit=10, tag_ids=[banner.id])
).images}
assert only_banner == {imgs[0].id}
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_scroll_sorts_by_earliest_post_date(db): async def test_scroll_sorts_by_earliest_post_date(db):
"""posted_new/posted_old key off earliest_post_date (original publish across """posted_new/posted_old key off earliest_post_date (original publish across
+24
View File
@@ -98,6 +98,30 @@ async def test_similar_excludes_presentation_tagged_images(db):
assert {i.id for i in res_from_banner} == {src.id, wipped.id, plain.id} assert {i.id for i in res_from_banner} == {src.id, wipped.id, plain.id}
@pytest.mark.asyncio
async def test_similar_exclude_wip_drops_wip_neighbors(db):
"""Explore passes exclude_wip=True to also hide work-in-progress from the
rabbit-hole (banner is always hidden; wip only when asked)."""
src = await _img(db, 1, _vec(1, 0))
bannered = await _img(db, 2, _vec(1, 0.02)) # always hidden
wipped = await _img(db, 3, _vec(1, 0.3)) # hidden only with exclude_wip
plain = await _img(db, 4, _vec(1, 0.6))
banner_tag = (await db.execute(select(Tag).where(
Tag.is_system.is_(True), Tag.name == "banner"))).scalar_one()
wip_tag = (await db.execute(select(Tag).where(
Tag.is_system.is_(True), Tag.name == "wip"))).scalar_one()
await db.execute(image_tag.insert().values(
image_record_id=bannered.id, tag_id=banner_tag.id, source="manual"))
await db.execute(image_tag.insert().values(
image_record_id=wipped.id, tag_id=wip_tag.id, source="manual"))
svc = GalleryService(db)
res = await svc.similar(src.id, limit=10, exclude_wip=True)
assert [i.id for i in res] == [plain.id] # wip + banner both gone
# Default (gallery "similar" button) still keeps wip (#1274).
res_default = await svc.similar(src.id, limit=10)
assert wipped.id in {i.id for i in res_default}
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_similar_composes_with_tag_filter(db): async def test_similar_composes_with_tag_filter(db):
src = await _img(db, 1, _vec(1, 0)) src = await _img(db, 1, _vec(1, 0))
+2 -2
View File
@@ -35,7 +35,7 @@ def _img(db, sha: str, emb) -> ImageRecord:
return img return img
def _head(db, tag_id: int, slot: int, *, threshold=0.5, n_pos=30): def _head(db, tag_id: int, slot: int, *, threshold=0.5, n_pos=60):
s = db.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one() s = db.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
w = [0.0] * 1152 w = [0.0] * 1152
w[slot] = 1.0 w[slot] = 1.0
@@ -88,7 +88,7 @@ def test_sweep_dry_run_counts_but_writes_nothing(db_sync):
def test_sweep_skips_under_supported_head(db_sync): def test_sweep_skips_under_supported_head(db_sync):
# n_pos below head_auto_apply_min_positives (default 30) → a precise-looking # n_pos below head_auto_apply_min_positives (default 50) → a precise-looking
# but under-supported head never fires. # but under-supported head never fires.
img = _img(db_sync, "c" * 64, _emb(0)) img = _img(db_sync, "c" * 64, _emb(0))
tag = Tag(name="weaktag", kind=TagKind.general) tag = Tag(name="weaktag", kind=TagKind.general)
+69
View File
@@ -0,0 +1,69 @@
"""Soft auto-apply (milestone 139): unconfirmed auto-applied tags do NOT train a
head. _ids_with_tag (positives) + _eligible_tag_ids (graduation count) count
human-applied + operator-confirmed tags only. Sklearn-free, so tested via
db_sync."""
import pytest
from backend.app.models import ImageRecord, Tag, TagKind, TagPositiveConfirmation
from backend.app.models.tag import image_tag
from backend.app.services.ml.heads import _eligible_tag_ids
from backend.app.services.ml.training_data import _ids_with_tag
pytestmark = pytest.mark.integration
def _img(db, sha: str) -> ImageRecord:
img = ImageRecord(
path=f"/images/{sha}.jpg", sha256=sha, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown",
)
db.add(img)
db.flush()
return img
def _tag(db, name: str) -> Tag:
t = Tag(name=name, kind=TagKind.general)
db.add(t)
db.flush()
return t
def _apply(db, image_id: int, tag_id: int, source: str) -> None:
db.execute(image_tag.insert().values(
image_record_id=image_id, tag_id=tag_id, source=source,
))
def test_positives_exclude_unconfirmed_auto(db_sync):
tag = _tag(db_sync, "glasses")
man = _img(db_sync, "a" * 64)
auto = _img(db_sync, "b" * 64)
conf = _img(db_sync, "c" * 64)
acc = _img(db_sync, "d" * 64)
_apply(db_sync, man.id, tag.id, "manual")
_apply(db_sync, auto.id, tag.id, "head_auto") # unconfirmed → excluded
_apply(db_sync, conf.id, tag.id, "head_auto") # confirmed → included
_apply(db_sync, acc.id, tag.id, "ml_accepted")
db_sync.add(TagPositiveConfirmation(image_record_id=conf.id, tag_id=tag.id))
db_sync.commit()
pos = set(_ids_with_tag(db_sync, tag.id))
assert pos == {man.id, conf.id, acc.id}
assert auto.id not in pos
def test_eligibility_counts_positives_only(db_sync):
# A concept whose only tags are unconfirmed auto-applies does NOT graduate.
tag = _tag(db_sync, "autotag")
for i in range(3):
_apply(db_sync, _img(db_sync, f"e{i}" * 32).id, tag.id, "head_auto")
db_sync.commit()
assert tag.id not in _eligible_tag_ids(db_sync, min_pos=2)
# Two human positives → now eligible at min_pos=2.
for i in range(2):
_apply(db_sync, _img(db_sync, f"h{i}" * 32).id, tag.id, "manual")
db_sync.commit()
assert tag.id in _eligible_tag_ids(db_sync, min_pos=2)
+89
View File
@@ -0,0 +1,89 @@
"""Hidden-view review endpoints (#141): list flagged auto-hides + keep / un-hide."""
import pytest
from sqlalchemy import select
from backend.app.models import (
ImageRecord,
PresentationReview,
Tag,
TagKind,
TagSuggestionRejection,
)
from backend.app.models.tag import image_tag
pytestmark = pytest.mark.integration
async def _setup_flag(db):
banner = (await db.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == "banner")
)).scalar_one()
content = Tag(name="looksreal", kind=TagKind.general)
db.add(content)
img = ImageRecord(
path="/images/rv.jpg", sha256="rv" * 32, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown",
)
db.add(img)
await db.flush()
await db.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=banner.id, source="presentation_auto"))
db.add(PresentationReview(
image_record_id=img.id, tag_id=banner.id,
conflict_tag_id=content.id, conflict_score=0.8,
))
await db.commit()
return img, banner, content
async def _source(db, image_id, tag_id):
return (await db.execute(
select(image_tag.c.source).where(
image_tag.c.image_record_id == image_id,
image_tag.c.tag_id == tag_id,
)
)).scalar_one_or_none()
@pytest.mark.asyncio
async def test_hidden_review_lists_flag(client, db):
img, banner, content = await _setup_flag(db)
body = await (await client.get("/api/gallery/hidden-review")).get_json()
assert len(body["items"]) == 1
it = body["items"][0]
assert it["image_id"] == img.id
assert it["tag_id"] == banner.id
assert it["conflict_tag_id"] == content.id
assert it["conflict_name"] == "looksreal"
assert it["conflict_score"] == 0.8
assert it["thumbnail_url"]
@pytest.mark.asyncio
async def test_hidden_review_keep_resolves_but_keeps_tag(client, db):
img, banner, _ = await _setup_flag(db)
resp = await client.post(
f"/api/gallery/hidden-review/{img.id}/{banner.id}/keep")
assert resp.status_code == 204
body = await (await client.get("/api/gallery/hidden-review")).get_json()
assert body["items"] == [] # flag resolved
assert await _source(db, img.id, banner.id) == "presentation_auto" # tag stays
@pytest.mark.asyncio
async def test_hidden_review_unhide_removes_tag_and_records_rejection(client, db):
img, banner, _ = await _setup_flag(db)
resp = await client.post(
f"/api/gallery/hidden-review/{img.id}/{banner.id}/unhide")
assert resp.status_code == 204
assert await _source(db, img.id, banner.id) is None # back in the gallery
rej = (await db.execute(
select(TagSuggestionRejection).where(
TagSuggestionRejection.image_record_id == img.id,
TagSuggestionRejection.tag_id == banner.id,
)
)).scalar_one_or_none()
assert rej is not None # head learns it misfired
body = await (await client.get("/api/gallery/hidden-review")).get_json()
assert body["items"] == []
+145
View File
@@ -0,0 +1,145 @@
"""Interpreter translation client (#143) — unit tests with mocked HTTP (no DB,
no real network)."""
import pytest
from backend.app.services import interpreter_client as ic
class _Resp:
def __init__(self, status_code=200, payload=None, headers=None):
self.status_code = status_code
self._payload = payload or {}
self.headers = headers or {}
def json(self):
return self._payload
def raise_for_status(self):
if self.status_code >= 400:
raise ic.requests.HTTPError(f"status {self.status_code}")
def test_translate_maps_batch_in_order(monkeypatch):
captured = {}
def fake_post(url, json, timeout):
captured["json"] = json
return _Resp(200, {
"translatedText": ["The cat is cute", "Blonde gal!"],
"detectedLanguage": {"language": "ja", "confidence": 0.98},
"interpreter": {"engine": "llm", "engine_version": "ollama:x:12b"},
})
monkeypatch.setattr(ic.session, "post", fake_post)
out = ic.translate(["ねこが可愛い", "金髪ギャル!"], base_url="http://i.lan")
assert out["translations"] == ["The cat is cute", "Blonde gal!"]
assert out["detected_lang"] == "ja"
assert out["detected_confidence"] == 0.98 # surfaced for the detection guard
assert out["engine_version"] == "ollama:x:12b"
assert captured["json"]["q"] == ["ねこが可愛い", "金髪ギャル!"]
assert captured["json"]["target"] == "en"
# Source MUST stay "auto": an explicit source makes Interpreter skip detection
# and report confidence 1.0, which would defeat the acceptance gate.
assert captured["json"]["source"] == "auto"
def test_translate_passthrough_unchanged(monkeypatch):
# Already-English items come back unchanged in their slot (engine "none").
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(200, {
"translatedText": ["already english"],
"detectedLanguage": {"language": "en"},
"interpreter": {"engine": "none", "engine_version": None},
}))
out = ic.translate(["already english"], base_url="http://i.lan")
assert out["translations"] == ["already english"]
assert out["detected_lang"] == "en"
assert out["engine"] == "none"
def test_translate_503_raises_unavailable(monkeypatch):
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(503, {}))
with pytest.raises(ic.InterpreterUnavailable):
ic.translate(["x"], base_url="http://i.lan")
def test_translate_connection_error_raises_unavailable(monkeypatch):
def boom(*a, **k):
raise ic.requests.ConnectionError("refused")
monkeypatch.setattr(ic.session, "post", boom)
with pytest.raises(ic.InterpreterUnavailable):
ic.translate(["x"], base_url="http://i.lan")
@pytest.mark.parametrize("code", [429, 500, 502, 503, 504])
def test_translate_429_and_5xx_raise_unavailable(monkeypatch, code):
# A gracefully-draining service behind a reverse proxy returns 429/502/503/504
# — all mean "retry later", not an opaque error.
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(code, {}))
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after is None
def test_translate_honours_retry_after_seconds(monkeypatch):
monkeypatch.setattr(
ic.session, "post",
lambda *a, **k: _Resp(503, {}, headers={"Retry-After": "42"}),
)
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after == 42
def test_translate_retry_after_past_http_date_clamps_to_zero(monkeypatch):
# HTTP-date form; a past date → non-negative clamp to 0 (deterministic).
monkeypatch.setattr(
ic.session, "post",
lambda *a, **k: _Resp(
503, {}, headers={"Retry-After": "Wed, 21 Oct 2015 07:28:00 GMT"}),
)
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after == 0.0
def test_translate_400_raises_bad_request(monkeypatch):
monkeypatch.setattr(
ic.session, "post", lambda *a, **k: _Resp(400, {"error": "bad target"})
)
with pytest.raises(ic.InterpreterBadRequest):
ic.translate(["x"], base_url="http://i.lan")
def test_translate_empty_is_noop(monkeypatch):
def boom(*a, **k):
raise AssertionError("should not call the service for an empty batch")
monkeypatch.setattr(ic.session, "post", boom)
assert ic.translate([], base_url="http://i.lan")["translations"] == []
def test_health_true_when_llm_up(monkeypatch):
monkeypatch.setattr(
ic.session, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": True}})
)
assert ic.health("http://i.lan") is True
def test_health_false_when_engine_down(monkeypatch):
monkeypatch.setattr(
ic.session, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": False}})
)
assert ic.health("http://i.lan") is False
def test_health_false_on_error(monkeypatch):
def boom(*a, **k):
raise ic.requests.ConnectionError("refused")
monkeypatch.setattr(ic.session, "get", boom)
assert ic.health("http://i.lan") is False
def test_health_false_when_url_empty():
assert ic.health("") is False
+22
View File
@@ -26,6 +26,28 @@ def _make_batch(session) -> int:
return batch.id return batch.id
def test_cleanup_orphaned_temp_files_removes_stale_only(tmp_path, monkeypatch):
import os
from backend.app.tasks import maintenance as m
monkeypatch.setattr(m, "IMAGES_ROOT", tmp_path)
stale = tmp_path / "artist" / "img.jpg.part" # killed download → orphan
stale.parent.mkdir(parents=True)
stale.write_bytes(b"x")
old = datetime.now(UTC).timestamp() - 8 * 3600 # older than the 6h guard
os.utime(stale, (old, old))
fresh = tmp_path / "in_progress.jpg.partial" # active download → keep
fresh.write_bytes(b"x")
keep = tmp_path / "real.jpg" # a real image → keep
keep.write_bytes(b"x")
assert m.cleanup_orphaned_temp_files() == 1
assert not stale.exists()
assert fresh.exists()
assert keep.exists()
def test_recover_interrupted_only_old(db_sync, monkeypatch): def test_recover_interrupted_only_old(db_sync, monkeypatch):
batch_id = _make_batch(db_sync) batch_id = _make_batch(db_sync)
now = datetime.now(UTC) now = datetime.now(UTC)
+6 -3
View File
@@ -62,9 +62,8 @@ async def test_head_suggestion_surfaces_for_matching_image(db):
s = general[0] s = general[0]
assert s.canonical_tag_id == tag.id assert s.canonical_tag_id == tag.id
assert s.source == "head" assert s.source == "head"
assert s.creates_new_tag is False
assert s.via_alias is False and s.raw_name is None
assert s.score > 0.5 assert s.score > 0.5
assert s.above_threshold is True # ~0.73 clears the 0.5 suggest cut
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -109,7 +108,10 @@ async def test_threshold_override_surfaces_below_cut(db):
svc = SuggestionService(db) svc = SuggestionService(db)
assert svc and not (await svc.for_image(img.id)).by_category.get("general") assert svc and not (await svc.for_image(img.id)).by_category.get("general")
flooded = await svc.for_image(img.id, threshold_override=0.0) flooded = await svc.for_image(img.id, threshold_override=0.0)
assert any(s.canonical_tag_id == tag.id for s in flooded.by_category["general"]) s = next(s for s in flooded.by_category["general"] if s.canonical_tag_id == tag.id)
# Included by the floor, but flagged below its own cut (0.5 < 0.6) — this is
# what lets the dropdown show it while the panel hides it.
assert s.above_threshold is False
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -281,6 +283,7 @@ async def test_ccip_character_surfaces_in_rail(db):
if c.canonical_tag_id == raven.id if c.canonical_tag_id == raven.id
) )
assert m.source == "ccip" assert m.source == "ccip"
assert m.above_threshold is True # CCIP only returns matches above its cut
# --- #1206 Step 4: on-demand grounding for ALREADY-APPLIED tag chips --------- # --- #1206 Step 4: on-demand grounding for ALREADY-APPLIED tag chips ---------
+24
View File
@@ -283,6 +283,30 @@ async def test_scroll_item_shape_minimal(db):
assert "description_full" not in item assert "description_full" not in item
@pytest.mark.asyncio
async def test_scroll_surfaces_translation_fields(db):
# #143: a translated post exposes the translated title/description + source
# lang, with the originals still present for the toggle.
artist = await _seed_artist(db, "alice-tr")
src = await _seed_source(db, artist.id, "pixiv", "https://p/alice-tr")
p = await _seed_post(
db, src.id, external_id="TR", post_date=datetime.now(UTC),
title="ねこ", description="<p>かわいい</p>",
)
p.post_title_translated = "cat"
p.description_translated = "cute"
p.translated_source_lang = "ja"
await db.commit()
page = await PostFeedService(db).scroll(cursor=None, limit=10)
item = page["items"][0]
assert item["post_title_translated"] == "cat"
assert item["description_translated"] == "cute"
assert item["translated_source_lang"] == "ja"
assert item["post_title"] == "ねこ" # original kept for the toggle
assert item["description_plain"] == "かわいい"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_scroll_description_truncates_long_text(db): async def test_scroll_description_truncates_long_text(db):
artist = await _seed_artist(db, "alice-long") artist = await _seed_artist(db, "alice-long")
+152
View File
@@ -0,0 +1,152 @@
"""Presentation-chrome auto-hide sweep (#141). numpy-only (no sklearn), so the
apply + guard logic is tested directly via the sync session."""
import pytest
from sqlalchemy import select
from backend.app.models import (
HeadAutoApplyRun,
ImageRecord,
MLSettings,
PresentationReview,
Tag,
TagHead,
TagKind,
)
from backend.app.models.tag import image_tag
from backend.app.services.ml.heads import (
auto_apply_sweep,
presentation_auto_apply_sweep,
)
pytestmark = pytest.mark.integration
def _emb(slot: int) -> list[float]:
v = [0.0] * 1152
v[slot] = 3.0
return v
def _img(db, sha: str, emb) -> ImageRecord:
img = ImageRecord(
path=f"/images/{sha}.jpg", sha256=sha, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown", siglip_embedding=emb,
)
db.add(img)
db.flush()
return img
def _head(db, tag_id: int, slot: int, *, weight=1.0):
# weight 3.0 → score sigmoid(3)=0.95 clears the 0.90 presentation floor;
# weight 1.0 → sigmoid(1)=0.73 clears the 0.50 conflict floor.
s = db.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
w = [0.0] * 1152
w[slot] = weight
db.add(TagHead(
tag_id=tag_id, embedding_version=s.embedder_model_version,
weights=w, bias=0.0, suggest_threshold=0.5, auto_apply_threshold=0.5,
n_pos=60, n_neg=90, ap=0.9, precision_cv=0.98, recall=0.7,
))
def _system_tag(db, name):
return db.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == name)
).scalar_one()
def _source(db, image_id, tag_id):
return db.execute(
select(image_tag.c.source)
.where(image_tag.c.image_record_id == image_id)
.where(image_tag.c.tag_id == tag_id)
).scalar_one_or_none()
def test_presentation_sweep_hides_chrome(db_sync):
banner = _system_tag(db_sync, "banner")
_head(db_sync, banner.id, 0, weight=3.0)
img = _img(db_sync, "a" * 64, _emb(0))
db_sync.commit()
res = presentation_auto_apply_sweep(db_sync)
assert res["n_applied"] == 1
assert _source(db_sync, img.id, banner.id) == "presentation_auto"
def test_presentation_sweep_hard_skips_valued_image(db_sync):
# An image the operator already tagged with a content tag is never auto-hidden.
banner = _system_tag(db_sync, "banner")
_head(db_sync, banner.id, 0, weight=3.0)
content = Tag(name="mychar", kind=TagKind.character)
db_sync.add(content)
db_sync.flush()
img = _img(db_sync, "b" * 64, _emb(0))
db_sync.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=content.id, source="manual"))
db_sync.commit()
res = presentation_auto_apply_sweep(db_sync)
assert res["n_applied"] == 0
assert _source(db_sync, img.id, banner.id) is None
def test_presentation_sweep_flags_conflict(db_sync):
# Matches banner AND scores high on a content head → hidden but flagged with
# that content tag as the conflict.
banner = _system_tag(db_sync, "banner")
_head(db_sync, banner.id, 0, weight=3.0)
content = Tag(name="looksreal", kind=TagKind.general)
db_sync.add(content)
db_sync.flush()
_head(db_sync, content.id, 0, weight=1.0) # content head also fires
img = _img(db_sync, "c" * 64, _emb(0))
db_sync.commit()
res = presentation_auto_apply_sweep(db_sync)
assert res["n_applied"] == 1
assert res["n_flagged"] == 1
assert _source(db_sync, img.id, banner.id) == "presentation_auto"
flag = db_sync.execute(
select(PresentationReview).where(
PresentationReview.image_record_id == img.id,
PresentationReview.tag_id == banner.id,
)
).scalar_one()
assert flag.conflict_tag_id == content.id
assert flag.conflict_score >= 0.5
def test_presentation_sweep_disabled_is_noop(db_sync):
s = db_sync.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
s.presentation_auto_apply_enabled = False
banner = _system_tag(db_sync, "banner")
_head(db_sync, banner.id, 0, weight=3.0)
img = _img(db_sync, "d" * 64, _emb(0))
db_sync.commit()
res = presentation_auto_apply_sweep(db_sync)
assert res["n_applied"] == 0
assert _source(db_sync, img.id, banner.id) is None
def test_presentation_sweep_ignores_wip(db_sync):
# wip is a system tag but NOT presentation chrome → never auto-applied.
wip = _system_tag(db_sync, "wip")
_head(db_sync, wip.id, 0, weight=3.0)
img = _img(db_sync, "e" * 64, _emb(0))
db_sync.commit()
res = presentation_auto_apply_sweep(db_sync)
assert res["n_applied"] == 0
assert _source(db_sync, img.id, wip.id) is None
def test_content_sweep_never_fires_system_tags(db_sync):
# A graduated banner (system) head must NOT auto-apply via the content path.
banner = _system_tag(db_sync, "banner")
_head(db_sync, banner.id, 0, weight=3.0)
img = _img(db_sync, "f" * 64, _emb(0))
run = HeadAutoApplyRun(dry_run=False, params={}, status="running")
db_sync.add(run)
db_sync.flush()
db_sync.commit()
auto_apply_sweep(db_sync, run, dry_run=False)
assert _source(db_sync, img.id, banner.id) is None
+98
View File
@@ -0,0 +1,98 @@
"""Scheduled presentation tasks (#141): the daily auto-hide sweep wrapper +
resolved-flag retention. The task opens its own session via
_sync_session_factory, monkeypatched here to run against the test's db_sync."""
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import select
from backend.app.models import (
ImageRecord,
MLSettings,
PresentationReview,
Tag,
TagHead,
)
from backend.app.tasks.ml import (
prune_presentation_reviews,
scheduled_presentation_auto_apply,
)
pytestmark = pytest.mark.integration
class _Ctx:
def __init__(self, s):
self.s = s
def __enter__(self):
return self.s
def __exit__(self, *a):
return False
def _sf(db_sync):
class _SM:
def __call__(self):
return _Ctx(db_sync)
return _SM()
def _img(db, sha, emb=None):
img = ImageRecord(
path=f"/images/{sha}.jpg", sha256=sha * 32, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown", siglip_embedding=emb,
)
db.add(img)
db.flush()
return img
def test_scheduled_presentation_sweep_hides_chrome(db_sync, monkeypatch):
monkeypatch.setattr(
"backend.app.tasks.ml._sync_session_factory", lambda: _sf(db_sync)
)
banner = db_sync.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == "banner")
).scalar_one()
s = db_sync.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
w = [0.0] * 1152
w[0] = 3.0
db_sync.add(TagHead(
tag_id=banner.id, embedding_version=s.embedder_model_version,
weights=w, bias=0.0, suggest_threshold=0.5, auto_apply_threshold=0.5,
n_pos=60, n_neg=90, ap=0.9, precision_cv=0.98, recall=0.7,
))
emb = [0.0] * 1152
emb[0] = 3.0
_img(db_sync, "sc", emb)
db_sync.commit()
assert "applied=1" in scheduled_presentation_auto_apply()
def test_prune_presentation_reviews_drops_old_resolved(db_sync, monkeypatch):
monkeypatch.setattr(
"backend.app.tasks.ml._sync_session_factory", lambda: _sf(db_sync)
)
banner = db_sync.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == "banner")
).scalar_one()
a, b, c = _img(db_sync, "p1"), _img(db_sync, "p2"), _img(db_sync, "p3")
old = datetime.now(UTC) - timedelta(days=40)
recent = datetime.now(UTC) - timedelta(days=1)
# old resolved → pruned; recent resolved → kept; unresolved → kept.
db_sync.add(PresentationReview(
image_record_id=a.id, tag_id=banner.id, conflict_score=0.7, resolved_at=old))
db_sync.add(PresentationReview(
image_record_id=b.id, tag_id=banner.id, conflict_score=0.7, resolved_at=recent))
db_sync.add(PresentationReview(
image_record_id=c.id, tag_id=banner.id, conflict_score=0.7))
db_sync.commit()
assert prune_presentation_reviews() == "pruned=1"
remaining = db_sync.execute(
select(PresentationReview.image_record_id)
).scalars().all()
assert set(remaining) == {b.id, c.id}
+153
View File
@@ -0,0 +1,153 @@
"""Soft auto-apply (milestone 139): the retraction sweeps drop standing
head_auto/ccip_auto tags now below their threshold, keep the ones still above,
and never touch manual or operator-confirmed tags. Sync + sklearn-free (they
score with STORED weights/vectors), so tested directly via db_sync."""
import pytest
from sqlalchemy import select
from backend.app.models import (
CharacterPrototype,
ImageRecord,
ImageRegion,
MLSettings,
Tag,
TagHead,
TagKind,
TagPositiveConfirmation,
)
from backend.app.models.tag import image_tag
from backend.app.services.ml.character_prototypes import retract_auto_applied_ccip
from backend.app.services.ml.heads import retract_auto_applied_heads
pytestmark = pytest.mark.integration
def _emb(slot: int) -> list[float]:
v = [0.0] * 1152
v[slot] = 3.0
return v
def _ccip(slot: int) -> list[float]:
v = [0.0] * 768
v[slot] = 1.0
return v
def _img(db, sha: str, emb=None) -> ImageRecord:
img = ImageRecord(
path=f"/images/{sha}.jpg", sha256=sha, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown", siglip_embedding=emb,
)
db.add(img)
db.flush()
return img
def _figure(db, image_id: int, ccip) -> None:
db.add(ImageRegion(
image_record_id=image_id, kind="figure",
rx=0.0, ry=0.0, rw=1.0, rh=1.0,
ccip_embedding=ccip, embedding_version="ccip-test",
))
db.flush()
def _tag(db, name: str, kind: TagKind) -> Tag:
t = Tag(name=name, kind=kind)
db.add(t)
db.flush()
return t
def _apply(db, image_id: int, tag_id: int, source: str) -> None:
db.execute(image_tag.insert().values(
image_record_id=image_id, tag_id=tag_id, source=source,
))
def _version(db) -> str:
return db.execute(
select(MLSettings.embedder_model_version).where(MLSettings.id == 1)
).scalar_one()
def _head(db, tag_id: int, slot: int, threshold: float, version: str) -> None:
w = [0.0] * 1152
w[slot] = 1.0
db.add(TagHead(
tag_id=tag_id, embedding_version=version, weights=w, bias=0.0,
suggest_threshold=0.5, auto_apply_threshold=threshold,
n_pos=60, n_neg=180, ap=0.9, precision_cv=0.98, recall=0.7,
))
db.flush()
def _has_tag(db, image_id: int, tag_id: int) -> bool:
return db.execute(
select(image_tag.c.tag_id)
.where(image_tag.c.image_record_id == image_id)
.where(image_tag.c.tag_id == tag_id)
).first() is not None
def test_retract_head_auto(db_sync):
ver = _version(db_sync)
tag = _tag(db_sync, "glasses", TagKind.general)
_head(db_sync, tag.id, slot=0, threshold=0.7, version=ver)
hi = _img(db_sync, "a" * 64, _emb(0)) # aligned → ~0.73 ≥ 0.7 → keep
lo = _img(db_sync, "b" * 64, _emb(5)) # orthogonal → 0.5 < 0.7 → retract
man = _img(db_sync, "c" * 64, _emb(5)) # low score but manual → keep
conf = _img(db_sync, "d" * 64, _emb(5)) # low score, head_auto, CONFIRMED → keep
_apply(db_sync, hi.id, tag.id, "head_auto")
_apply(db_sync, lo.id, tag.id, "head_auto")
_apply(db_sync, man.id, tag.id, "manual")
_apply(db_sync, conf.id, tag.id, "head_auto")
db_sync.add(TagPositiveConfirmation(image_record_id=conf.id, tag_id=tag.id))
db_sync.commit()
assert retract_auto_applied_heads(db_sync) == 1
assert not _has_tag(db_sync, lo.id, tag.id) # retracted (below threshold)
assert _has_tag(db_sync, hi.id, tag.id) # kept (still above)
assert _has_tag(db_sync, man.id, tag.id) # kept (manual, not auto)
assert _has_tag(db_sync, conf.id, tag.id) # kept (operator-confirmed)
def test_retract_head_auto_noop_when_disabled(db_sync):
s = db_sync.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
s.head_auto_apply_enabled = False
ver = _version(db_sync)
tag = _tag(db_sync, "glasses", TagKind.general)
_head(db_sync, tag.id, slot=0, threshold=0.7, version=ver)
lo = _img(db_sync, "e" * 64, _emb(5)) # would be below threshold
_apply(db_sync, lo.id, tag.id, "head_auto")
db_sync.commit()
assert retract_auto_applied_heads(db_sync) == 0
assert _has_tag(db_sync, lo.id, tag.id) # switch off → nothing retracted
def test_retract_ccip_auto(db_sync):
char = _tag(db_sync, "Raven", TagKind.character)
db_sync.add(CharacterPrototype(tag_id=char.id, ccip_embedding=_ccip(0)))
hi = _img(db_sync, "f" * 64) # figure matches prototype → keep
lo = _img(db_sync, "g" * 64) # figure orthogonal → retract
conf = _img(db_sync, "h" * 64) # orthogonal, CONFIRMED → keep
man = _img(db_sync, "i" * 64) # orthogonal, manual → keep
_figure(db_sync, hi.id, _ccip(0))
_figure(db_sync, lo.id, _ccip(5))
_figure(db_sync, conf.id, _ccip(5))
_figure(db_sync, man.id, _ccip(5))
_apply(db_sync, hi.id, char.id, "ccip_auto")
_apply(db_sync, lo.id, char.id, "ccip_auto")
_apply(db_sync, conf.id, char.id, "ccip_auto")
_apply(db_sync, man.id, char.id, "manual")
db_sync.add(TagPositiveConfirmation(image_record_id=conf.id, tag_id=char.id))
db_sync.commit()
assert retract_auto_applied_ccip(db_sync) == 1
assert not _has_tag(db_sync, lo.id, char.id) # retracted (below threshold)
assert _has_tag(db_sync, hi.id, char.id) # kept (match ≥ threshold)
assert _has_tag(db_sync, conf.id, char.id) # kept (operator-confirmed)
assert _has_tag(db_sync, man.id, char.id) # kept (manual, not auto)
+3
View File
@@ -15,6 +15,8 @@ def test_serialize_tag_with_enum_kind():
assert serialize_tag(row) == { assert serialize_tag(row) == {
"id": 1, "name": "Sasuke Uchiha", "kind": "character", "id": 1, "name": "Sasuke Uchiha", "kind": "character",
"fandom_id": 5, "fandom_name": "Naruto", "is_system": False, "fandom_id": 5, "fandom_name": "Naruto", "is_system": False,
# Applied-tag context defaults for rows without image scope (m139).
"source": None, "confirmed": False,
} }
@@ -27,6 +29,7 @@ def test_serialize_tag_with_string_kind_and_no_fandom():
assert serialize_tag(row) == { assert serialize_tag(row) == {
"id": 2, "name": "solo", "kind": "general", "id": 2, "name": "solo", "kind": "general",
"fandom_id": None, "fandom_name": None, "is_system": False, "fandom_id": None, "fandom_name": None, "is_system": False,
"source": None, "confirmed": False,
} }
+531
View File
@@ -0,0 +1,531 @@
"""translate_posts sweep (#143) — integration, with a stubbed Interpreter client
and the task's session factory pointed at the test's db_sync."""
import pytest
from sqlalchemy import select
from backend.app.models import Artist, ImportSettings, Post
from backend.app.services import interpreter_client as ic
from backend.app.tasks.translation import (
_RETRANSLATE_COUNTDOWN,
retranslate_posts,
translate_posts,
)
pytestmark = pytest.mark.integration
class _Ctx:
def __init__(self, s):
self.s = s
def __enter__(self):
return self.s
def __exit__(self, *a):
return False
def _sf(db_sync):
class _SM:
def __call__(self):
return _Ctx(db_sync)
return _SM()
def _artist(db, name="A", slug="a"):
a = Artist(name=name, slug=slug)
db.add(a)
db.flush()
return a
def _post(db, artist_id, *, title=None, desc=None, ext="p1"):
p = Post(
artist_id=artist_id, external_post_id=ext,
post_title=title, description=desc,
)
db.add(p)
db.flush()
return p
def _mark_translated(p, *, lang="ja", ver="v1"):
"""Simulate a prior (old-model) translation so retranslate has something to
reset."""
p.post_title_translated = "OLD"
p.description_translated = "OLD"
p.translated_source_lang = lang
p.translation_engine_version = ver
def _enable(db, url="http://i.lan"):
cfg = db.execute(
select(ImportSettings).where(ImportSettings.id == 1)
).scalar_one()
cfg.translation_enabled = True
cfg.interpreter_base_url = url
db.flush()
def _patch(monkeypatch, db_sync):
monkeypatch.setattr(
"backend.app.tasks.translation._sync_session_factory",
lambda: _sf(db_sync),
)
def test_translate_posts_stores_translation(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": "ja", "engine": "llm", "engine_version": "v1",
},
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", desc="かわいい")
_enable(db_sync)
db_sync.commit()
assert "translated=1" in translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:ねこ"
assert p.description_translated == "EN:かわいい"
assert p.translated_source_lang == "ja"
assert p.translation_engine_version == "v1"
def test_translate_posts_passthrough_english_marks_handled(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": list(texts), "detected_lang": "en",
"engine": "none", "engine_version": None,
},
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="hello world", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.translated_source_lang == "en" # marked handled...
assert p.post_title_translated is None # ...but nothing to show
def test_translate_posts_mixed_language_translates_nonenglish_field(db_sync, monkeypatch):
# English title + Japanese description: the description is still translated
# even though the title is already English (per-field detection, not the old
# aggregate first-item bail). Source lang comes from the translated field.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def fake_translate(texts, **k):
t = texts[0]
if t.isascii(): # already English → passthrough
return {"translations": [t], "detected_lang": "en",
"engine": "none", "engine_version": None}
return {"translations": [f"EN:{t}"], "detected_lang": "ja",
"engine": "llm", "engine_version": "v9"}
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", fake_translate)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="hello", desc="ねこ")
_enable(db_sync)
db_sync.commit()
assert "translated=1" in translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # English title untouched
assert p.description_translated == "EN:ねこ" # Japanese desc translated
assert p.translated_source_lang == "ja" # from the translated field
assert p.translation_engine_version == "v9"
def test_translate_posts_disabled_is_noop(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p3")
db_sync.commit() # translation_enabled defaults False
assert translate_posts() == "disabled"
db_sync.refresh(p)
assert p.translated_source_lang is None
def test_translate_posts_service_down_leaves_untranslated(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: False)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p4")
_enable(db_sync)
db_sync.commit()
assert translate_posts() == "interpreter unavailable"
db_sync.refresh(p)
assert p.translated_source_lang is None # will retry next run
def test_translate_posts_interrupt_reenqueues_after_backoff(db_sync, monkeypatch):
# A drain mid-sweep (health passed, translate 503s w/ Retry-After) re-enqueues
# the daily sweep after the backoff instead of waiting for tomorrow's beat.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining", retry_after=30)
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_enable(db_sync)
db_sync.commit()
assert "interrupted" in translate_posts()
assert len(calls) == 1
assert calls[0][1]["countdown"] == 30
def _mock_ok(monkeypatch):
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": "ja", "engine": "llm", "engine_version": "v1",
},
)
def test_translate_posts_drain_chases_tail(db_sync, monkeypatch):
# "Translate now" (drain=True): with the chunk capped at 1 and 2 untranslated
# posts, the first chunk re-enqueues itself — drain preserved — to finish the
# backlog rather than waiting for the next beat.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts(drain=True)
assert len(calls) == 1 # one re-enqueue for the tail
args, kw = calls[0]
assert args[1] == {"drain": True} # drain flag carried forward
assert kw["countdown"] == _RETRANSLATE_COUNTDOWN
def test_translate_posts_beat_single_chunk_no_tail_chase(db_sync, monkeypatch):
# The periodic beat (drain defaults False) does exactly ONE chunk and never
# re-enqueues — the 8h cadence drives the rest.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts() # drain=False
assert calls == [] # no self-re-enqueue
def _mock_lang(monkeypatch, *, lang, conf, ver="v1"):
# Interpreter is up and returns a translation with the given detected language
# + confidence, so the acceptance gate can be exercised end-to-end.
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": lang, "detected_confidence": conf,
"engine": "llm", "engine_version": ver,
},
)
def test_translate_posts_rejects_low_confidence_latin(db_sync, monkeypatch):
# A short English title Interpreter mis-labels as German with LOW confidence is
# NOT stored — the original is kept, and the post is marked handled (source ==
# target) so the sweep won't revisit it.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.42)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Nami Heroines - WIP Part 1", ext="p1")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # mis-flag rejected, kept
assert p.translated_source_lang == "en" # marked handled (== target)
def test_translate_posts_accepts_high_confidence_latin(db_sync, monkeypatch):
# A confident latin-script detection (real German) clears the floor → stored.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.98)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Das Mädchen mit dem Perlenohrring", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:Das Mädchen mit dem Perlenohrring"
assert p.translated_source_lang == "de"
def test_translate_posts_accepts_low_confidence_cjk(db_sync, monkeypatch):
# CJK is script-detected and trusted regardless of the reported number — a
# zh 0.75 (e.g. pure-kanji Japanese) is still translated, not rejected.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="zh", conf=0.75)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="猫が可愛い", ext="p3")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:猫が可愛い"
assert p.translated_source_lang == "zh"
def test_translate_posts_force_override_translates_below_floor(db_sync, monkeypatch):
# override='force' stores the translation even below the confidence floor —
# rescuing a legitimately-foreign title the gate would otherwise skip.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.42) # below the 0.90 default floor
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Kurz", ext="p1")
p.translation_override = "force"
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:Kurz" # forced through despite 0.42
assert p.translated_source_lang == "de"
def test_translate_posts_original_override_keeps_original(db_sync, monkeypatch):
# override='original' never translates — even a confident (0.98) detection is
# kept as the original. The sweep marks it handled (== target) and stores
# nothing; Interpreter isn't consulted for the decision.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.98) # would be accepted under auto
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Das Mädchen", ext="p1")
p.translation_override = "original"
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # kept original
assert p.translated_source_lang == "en" # marked handled
def _mock_new_model(monkeypatch, *, ver="v2"):
"""Interpreter is up and translates with a NEW engine version."""
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"NEW:{t}" for t in texts],
"detected_lang": "ja", "engine": "llm", "engine_version": ver,
},
)
def test_retranslate_resets_and_reruns(db_sync, monkeypatch):
# A post translated by the old model gets cleared + re-run through the new
# one (new engine_version proves the reset happened, not a cache short-circuit).
_patch(monkeypatch, db_sync)
_mock_new_model(monkeypatch)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", desc="かわいい")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
db_sync.refresh(p)
assert p.post_title_translated == "NEW:ねこ"
assert p.description_translated == "NEW:かわいい"
assert p.translation_engine_version == "v2"
def test_retranslate_leaves_keep_original_posts(db_sync, monkeypatch):
# A 'keep original' post survives a Re-translate-all: _reset_translations skips
# override='original', so it stays handled + untranslated instead of being
# re-run (which would re-introduce the mis-flag the operator killed).
_patch(monkeypatch, db_sync)
_mock_new_model(monkeypatch)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
p.translation_override = "original"
p.translated_source_lang = "en" # handled by a prior 'keep original' apply
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
db_sync.refresh(p)
assert p.translated_source_lang == "en" # not reset
assert p.post_title_translated is None # not re-translated
def test_retranslate_artist_scope_leaves_others(db_sync, monkeypatch):
# Scoping to one artist must not touch another artist's translations.
_patch(monkeypatch, db_sync)
_mock_new_model(monkeypatch)
a1 = _artist(db_sync)
a2 = _artist(db_sync, name="B", slug="b")
p1 = _post(db_sync, a1.id, title="ねこ", ext="p1")
p2 = _post(db_sync, a2.id, title="いぬ", ext="p2")
_mark_translated(p1)
_mark_translated(p2)
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a1.id])
db_sync.refresh(p1)
db_sync.refresh(p2)
assert p1.post_title_translated == "NEW:ねこ" # re-run
assert p1.translation_engine_version == "v2"
assert p2.post_title_translated == "OLD" # untouched
assert p2.translation_engine_version == "v1"
def test_retranslate_runs_until_done(db_sync, monkeypatch):
# With the chunk capped at 1 and 2 posts to redo, the first chunk re-enqueues
# itself (with _reset_done=True so it doesn't wipe the fresh work) to finish.
_patch(monkeypatch, db_sync)
_mock_new_model(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
p1 = _post(db_sync, a.id, title="ねこ", ext="p1")
p2 = _post(db_sync, a.id, title="いぬ", ext="p2")
_mark_translated(p1)
_mark_translated(p2)
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
assert len(calls) == 1 # one re-enqueue for the tail
args, _kw = calls[0]
assert args[1]["_reset_done"] is True
assert args[1]["artist_ids"] == [a.id]
def test_retranslate_disabled_does_not_reset(db_sync, monkeypatch):
# Never wipe translations we can't rebuild — a disabled service leaves them.
_patch(monkeypatch, db_sync)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
db_sync.commit() # translation_enabled defaults False
assert retranslate_posts(artist_ids=[a.id]) == "disabled"
db_sync.refresh(p)
assert p.translated_source_lang == "ja"
assert p.post_title_translated == "OLD"
def test_retranslate_unhealthy_does_not_reset(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: False)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
assert retranslate_posts(artist_ids=[a.id]) == "interpreter unavailable"
db_sync.refresh(p)
assert p.translated_source_lang == "ja"
assert p.post_title_translated == "OLD"
def test_retranslate_interrupt_resumes_after_retry_after(db_sync, monkeypatch):
# Interpreter drains mid-chunk (health passed, then translate 503s with a
# Retry-After): the bulk re-translate re-enqueues itself after the hinted
# backoff, with _reset_done=True so it never re-wipes fresh work — instead of
# stalling until the daily beat.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining", retry_after=42)
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
assert "interrupted" in retranslate_posts(artist_ids=[a.id])
assert len(calls) == 1 # resume re-enqueued
args, kw = calls[0]
assert args[1]["_reset_done"] is True
assert args[1]["artist_ids"] == [a.id]
assert kw["countdown"] == 42 # honoured the Retry-After
db_sync.refresh(p)
assert p.translated_source_lang is None # reset happened, not re-run
def test_retranslate_interrupt_default_backoff_without_hint(db_sync, monkeypatch):
# No Retry-After header → fall back to the default backoff (60s).
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining")
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
assert len(calls) == 1
assert calls[0][1]["countdown"] == 60
+48
View File
@@ -0,0 +1,48 @@
"""Translation acceptance gate (#1376, #155) — pure unit tests for ``_accept``,
which decides whether curator stores Interpreter's translation or keeps the
original. Curator does no detection of its own; it only thresholds Interpreter's
reported detected language + confidence against the operator-tunable floor
(Scribe rule 133). No DB, no network."""
from backend.app.tasks.translation import _DEFAULT_MIN_LATIN_CONFIDENCE, _accept
def test_accept_trusts_cjk_regardless_of_confidence_or_floor():
# ja/ko/zh are script-detected — accepted even at a low reported number and
# even against a high floor (pure-kanji Japanese can come back labelled zh
# ~0.75, still translated).
assert _accept("ja", None) is True
assert _accept("ja", 0.10, 0.99) is True
assert _accept("ko", 0.99) is True
assert _accept("zh", 0.75) is True
assert _accept("zh-CN", 0.20) is True # region suffix is still CJK
def test_accept_default_floor_is_strict():
# Default floor is 0.90 (#155): the ~0.86 band — where genuine German AND
# mis-flagged short English collide — is now REJECTED (the whole point of the
# stricter default). Genuine high-confidence non-English (~1.0) still passes.
assert _accept("de", 0.98) is True
assert _accept("de", 0.86) is False # the collision band, now cut
assert _accept("de", _DEFAULT_MIN_LATIN_CONFIDENCE) is True # at floor
assert _accept("de", _DEFAULT_MIN_LATIN_CONFIDENCE - 0.01) is False
assert _accept("fr", 0.40) is False # short mis-flag
def test_accept_honors_an_explicit_floor():
# The floor is a parameter now (fed from ImportSettings). A looser floor keeps
# the 0.86 band; a stricter one rejects even a fairly confident detection.
assert _accept("de", 0.86, 0.80) is True
assert _accept("de", 0.86, 0.95) is False
assert _accept("de", 0.96, 0.95) is True
def test_accept_is_case_insensitive():
assert _accept("JA", None) is True
assert _accept("DE", 0.40) is False
def test_accept_missing_confidence_fails_open():
# No confidence reported → don't silently drop a translation.
assert _accept("de", None) is True
assert _accept("", None) is True
assert _accept("de", None, 0.99) is True # fail-open beats a high floor