Compare commits

...

37 Commits

Author SHA1 Message Date
bvandeusen fac5ae6ce5 feat(explore): reach dial to escape dense clusters + anti-revisit (#1476)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m50s
The Explore walk got stuck in dense signatures — neighbours all too similar, so
forward-arrow couldn't escape and Random was the only exit. Root cause: MMR only
diversifies WITHIN the nearest ~400 pool; in a dense cluster that whole pool is
near-identical, so there's no escape route in it.

- gallery_service.similar(reach=0.0, exclude_ids=None): reach>0 widens the pool
  (cap 400→1000) and _reach_sample strides across an outward-growing distance span
  so the set handed to MMR spans near→mid-far (guaranteed escape routes), not just
  the tight cluster. exclude_ids drops already-walked images. Gallery 'more like
  this' (reach=0) is unchanged.
- api/gallery similar: parse reach + exclude_ids.
- explore store: default reach 0.4 (auto-diversifies without touching the dial),
  pass the breadcrumb as exclude_ids, setReach action.
- ExploreView: a Near↔Far reach slider in the trail.
- tests: _reach_sample math (deeper ranks with higher reach, near kept); similar
  exclude_ids drops walked + reach path runs clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 12:06:22 -04:00
bvandeusen af0d39ed52 feat(wip): soft title tier — sketch/doodle vocab + ring-loud audit (#1474)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m53s
Extends WIP title-tagging to lower-precision cues (sketch/doodle/scribble) safely.

- wip_title.py: soft matcher (word-anchored; sketchbook/kadoodle don't trip it);
  WIP_TITLE_SOFT_SOURCE + soft SQL prefilter; apply_wip_image_tags takes a source arg.
- training_data._AUTO_SOURCES += 'wip_title_soft' → the soft tier is PROVISIONAL and
  never trains the wip head (a finished "sketch" can't pollute it). Only the hard
  tier (wip_title) + manual train.
- ImportSettings.wip_soft_title_tagging_enabled (OFF by default, opt-in). Migration 0087.
- importer: hard tier wins, soft is the fallback (source wip_title_soft).
- backfill: refactored into a shared _backfill_wip_tier; hard always, soft when enabled.
- heads.soft_wip_conflict_audit + daily beat: score soft-tagged images against content
  heads, flag ring-loud ones (PresentationReview mode=process) for the review strip —
  the operator's "measure if they got falsely tagged" safety.
- api settings toggle; ImportFiltersForm soft toggle.
- tests: soft matcher pos/neg; soft source not a training positive; audit flags
  ring-loud + spares quiet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 10:14:38 -04:00
bvandeusen d9a14e890d feat(system-tags): process auto-tag settings UI + mode-aware review strip (#1464)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Successful in 3m44s
Frontend for the system-tag refactor (milestone #157 step 6).

- HeadsCard: new 'Auto-tag work-in-progress' section (enable + tag-confidence +
  conflict knobs) for wip/editor process auto-apply, mirroring the chrome card;
  copy notes they stay VISIBLE and the head only learns from titles/manual (no
  runaway). Presentation copy narrowed to banner-only.
- HiddenReviewStrip: mode-aware — chrome flags read 'hidden as X / Keep hidden /
  Un-hide'; process flags read 'auto-tagged X / Keep tag / Remove tag'. Same
  endpoints (the backend returns mode), different words.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 23:22:13 -04:00
bvandeusen ad2a5fc5fe feat(system-tags): process vs chrome groups + WIP provisional auto-apply (#1464)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 24s
CI / backend-lint-and-test (push) Successful in 39s
CI / integration (push) Successful in 3m48s
Backend for the system-tag behavior refactor (milestone #157). editor screenshot
moves from chrome (hidden) to the PROCESS group (shown, like wip); wip+editor gain
provisional auto-apply so they stop needing endless manual identification —
without a runaway loop.

- tag.py: split PRESENTATION_SYSTEM_TAGS → CHROME_SYSTEM_TAGS (banner) +
  PROCESS_SYSTEM_TAGS (wip, editor screenshot).
- heads.py: generalize presentation_auto_apply_sweep → system_tag_auto_apply_sweep
  (mode chrome|process). Same Guard 1 (skip human/confirmed) + Guard 2 (ring-loud
  conflict → PresentationReview). process mode uses source 'process_auto' and does
  NOT hide (hide is a gallery-query effect of group membership).
- training_data._AUTO_SOURCES += 'process_auto' → the head never trains on its own
  auto-applied output; only wip_title/manual train it (the runaway break).
- ml_settings: process_auto_apply_enabled (OFF, opt-in) + threshold + conflict
  threshold. presentation_review.mode ('chrome'|'process'). Migration 0086.
- gallery_service: default-hide reads CHROME only (editor now shows); Explore
  neighbors exclude the whole PROCESS group.
- tasks/ml + celery beat: scheduled_process_auto_apply (daily, opt-in); prune
  covers both modes.
- api: ml_admin process_* CRUD+validation; hidden-review returns mode.
- tests: rename chrome sweep calls; new test_process_auto_apply (apply, guards,
  mode flag, no-self-train); gallery test asserts editor now visible.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 23:15:59 -04:00
bvandeusen 0da0e47784 fix(wip-title): count inserts via pre-SELECT, not driver rowcount
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m51s
apply_wip_image_tags relied on result.rowcount, but psycopg reports -1 for a
multi-row INSERT ... ON CONFLICT DO NOTHING (executemany path), so the return
count (and the backfill's reported total) was wrong. Compute the count from a
pre-SELECT of already-tagged ids within the same transaction; keep ON CONFLICT
DO NOTHING as a race-safety belt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:22:25 -04:00
bvandeusen 503c8854bc style(importer): sort wip_title import into the local-import block
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Failing after 3m49s
Ruff I001 fixup for 5719387 — the new .wip_title import belongs after
.thumbnailer (alphabetical), not after .archive_extractor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:16:11 -04:00
bvandeusen 571938781a feat(tagging): title-based WIP auto-tagging (#1458)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Failing after 3m50s
Auto-apply the `wip` system tag to posts whose TITLE explicitly declares
work-in-progress ("WIP" / "work in progress") — a deterministic, high-precision
complement to the image-based ML `wip` head. WIP images are excluded from the
Explore/gallery browse, so honouring the artist's own label keeps unfinished
pieces out of the main browse.

- services/wip_title.py: precision-first token-anchored matcher (swipe/wiped
  never trip it) + sync apply helpers (source='wip_title', ON CONFLICT DO
  NOTHING, chunked under the psycopg param ceiling).
- importer: live hook on FRESH import only (never on deep-scan/supersede), so a
  manually-removed WIP tag is never re-applied by a routine re-scan.
- maintenance.backfill_wip_title_tags: operator-triggered back-catalogue sweep
  (coarse SQL prefilter + regex confirm, keyset-paginated). Deliberately NOT a
  beat — a periodic re-run would silently undo manual removals.
- ImportSettings.wip_title_tagging_enabled (default ON, migration 0085) gating
  the live hook; GET/PATCH + POST /settings/wip-title/scan.
- Settings UI: toggle + "Scan existing posts" button.
- Tests: pure matcher unit tests + integration (apply idempotency, backfill
  precision).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 12:12:19 -04:00
bvandeusen 0cf3a02797 Merge pull request 'chore(deps): update node.js to v24' (#220) from renovate/node-24.x into dev
CI / lint (push) Successful in 3s
extension / lint (push) Successful in 13s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 50s
CI / integration (push) Successful in 3m54s
2026-07-11 21:38:33 -04:00
Renovate Bot 6ab495292a chore(deps): update node.js to v24
renovate/stability-days Updates have met minimum release age requirement
CI / lint (pull_request) Successful in 2s
CI / frontend-build (pull_request) Successful in 16s
CI / backend-lint-and-test (pull_request) Successful in 28s
CI / integration (pull_request) Successful in 3m49s
2026-07-12 01:33:07 +00:00
bvandeusen c98db303d0 chore(frontend): drop dead vue-tsc devDep + check script
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 48s
CI / integration (push) Successful in 3m58s
Frontend is pure JS (no .ts/JSDoc); CI never ran vue-tsc. Removed the devDep and its orphaned `check` script instead of bumping to v3, and updated the ci.yml comment to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 21:31:36 -04:00
bvandeusen 2d0fca8729 Merge pull request 'chore(deps): update frontend build/test toolchain (major)' (#219) from renovate/major-frontend-buildtest-toolchain into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 43s
CI / integration (push) Successful in 3m50s
2026-07-11 21:30:29 -04:00
Renovate Bot a2858892e9 chore(deps): update frontend build/test toolchain
renovate/stability-days Updates have met minimum release age requirement
CI / lint (pull_request) Successful in 5s
CI / frontend-build (pull_request) Successful in 26s
CI / backend-lint-and-test (pull_request) Successful in 1m5s
CI / integration (pull_request) Successful in 3m44s
2026-07-12 01:24:54 +00:00
bvandeusen 7f5e0603de ci(renovate): pre-merge CI for renovate PRs + group FE toolchain
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 49s
CI / integration (push) Successful in 3m42s
ci.yml: add pull_request trigger (base dev) so renovate/* PRs get validated before merge; dev→main flow unchanged (base main), no duplicate runs. renovate.json: group vite/vitest/plugin-vue/happy-dom/@vue-test-utils into one PR (version-coupled).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 21:21:17 -04:00
bvandeusen 49f6765326 Merge pull request 'chore(deps): update dependency structlog to v26' (#218) from renovate/structlog-26.x into dev
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 27s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m45s
2026-07-11 20:00:09 -04:00
bvandeusen b23b19bf58 Merge pull request 'chore(deps): update dependency gdown to v6' (#217) from renovate/gdown-6.x into dev
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m46s
2026-07-11 20:00:06 -04:00
bvandeusen c17a6e6c40 Merge pull request 'chore(deps): update docker/dockerfile docker tag to v1.25' (#216) from renovate/docker-dockerfile-1.x into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 44s
CI / integration (push) Successful in 3m58s
2026-07-11 20:00:02 -04:00
bvandeusen 64053a5f57 Merge pull request 'chore(deps): update dependency pgvector to >=0.5,<0.6' (#215) from renovate/pgvector-0.x into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 49s
CI / integration (push) Successful in 3m56s
2026-07-11 19:59:58 -04:00
Renovate Bot b6c5638eab chore(deps): update dependency structlog to v26
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:58:56 +00:00
Renovate Bot 466ee898ab chore(deps): update dependency gdown to v6
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:58:55 +00:00
Renovate Bot e00deee451 chore(deps): update docker/dockerfile docker tag to v1.25
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:58:54 +00:00
Renovate Bot 87d3198f89 chore(deps): update dependency pgvector to >=0.5,<0.6
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:58:52 +00:00
bvandeusen 5010de6178 Merge pull request 'chore(deps): update dependency nh3 to >=0.3,<0.4' (#213) from renovate/nh3-0.x into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 41s
CI / integration (push) Successful in 3m47s
2026-07-11 19:53:11 -04:00
bvandeusen 3063af6a74 Merge pull request 'chore(deps): update dependency cryptography to v49' (#214) from renovate/cryptography-49.x into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 47s
CI / integration (push) Successful in 3m53s
2026-07-11 19:53:07 -04:00
Renovate Bot ba2de60439 chore(deps): update dependency cryptography to v49
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:50:38 +00:00
Renovate Bot ff9846ce64 chore(deps): update dependency nh3 to >=0.3,<0.4
renovate/stability-days Updates have met minimum release age requirement
2026-07-11 23:50:37 +00:00
bvandeusen 279b0560d9 Merge pull request 'chore: Configure Renovate' (#211) from renovate/configure into dev
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m55s
2026-07-11 18:45:52 -04:00
Renovate Bot af09bf58af Add renovate.json 2026-07-11 18:34:27 +00:00
bvandeusen aea2701c28 feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 27s
CI / integration (push) Successful in 3m52s
The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh ==
cached, verified by probe) confidently mis-detects short ASCII English like
"... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a
re-translate reproduced it. Confidence alone can't separate the 0.86 collision
(genuine German lands there too), and single-word mis-flags sit at a confident
1.0 no floor catches.

Two operator-approved levers:

- Acceptance floor is now a live Settings value (ImportSettings.
  translation_min_confidence, default 0.90; surfaced in the Translation card), so
  it's tunable without a redeploy. _accept takes the threshold as a parameter.

- Per-post sticky override (Post.translation_override: auto/force/original).
  'force' stores a translation even below the floor (rescue a skipped
  legit-foreign title); 'original' keeps the original and clears any stored
  translation (kill a confident mis-flag no floor catches). The sweep honors it
  on every run and _reset_translations skips 'original', so the choice survives a
  Re-translate-all. POST /api/posts/<id>/translation-override applies it
  immediately (translate now when the service is up, else queue for the sweep).
  UI: PostTranslationControl on the posts-feed card.

Migration 0084 (both columns + a CHECK on the override). The feed + provenance
serializers expose translation_override.

With a stricter floor the rollback finally works: raise it -> Re-translate all ->
the 0.86 mis-flags are rejected and restored to the original; force /
keep-original handle the residual either way.

Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor
cases); sweep force/original + re-translate-skips-original; override endpoint
(validation, original clears, force queues when disabled, feed exposes it);
settings min_confidence default/save/validate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 22:38:25 -04:00
bvandeusen a444cf82d1 refactor(tags): unify suggestion source — one canonical DB-tag dropdown, drop dead raw/alias machinery (#154)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 36s
CI / integration (push) Successful in 3m45s
Every tag suggestion is a canonical DB tag now (tagging-v2 #114: heads + CCIP
score EXISTING concept tags). The pre-heads apparatus for model-predicted tags
that didn't exist in the DB — creates_new_tag / raw_name / via_alias, the
/suggestions/alias endpoint + add_alias_and_accept, AliasPickerDialog, and the
store's aliasAccept/removeAlias — was dead and is removed.

The type-to-add dropdown was TWO row sources (server autocomplete + the image's
ML suggestions) merged with a dedup that dropped the %-bearing suggestion row
when the debounced server hit landed — the operator's "confidence % flickers
then vanishes". Now it's ONE list of DB-tag matches, each annotated with the
model's confidence (join by canonical_tag_id) when the tag was scored for this
image. No dedup, no flicker; picking a suggested tag still records acceptance
via TagPanel.findPending.

Single per-image fetch: score_image now reports above_threshold per row
(computed vs the head's own suggest cut, separate from the inclusion floor), so
the rail makes ONE min=0 request and derives the panel (above_threshold) and the
dropdown (all, text-filtered) client-side — the two /suggestions calls collapse
to one. Manual "Create 'X' as <kind>" (novel typed names) is unchanged; the
alias table + tag-side alias admin + auto-apply alias matching are untouched.

Tests: gate/serializer assertions updated (above_threshold; dropped dead-field
+ alias-endpoint checks); frontend spec seeds via the single load and covers the
byCategory/aboveByCategory split.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-10 15:22:51 -04:00
bvandeusen 6ab7fd5c7f tune(translation): set latin acceptance floor to 0.80 (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 31s
CI / integration (push) Successful in 3m42s
Calibrated against fresh probes once Interpreter returned real langdetect
confidence: genuine German detected at 1.0, a correctly-detected but ambiguous
latin string at 0.86. Set _MIN_LATIN_CONFIDENCE to 0.80 (below that band) so
legitimate ambiguous non-English still translates while genuinely-unsure guesses
are rejected. Real langdetect also fixed the original mis-flag at the source, so
this floor is a safety net, not the primary fix. Pin 0.86-accepted in the gate
test to guard against bumping the floor back up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-09 16:40:02 -04:00
bvandeusen 7ddf94220d feat(translation): accept/reject gate on Interpreter's detection confidence (#1376)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m46s
Interpreter now returns a real per-detection confidence (source stays "auto"),
so curator can reject the mis-detections it was blindly storing — e.g. a short
English title mis-labelled as German and rewritten into the archive.

The gate consumes ONLY Interpreter's own reported detection — curator does no
language detection of its own (Scribe rule 133): a field is stored when the
engine actually translated it AND either the detected language is CJK
(script-detected, reliably high — ja/ko/zh trusted outright, incl. pure-kanji
Japanese that lands as zh ~0.75) or the reported confidence clears a
latin-script floor (_MIN_LATIN_CONFIDENCE = 0.90). A latin detection below the
floor keeps the original and marks the post handled; a missing confidence fails
open. The client already sent source="auto" and parsed confidence, so this is
purely the gate + tests.

Tests: pinned interpreter-client test now asserts source stays "auto"; new
pure-unit gate tests (CJK trusted / latin floor / case-insensitive / fail-open)
in the fast lane; end-to-end reject-low-latin, accept-high-latin,
accept-low-cjk sweeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-09 16:17:56 -04:00
bvandeusen ffdbdbaf07 feat(translation): 8h sweep + drain button + detection probe (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 46s
CI / integration (push) Successful in 3m55s
Throughput: translate_posts now runs every 8h (was daily) as the
steady-state cadence for newly-imported posts, and the Settings
"Translate now" button runs it in drain mode (run-until-done, no reset)
so one press clears the whole untranslated backlog instead of a single
300-post chunk. The interrupt/backoff re-enqueue now preserves the drain
flag so a bulk drain resumes cleanly after an Interpreter restart.

Misdetection groundwork: surface the detector's confidence from the
Interpreter client (it was in the detectedLanguage payload but discarded)
and add a read-only "Test translation" box — POST /settings/translation/
probe + TranslationCard UI — that shows detected language + confidence +
engine + result for pasted text, without saving. Lets the operator see
why a short/abbreviation-heavy English title gets mis-detected so the
detection guard (min-length + confidence floor) can be tuned from real
numbers. The guard itself follows once the mis-detected cases are probed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-08 20:13:49 -04:00
bvandeusen a017771621 feat(ui): thin kind-coloured border on tag chips so they don't wash out
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m49s
The tonal fill (esp. character = info) is intentionally faint and blends into
the dark tag rail. Add a thin border in each chip's own kind colour via
color-mix on currentColor (the tonal chip's themed foreground), defining the
edge without changing the fill. Theme-aware in both light and dark.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:31:44 -04:00
bvandeusen 25e555cab6 fix(ui): stop leading tag-chip icon clipping on the left edge
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m50s
The larger size=default chip widened Vuetify's negative start-margin on the
leading kind-icon, placing it left of the .v-chip__content box whose
overflow:hidden (the name-truncation guard) then clipped its left edge.
Zero the icon's negative inline-start margin so it sits inside the clip box;
the chip's 12px padding keeps a comfortable left inset.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:29:28 -04:00
bvandeusen 3c7ab44e74 feat(translation): per-field language detection for mixed-language posts
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m43s
_translate_one translated [title, description] in ONE Interpreter call and keyed
the whole-post passthrough on the aggregate detected_lang (the FIRST item). So an
English title + non-English description detected "en" and marked the post handled,
leaving the description untranslated. Now each field is translated independently
(its own detected_lang / passthrough) and the non-target field is stored on its
own; translated_source_lang reflects the translated field's language.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:31:04 -04:00
bvandeusen 06f98acf3e feat(tagging): bolder auto-tag accept/reject + larger tag chips in the rail
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m45s
The in-pill ✓/✗ on unconfirmed auto-tags read muted — a faint colored icon on a
tonal chip (worst on character tags, kind=info) that only lit up on hover. Make
them solid green/red circles with a white glyph (22px, icon 15), mirroring the
Suggestions rail's verdict buttons so accept/reject read identically. Also bump
the applied-tag chips from size=small to default and the leading kind icon to
match — bigger, clearer tags throughout the rail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:17:18 -04:00
bvandeusen 4371ddb7e7 feat(translation): live re-translate progress in the Settings card
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m45s
translation/status now reports `active` (a translate/retranslate sweep is
running, from the TaskRun table) and `last_run` (the most recent finished run's
task + status). The Settings card polls live while a sweep runs, showing a
spinner + "Translating… N remaining" that ticks down, and flags a last run that
ended in error/timeout. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:20:32 -04:00
68 changed files with 2839 additions and 741 deletions
+13 -8
View File
@@ -9,10 +9,15 @@ name: CI
on: on:
push: push:
branches: [dev, main] branches: [dev, main]
# pull_request trigger intentionally absent — with branches: [dev, main] # Renovate opens PRs from `renovate/*` branches into `dev`. Those branches
# above, every PR commit already fires CI via the push event on dev. Adding # never push to dev/main, so the push trigger above gives them NO pre-merge
# pull_request would duplicate runs on dev→main PRs. FC has no fork PRs # CI — a bump could only be validated after it was already merged. This
# (single-operator Forgejo repo) so push coverage is complete. # pull_request trigger (base `dev` only) validates Renovate PRs before merge.
# It deliberately does NOT fire on dev→main PRs (base `main`), which still
# rely on the dev push run — so no duplicate runs. FC has no fork PRs
# (single-operator Forgejo repo), so secrets-on-PR is not a concern.
pull_request:
branches: [dev]
jobs: jobs:
# Fast-fail lint lane. ruff is pre-installed in the ci-python image, so # Fast-fail lint lane. ruff is pre-installed in the ci-python image, so
@@ -92,10 +97,10 @@ jobs:
# If we want strict lockfile-based reproducibility later, commit a # If we want strict lockfile-based reproducibility later, commit a
# package-lock.json and flip this back to `npm ci`. # package-lock.json and flip this back to `npm ci`.
- run: npm install --no-audit --no-fund - run: npm install --no-audit --no-fund
# `npm run check` (vue-tsc --noEmit) skipped: the frontend is pure JS # No type-check step: the frontend is pure JS (no .ts files, no JSDoc),
# with no .ts files and no JSDoc annotations, so vue-tsc has nothing # so a type-checker has nothing to do. The vue-tsc devDep + its `check`
# to type-check. Re-enable once we add a tsconfig.json and either # script were dropped 2026-07-11 rather than bumped to v3. If we add
# convert to TS or add JSDoc. # TS/JSDoc later, re-add a tsconfig.json + vue-tsc + a type-check step.
- run: npm run test:unit - run: npm run test:unit
- run: npm run build - run: npm run build
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
lint: lint:
runs-on: python-ci runs-on: python-ci
container: container:
image: node:22-bookworm-slim image: node:24-bookworm-slim
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Install web-ext - name: Install web-ext
+2 -2
View File
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.7 # syntax=docker/dockerfile:1.25
FROM node:22-alpine AS frontend-builder FROM node:24-alpine AS frontend-builder
WORKDIR /build WORKDIR /build
COPY frontend/package.json frontend/package-lock.json* ./ COPY frontend/package.json frontend/package-lock.json* ./
# No package-lock.json is tracked yet (we don't run npm locally per # No package-lock.json is tracked yet (we don't run npm locally per
+1 -1
View File
@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.7 # syntax=docker/dockerfile:1.25
FROM python:3.14-slim FROM python:3.14-slim
ENV PYTHONUNBUFFERED=1 \ ENV PYTHONUNBUFFERED=1 \
@@ -0,0 +1,51 @@
"""translation strictness setting + per-post translation override (milestone 155)
ImportSettings gains ``translation_min_confidence`` (the latin-script acceptance
floor, now operator-tunable in the UI; default 0.9 — stricter than the old
hardcoded 0.8, since Interpreter confidently mis-detects short ASCII English at
~0.86). Post gains ``translation_override`` — a sticky per-post choice of
auto / force / original so the operator can force a skipped translation on, or
knock a wrongly-translated one back to the original, and have it survive a
Re-translate-all.
Revision ID: 0084
Revises: 0083
Create Date: 2026-07-10
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0084"
down_revision: Union[str, None] = "0083"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"translation_min_confidence", sa.Float(), nullable=False,
server_default=sa.text("0.9"),
),
)
op.add_column(
"post",
sa.Column(
"translation_override", sa.String(16), nullable=False,
server_default="auto",
),
)
op.create_check_constraint(
"ck_post_translation_override",
"post",
"translation_override IN ('auto', 'force', 'original')",
)
def downgrade() -> None:
op.drop_constraint("ck_post_translation_override", "post", type_="check")
op.drop_column("post", "translation_override")
op.drop_column("import_settings", "translation_min_confidence")
@@ -0,0 +1,35 @@
"""title-based WIP auto-tagging (task #1458) — ImportSettings toggle
ImportSettings gains wip_title_tagging_enabled (ON by default): when a freshly
imported post's title explicitly declares work-in-progress ("WIP" / "work in
progress"), the importer applies the `wip` system tag to its images. No new
table — the tag itself is the seeded `wip` system tag (migration 0075) and the
application reuses image_tag with source='wip_title'.
Revision ID: 0085
Revises: 0084
Create Date: 2026-07-12
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0085"
down_revision: Union[str, None] = "0084"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"wip_title_tagging_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("true"),
),
)
def downgrade() -> None:
op.drop_column("import_settings", "wip_title_tagging_enabled")
@@ -0,0 +1,61 @@
"""process auto-apply settings + review mode (#1464) — system-tag refactor
The system-tag behavior refactor gives `wip` / `editor screenshot` (the PROCESS
group) their own provisional auto-apply, parallel to the presentation (chrome)
sweep. MLSettings gains three knobs: enabled (OFF by default — a new whole-library
auto-tagger is opt-in), the flat apply threshold, and the ring-loud conflict
threshold. presentation_review gains a `mode` column so one review surface serves
both chrome and process flags (existing rows backfill 'chrome'). server_defaults
so the existing rows fill cleanly.
Revision ID: 0086
Revises: 0085
Create Date: 2026-07-13
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0086"
down_revision: Union[str, None] = "0085"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"ml_settings",
sa.Column(
"process_auto_apply_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
op.add_column(
"ml_settings",
sa.Column(
"process_auto_apply_threshold", sa.Float(), nullable=False,
server_default="0.90",
),
)
op.add_column(
"ml_settings",
sa.Column(
"process_conflict_threshold", sa.Float(), nullable=False,
server_default="0.50",
),
)
op.add_column(
"presentation_review",
sa.Column(
"mode", sa.String(16), nullable=False,
server_default="chrome",
),
)
def downgrade() -> None:
op.drop_column("presentation_review", "mode")
op.drop_column("ml_settings", "process_conflict_threshold")
op.drop_column("ml_settings", "process_auto_apply_threshold")
op.drop_column("ml_settings", "process_auto_apply_enabled")
@@ -0,0 +1,33 @@
"""soft WIP title tier toggle (#1474) — ImportSettings.wip_soft_title_tagging_enabled
The soft tier also tags sketch/doodle/scribble titles, but with a provisional source
that never trains the head. OFF by default (a lower-precision tier is opt-in).
server_default so the existing singleton row (id=1) fills cleanly.
Revision ID: 0087
Revises: 0086
Create Date: 2026-07-13
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "0087"
down_revision: Union[str, None] = "0086"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column(
"import_settings",
sa.Column(
"wip_soft_title_tagging_enabled", sa.Boolean(), nullable=False,
server_default=sa.text("false"),
),
)
def downgrade() -> None:
op.drop_column("import_settings", "wip_soft_title_tagging_enabled")
+19 -3
View File
@@ -148,6 +148,17 @@ async def similar():
# Explore passes exclude_wip=1 to also drop work-in-progress from the # Explore passes exclude_wip=1 to also drop work-in-progress from the
# rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274). # rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274).
exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True") exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True")
# Explore reach (#1476): 0 = nearest (gallery default), →1 reaches into farther
# distance bands so the walk can escape a dense cluster. exclude_ids = the
# breadcrumb, so already-walked images aren't re-served as neighbours.
try:
reach = max(0.0, min(1.0, float(request.args.get("reach", "0"))))
except ValueError:
reach = 0.0
exclude_ids = [
int(x) for x in request.args.get("exclude_ids", "").split(",")
if x.strip().isdigit()
] or None
# post_id is the exclusive post-detail view — not a similarity scope. # post_id is the exclusive post-detail view — not a similarity scope.
# include_hidden is a gallery-browse flag; similar() has its OWN presentation # include_hidden is a gallery-browse flag; similar() has its OWN presentation
# exclusion (a similarity-quality concern, #1274), so drop it here (#141). # exclusion (a similarity-quality concern, #1274), so drop it here (#141).
@@ -158,7 +169,8 @@ async def similar():
svc = GalleryService(session) svc = GalleryService(session)
try: try:
images = await svc.similar( images = await svc.similar(
image_id=similar_to, limit=limit, exclude_wip=exclude_wip, **scope) image_id=similar_to, limit=limit, exclude_wip=exclude_wip,
reach=reach, exclude_ids=exclude_ids, **scope)
except ValueError as exc: except ValueError as exc:
return jsonify({"error": str(exc)}), 400 return jsonify({"error": str(exc)}), 400
if images is None: if images is None:
@@ -236,8 +248,10 @@ async def jump():
# content", surfaced in the gallery's Show-hidden review strip. ----------- # content", surfaced in the gallery's Show-hidden review strip. -----------
@gallery_bp.route("/hidden-review", methods=["GET"]) @gallery_bp.route("/hidden-review", methods=["GET"])
async def hidden_review(): async def hidden_review():
"""Unresolved presentation auto-hide flags, most-concerning first (highest """Unresolved system-tag auto-apply review flags (chrome + process, #1464),
content score) — for the gallery's Hidden-view review strip.""" most-concerning first (highest content score) — for the review strip. `mode`
tells the client whether the flagged tag hid the image ('chrome') or left it
visible ('process'), which decides the resolve labels (un-hide vs remove-tag)."""
ptag = aliased(Tag) ptag = aliased(Tag)
ctag = aliased(Tag) ctag = aliased(Tag)
async with get_session() as session: async with get_session() as session:
@@ -247,6 +261,7 @@ async def hidden_review():
PresentationReview.tag_id, PresentationReview.tag_id,
PresentationReview.conflict_tag_id, PresentationReview.conflict_tag_id,
PresentationReview.conflict_score, PresentationReview.conflict_score,
PresentationReview.mode,
ImageRecord.path, ImageRecord.thumbnail_path, ImageRecord.path, ImageRecord.thumbnail_path,
ImageRecord.sha256, ImageRecord.mime, ImageRecord.sha256, ImageRecord.mime,
ptag.name.label("tag_name"), ptag.name.label("tag_name"),
@@ -266,6 +281,7 @@ async def hidden_review():
"conflict_tag_id": r.conflict_tag_id, "conflict_tag_id": r.conflict_tag_id,
"conflict_name": r.conflict_name, "conflict_name": r.conflict_name,
"conflict_score": r.conflict_score, "conflict_score": r.conflict_score,
"mode": r.mode,
"thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime), "thumbnail_url": thumbnail_url(r.thumbnail_path, r.sha256, r.mime),
"image_url": image_url(r.path), "image_url": image_url(r.path),
} }
+12
View File
@@ -42,6 +42,9 @@ _EDITABLE = (
"presentation_auto_apply_enabled", "presentation_auto_apply_enabled",
"presentation_auto_apply_threshold", "presentation_auto_apply_threshold",
"presentation_conflict_threshold", "presentation_conflict_threshold",
"process_auto_apply_enabled",
"process_auto_apply_threshold",
"process_conflict_threshold",
"embedder_model_name", "embedder_model_name",
"embedder_model_version", "embedder_model_version",
*_DETECTOR_FIELDS, *_DETECTOR_FIELDS,
@@ -102,6 +105,9 @@ async def get_settings():
"presentation_auto_apply_enabled": s.presentation_auto_apply_enabled, "presentation_auto_apply_enabled": s.presentation_auto_apply_enabled,
"presentation_auto_apply_threshold": s.presentation_auto_apply_threshold, "presentation_auto_apply_threshold": s.presentation_auto_apply_threshold,
"presentation_conflict_threshold": s.presentation_conflict_threshold, "presentation_conflict_threshold": s.presentation_conflict_threshold,
"process_auto_apply_enabled": s.process_auto_apply_enabled,
"process_auto_apply_threshold": s.process_auto_apply_threshold,
"process_conflict_threshold": s.process_conflict_threshold,
"embedder_model_name": s.embedder_model_name, "embedder_model_name": s.embedder_model_name,
**{f: getattr(s, f) for f in _DETECTOR_FIELDS}, **{f: getattr(s, f) for f in _DETECTOR_FIELDS},
} }
@@ -162,6 +168,12 @@ def _validate(p: dict) -> str | None:
return "presentation_auto_apply_threshold must be between 0.5 and 0.999" return "presentation_auto_apply_threshold must be between 0.5 and 0.999"
if not (0.0 <= float(p["presentation_conflict_threshold"]) <= 1.0): if not (0.0 <= float(p["presentation_conflict_threshold"]) <= 1.0):
return "presentation_conflict_threshold must be between 0 and 1" return "presentation_conflict_threshold must be between 0 and 1"
# Process auto-apply (#1464). wip/editor stay VISIBLE so a false apply is
# low-harm (excludes-from-training + a review flag), but keep the same bar.
if not (0.5 <= float(p["process_auto_apply_threshold"]) <= 0.999):
return "process_auto_apply_threshold must be between 0.5 and 0.999"
if not (0.0 <= float(p["process_conflict_threshold"]) <= 1.0):
return "process_conflict_threshold must be between 0 and 1"
# Embedder model swap (#1190): both must be non-empty. Changing them means a # Embedder model swap (#1190): both must be non-empty. Changing them means a
# different embedding space — the operator must re-embed + retrain after. # different embedding space — the operator must re-embed + retrain after.
for key in ("embedder_model_name", "embedder_model_version"): for key in ("embedder_model_name", "embedder_model_version"):
+83
View File
@@ -3,12 +3,28 @@
from quart import Blueprint, jsonify, request from quart import Blueprint, jsonify, request
from ..extensions import get_session from ..extensions import get_session
from ..models import ImportSettings, Post
from ..services import interpreter_client as ic
from ..services.post_feed_service import PostFeedService from ..services.post_feed_service import PostFeedService
from ..services.source_service import KNOWN_PLATFORMS from ..services.source_service import KNOWN_PLATFORMS
from ..utils.text import html_to_plain
from ._responses import error_response as _bad from ._responses import error_response as _bad
posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts") posts_bp = Blueprint("posts", __name__, url_prefix="/api/posts")
_TRANSLATION_OVERRIDES = ("auto", "force", "original")
def _queue_for_sweep(post: Post) -> None:
"""Mark a post untranslated (all translation columns NULL) so the periodic
sweep re-runs it under its new override — used when Interpreter is down and we
can't translate inline."""
post.post_title_translated = None
post.description_translated = None
post.translated_source_lang = None
post.translation_engine_version = None
post.translated_at = None
@posts_bp.route("", methods=["GET"]) @posts_bp.route("", methods=["GET"])
async def list_posts(): async def list_posts():
@@ -82,3 +98,70 @@ async def get_post(post_id: int):
if item is None: if item is None:
return _bad("not_found", status=404, detail=f"post id={post_id}") return _bad("not_found", status=404, detail=f"post id={post_id}")
return jsonify(item) return jsonify(item)
@posts_bp.route("/<int:post_id>/translation-override", methods=["POST"])
async def set_translation_override(post_id: int):
"""Sticky per-post translation override (milestone 155). Body:
``{"override": "auto" | "force" | "original"}``.
'original' keeps the original (clears any stored translation now — no
Interpreter needed). 'force'/'auto' translate the post immediately if the
service is up (force bypasses the acceptance floor; auto re-runs the gate);
if it's down we save the flag and mark the post untranslated so the next sweep
applies it. The override persists, so the sweep + Re-translate-all keep
honoring it. Returns the updated translation fields + an ``applied`` status."""
body = await request.get_json(silent=True) or {}
override = body.get("override")
if override not in _TRANSLATION_OVERRIDES:
return _bad(
"invalid_override",
detail=f"override must be one of {list(_TRANSLATION_OVERRIDES)}",
)
# Lazy import (mirrors settings.py) so the API module doesn't pull the celery
# task graph at import time.
from ..tasks.translation import _store_translation, _translate_field
async with get_session() as session:
post = await session.get(Post, post_id)
if post is None:
return _bad("not_found", status=404, detail=f"post id={post_id}")
post.translation_override = override
cfg = await ImportSettings.load(session)
target = (cfg.translation_target_lang or "en").strip() or "en"
if override == "original":
_store_translation(post, (None, None, None), (None, None, None), target)
applied = "cleared"
else:
base_url = (cfg.interpreter_base_url or "").strip()
if cfg.translation_enabled and base_url and ic.health(base_url):
force = override == "force"
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
mc = cfg.translation_min_confidence
try:
title_res = _translate_field(title, base_url, target, mc, force=force)
desc_res = _translate_field(desc, base_url, target, mc, force=force)
except ic.InterpreterUnavailable:
_queue_for_sweep(post)
applied = "queued"
else:
_store_translation(post, title_res, desc_res, target)
applied = "translated"
else:
# Disabled / no URL / unhealthy → let the sweep apply it later.
_queue_for_sweep(post)
applied = "queued"
await session.commit()
return jsonify({
"id": post.id,
"translation_override": post.translation_override,
"post_title_translated": post.post_title_translated,
"description_translated": post.description_translated,
"translated_source_lang": post.translated_source_lang,
"applied": applied,
})
+106 -2
View File
@@ -16,6 +16,7 @@ from ..models import (
ImportTask, ImportTask,
Post, Post,
Tag, Tag,
TaskRun,
) )
from ..services import interpreter_client as ic from ..services import interpreter_client as ic
@@ -46,6 +47,9 @@ _EDITABLE_FIELDS = (
"translation_enabled", "translation_enabled",
"interpreter_base_url", "interpreter_base_url",
"translation_target_lang", "translation_target_lang",
"translation_min_confidence",
"wip_title_tagging_enabled",
"wip_soft_title_tagging_enabled",
) )
# Per-host external-download toggles — all plain booleans, validated uniformly. # Per-host external-download toggles — all plain booleans, validated uniformly.
@@ -86,6 +90,9 @@ async def get_import_settings():
"translation_enabled": row.translation_enabled, "translation_enabled": row.translation_enabled,
"interpreter_base_url": row.interpreter_base_url, "interpreter_base_url": row.interpreter_base_url,
"translation_target_lang": row.translation_target_lang, "translation_target_lang": row.translation_target_lang,
"translation_min_confidence": row.translation_min_confidence,
"wip_title_tagging_enabled": row.wip_title_tagging_enabled,
"wip_soft_title_tagging_enabled": row.wip_soft_title_tagging_enabled,
}) })
@@ -154,12 +161,32 @@ async def update_import_settings():
for key in ("interpreter_base_url", "translation_target_lang"): for key in ("interpreter_base_url", "translation_target_lang"):
if key in body and not isinstance(body[key], str): if key in body and not isinstance(body[key], str):
return jsonify({"error": f"{key} must be a string"}), 400 return jsonify({"error": f"{key} must be a string"}), 400
# Acceptance floor (milestone 155): latin-script translations below this
# Interpreter confidence are kept as the original.
if "translation_min_confidence" in body:
v = body["translation_min_confidence"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
return jsonify(
{"error": "translation_min_confidence must be a number in [0, 1]"}
), 400
if "series_suggest_threshold" in body: if "series_suggest_threshold" in body:
v = body["series_suggest_threshold"] v = body["series_suggest_threshold"]
if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1: if not isinstance(v, (int, float)) or isinstance(v, bool) or v < 0 or v > 1:
return jsonify( return jsonify(
{"error": "series_suggest_threshold must be a number in [0, 1]"} {"error": "series_suggest_threshold must be a number in [0, 1]"}
), 400 ), 400
if "wip_title_tagging_enabled" in body and not isinstance(
body["wip_title_tagging_enabled"], bool
):
return jsonify(
{"error": "wip_title_tagging_enabled must be a boolean"}
), 400
if "wip_soft_title_tagging_enabled" in body and not isinstance(
body["wip_soft_title_tagging_enabled"], bool
):
return jsonify(
{"error": "wip_soft_title_tagging_enabled must be a boolean"}
), 400
async with get_session() as session: async with get_session() as session:
row = await ImportSettings.load(session) row = await ImportSettings.load(session)
@@ -171,6 +198,18 @@ async def update_import_settings():
return await get_import_settings() return await get_import_settings()
@settings_bp.route("/settings/wip-title/scan", methods=["POST"])
async def wip_title_scan():
"""Enqueue the back-catalogue WIP-title scan (task #1458 Settings button):
apply the `wip` system tag to EXISTING posts whose title declares
work-in-progress. New imports are tagged live by the importer; this catches
the existing library. Returns the Celery task id (202)."""
from ..tasks.maintenance import backfill_wip_title_tags
r = backfill_wip_title_tags.delay()
return jsonify({"celery_task_id": r.id}), 202
@settings_bp.route("/system/stats", methods=["GET"]) @settings_bp.route("/system/stats", methods=["GET"])
async def system_stats(): async def system_stats():
async with get_session() as session: async with get_session() as session:
@@ -306,6 +345,10 @@ async def translation_status():
"""For the Settings card: is it on, is a URL set, is the service reachable, """For the Settings card: is it on, is a URL set, is the service reachable,
and how many posts still await translation. Health runs the sync client in a and how many posts still await translation. Health runs the sync client in a
thread so the event loop isn't blocked.""" thread so the event loop isn't blocked."""
translation_tasks = (
"backend.app.tasks.translation.translate_posts",
"backend.app.tasks.translation.retranslate_posts",
)
async with get_session() as session: async with get_session() as session:
cfg = await ImportSettings.load(session) cfg = await ImportSettings.load(session)
untranslated = (await session.execute( untranslated = (await session.execute(
@@ -315,6 +358,21 @@ async def translation_status():
Post.post_title.is_not(None), Post.description.is_not(None), Post.post_title.is_not(None), Post.description.is_not(None),
)) ))
)).scalar_one() )).scalar_one()
# Live progress: is a sweep running now, and what did the last one do?
# (run-until-done re-enqueues itself, so `active` stays true across a
# bulk re-translate; `last_run` surfaces a completed run's outcome.)
active = (await session.execute(
select(func.count(TaskRun.id))
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.status == "running")
)).scalar_one()
last = (await session.execute(
select(TaskRun.task_name, TaskRun.status, TaskRun.finished_at)
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.finished_at.is_not(None))
.order_by(TaskRun.finished_at.desc())
.limit(1)
)).first()
base_url = (cfg.interpreter_base_url or "").strip() base_url = (cfg.interpreter_base_url or "").strip()
healthy = await asyncio.to_thread(ic.health, base_url) if base_url else False healthy = await asyncio.to_thread(ic.health, base_url) if base_url else False
return jsonify({ return jsonify({
@@ -322,6 +380,12 @@ async def translation_status():
"base_url_set": bool(base_url), "base_url_set": bool(base_url),
"healthy": healthy, "healthy": healthy,
"untranslated_count": int(untranslated), "untranslated_count": int(untranslated),
"active": int(active) > 0,
"last_run": {
"task": last[0].rsplit(".", 1)[-1],
"status": last[1],
"finished_at": last[2].isoformat() if last[2] else None,
} if last else None,
}) })
@@ -338,9 +402,49 @@ async def translation_test():
return jsonify({"healthy": healthy}) return jsonify({"healthy": healthy})
@settings_bp.route("/settings/translation/probe", methods=["POST"])
async def translation_probe():
"""Diagnostic for the Settings 'Test translation' box: translate a pasted
snippet WITHOUT saving anything, returning what Interpreter *detected*
(language + confidence) alongside the result. Lets the operator see why a
given string was (mis-)detected — e.g. a short English title flagged as
another language — so a detection guard can be tuned from real numbers.
Read-only: no post is touched. Uses the currently-saved base URL + target."""
body = await request.get_json(silent=True)
body = body if isinstance(body, dict) else {}
text = (body.get("text") or "").strip()
if not text:
return jsonify({"error": "provide text to translate"}), 400
async with get_session() as session:
cfg = await ImportSettings.load(session)
base_url = (cfg.interpreter_base_url or "").strip()
if not base_url:
return jsonify({"error": "no Interpreter base URL is set"}), 400
target = (cfg.translation_target_lang or "en").strip() or "en"
try:
res = await asyncio.to_thread(
ic.translate, [text], base_url=base_url, target=target,
)
except ic.InterpreterUnavailable as e:
return jsonify({"error": f"Interpreter unavailable: {e}"}), 503
except ic.InterpreterBadRequest as e:
return jsonify({"error": f"Interpreter rejected the request: {e}"}), 400
translations = res.get("translations") or []
return jsonify({
"target": target,
"detected_lang": res.get("detected_lang"),
"detected_confidence": res.get("detected_confidence"),
"engine": res.get("engine"),
"engine_version": res.get("engine_version"),
"translated": translations[0] if translations else None,
})
@settings_bp.route("/settings/translation/run", methods=["POST"]) @settings_bp.route("/settings/translation/run", methods=["POST"])
async def translation_run(): async def translation_run():
"""Enqueue the translate sweep now (the Settings 'Translate now' button).""" """Enqueue the translate sweep now (the Settings 'Translate now' button).
Runs in drain mode — run-until-done — so one press chases the whole
untranslated backlog to zero rather than a single 300-post chunk."""
async with get_session() as session: async with get_session() as session:
cfg = await ImportSettings.load(session) cfg = await ImportSettings.load(session)
if not cfg.translation_enabled or not (cfg.interpreter_base_url or "").strip(): if not cfg.translation_enabled or not (cfg.interpreter_base_url or "").strip():
@@ -349,7 +453,7 @@ async def translation_run():
), 400 ), 400
from ..tasks.translation import translate_posts from ..tasks.translation import translate_posts
r = translate_posts.delay() r = translate_posts.delay(drain=True)
return jsonify({"celery_task_id": r.id}), 202 return jsonify({"celery_task_id": r.id}), 202
+10 -30
View File
@@ -11,10 +11,11 @@ suggestions_bp = Blueprint("suggestions", __name__, url_prefix="/api")
@suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"]) @suggestions_bp.route("/images/<int:image_id>/suggestions", methods=["GET"])
async def get_suggestions(image_id: int): async def get_suggestions(image_id: int):
# ?min=<float> overrides the configured per-category thresholds so the typed # ?min=<float> overrides the per-head suggest thresholds for INCLUSION. The
# tag-input dropdown can surface EVERY stored prediction (min=0), including # rail sends min=0 in its single per-image fetch to get EVERY head (each row
# low-confidence actions/features, in canonical formatting. Omitted → the # still carries above_threshold vs its natural cut), then derives the panel
# curated above-threshold list the Suggestions panel uses. # (above_threshold) and the typed dropdown (all, filtered by text) client-side
# — no second request. Omitted → only above-threshold rows.
override = None override = None
raw_min = request.args.get("min") raw_min = request.args.get("min")
if raw_min is not None: if raw_min is not None:
@@ -36,12 +37,11 @@ async def get_suggestions(image_id: int):
"category": s.category, "category": s.category,
"score": round(s.score, 4), "score": round(s.score, 4),
"source": s.source, "source": s.source,
"creates_new_tag": s.creates_new_tag, # whether the score cleared the head's own suggest cut.
# raw model key (alias is stored under this) + whether an # The single min=0 fetch returns every head; the panel
# operator alias produced this suggestion — drive the # shows above_threshold, the typed dropdown shows all and
# modal's "Treat as alias"/"Remove alias" affordances. # annotates each match with its score.
"raw_name": s.raw_name, "above_threshold": s.above_threshold,
"via_alias": s.via_alias,
# operator dismissed this tag for this image — surfaced # operator dismissed this tag for this image — surfaced
# (not dropped) so the rail can show it rejected + offer # (not dropped) so the rail can show it rejected + offer
# one-click un-reject. # one-click un-reject.
@@ -73,26 +73,6 @@ async def accept_suggestion(image_id: int):
return jsonify({"accepted": True, "tag_id": tag_id}) return jsonify({"accepted": True, "tag_id": tag_id})
@suggestions_bp.route(
"/images/<int:image_id>/suggestions/alias", methods=["POST"]
)
async def alias_suggestion(image_id: int):
body = await request.get_json()
required = {"alias_string", "alias_category", "canonical_tag_id"}
if not body or not required.issubset(body):
return jsonify({"error": f"required: {sorted(required)}"}), 400
canonical_tag_id = body["canonical_tag_id"]
async with get_session() as session:
await AllowlistService(session).add_alias_and_accept(
image_id,
body["alias_string"],
body["alias_category"],
canonical_tag_id,
)
await session.commit()
return jsonify({"accepted": True, "tag_id": canonical_tag_id})
@suggestions_bp.route( @suggestions_bp.route(
"/images/<int:image_id>/suggestions/dismiss", methods=["POST"] "/images/<int:image_id>/suggestions/dismiss", methods=["POST"]
) )
+17 -3
View File
@@ -171,16 +171,30 @@ def make_celery() -> Celery:
}, },
"presentation-auto-apply-daily": { "presentation-auto-apply-daily": {
"task": "backend.app.tasks.ml.scheduled_presentation_auto_apply", "task": "backend.app.tasks.ml.scheduled_presentation_auto_apply",
"schedule": 86400.0, # auto-hide banner/editor chrome (#141); "schedule": 86400.0, # auto-hide banner chrome (#141);
# no-op unless presentation_auto_apply_enabled # no-op unless presentation_auto_apply_enabled
}, },
"process-auto-apply-daily": {
"task": "backend.app.tasks.ml.scheduled_process_auto_apply",
"schedule": 86400.0, # auto-tag wip/editor process art (#1464);
# no-op unless process_auto_apply_enabled (opt-in)
},
"soft-wip-conflict-audit-daily": {
"task": "backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
"schedule": 86400.0, # flag ring-loud soft-WIP (sketch/doodle) tags
# for review (#1474); no-op with no content heads
},
"prune-presentation-reviews-daily": { "prune-presentation-reviews-daily": {
"task": "backend.app.tasks.ml.prune_presentation_reviews", "task": "backend.app.tasks.ml.prune_presentation_reviews",
"schedule": 86400.0, # retention: drop resolved review flags >30d "schedule": 86400.0, # retention: drop resolved review flags >30d
}, },
"translate-posts-daily": { "translate-posts-8h": {
"task": "backend.app.tasks.translation.translate_posts", "task": "backend.app.tasks.translation.translate_posts",
"schedule": 86400.0, # no-op unless translation configured + healthy "schedule": 28800.0, # every 8h: steady-state cadence for the
# trickle of newly-imported posts (no-op unless translation
# configured + healthy). One bounded 300-chunk per fire — the
# one-time backlog drains via the "Translate now" button
# (drain=True, run-until-done), not this sweep.
}, },
"snapshot-head-metrics-daily": { "snapshot-head-metrics-daily": {
"task": "backend.app.tasks.maintenance.snapshot_head_metrics", "task": "backend.app.tasks.maintenance.snapshot_head_metrics",
+27
View File
@@ -106,6 +106,33 @@ class ImportSettings(Base):
translation_target_lang: Mapped[str] = mapped_column( translation_target_lang: Mapped[str] = mapped_column(
Text, nullable=False, default="en", server_default="en", Text, nullable=False, default="en", server_default="en",
) )
# The latin-script acceptance floor for the translation gate: a translation
# whose Interpreter-reported confidence is below this is kept as the original
# (operator-tunable, milestone 155). Default 0.9 — stricter than the old
# hardcoded 0.8, because Interpreter confidently mis-detects short ASCII
# English (e.g. "… WIP Part 1") as a European language at ~0.86. CJK stays
# trusted regardless (script-detected). Per-post overrides handle the misses.
translation_min_confidence: Mapped[float] = mapped_column(
Float, nullable=False, default=0.9, server_default="0.9",
)
# Title-based WIP auto-tagging (task #1458). When a freshly-imported post's
# TITLE explicitly declares work-in-progress ("WIP" / "work in progress"),
# the importer applies the `wip` system tag to its images — the artist's own
# label, used to keep unfinished pieces out of the Explore/gallery browse. ON
# by default (rule 26 — the feature works out of the box). Gates only the
# LIVE import hook; the existing catalogue is caught by the operator-triggered
# "Scan existing posts" backfill (which runs regardless of this flag).
wip_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default="true",
)
# Soft WIP title tier (#1474): also tag sketch/doodle/scribble titles, but with
# a PROVISIONAL source (`wip_title_soft`) that never trains the head, since these
# are lower-precision (a finished "sketch" isn't WIP). OFF by default — a lower-
# precision tier is opt-in (the ring-loud audit surfaces false positives).
wip_soft_title_tagging_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default="false",
)
@classmethod @classmethod
async def load(cls, session) -> ImportSettings: async def load(cls, session) -> ImportSettings:
+28 -6
View File
@@ -85,12 +85,14 @@ class MLSettings(Base):
Float, nullable=False, default=0.95 Float, nullable=False, default=0.95
) )
# -- Presentation chrome auto-hide (#141) ------------------------------- # -- Presentation chrome auto-hide (#141) -------------------------------
# banner / editor screenshot auto-apply on the sweep with their OWN flat # `banner` (chrome — clusters on UI, not content) auto-applies on the sweep
# threshold (decoupled from content-head graduation). Hiding is consequential # with its OWN flat threshold (decoupled from content-head graduation) and is
# so it runs HIGH. `wip` is never auto-applied. When an image would be # HIDDEN from the gallery. Hiding is consequential so it runs HIGH. When an
# auto-hidden but ALSO scores >= presentation_conflict_threshold on a content # image would be auto-hidden but ALSO scores >= presentation_conflict_threshold
# head, it's still hidden but flagged for review (PresentationReview) instead # on a content head, it's still hidden but flagged for review
# of buried silently. ON by default (opt-out); every auto-tag is reversible. # (PresentationReview, mode='chrome') instead of buried silently. ON by default
# (opt-out); every auto-tag is reversible. NOTE (#1464): `wip` + `editor
# screenshot` are no longer chrome — they went to the PROCESS path below.
presentation_auto_apply_enabled: Mapped[bool] = mapped_column( presentation_auto_apply_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True Boolean, nullable=False, default=True
) )
@@ -100,6 +102,26 @@ class MLSettings(Base):
presentation_conflict_threshold: Mapped[float] = mapped_column( presentation_conflict_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.50 Float, nullable=False, default=0.50
) )
# -- Process auto-apply (#1464) ----------------------------------------
# `wip` / `editor screenshot` are PROCESS art — unfinished pieces + program
# screenshots that must stay OUT of head/CCIP training but, unlike chrome,
# remain VISIBLE in the gallery (operator 2026-07-12). They auto-apply on the
# sweep with their OWN flat threshold and a PROVISIONAL source (`process_auto`,
# in training_data._AUTO_SOURCES) so the head NEVER trains on its own output —
# it learns only from title (`wip_title`) + manual labels, which breaks the
# runaway loop. When a process tag would be applied but the image ALSO scores
# >= process_conflict_threshold on a content head, it's flagged for review
# (PresentationReview, mode='process') rather than silently marked. OFF by
# default — a new whole-library auto-tagger is opt-in; every auto-tag reversible.
process_auto_apply_enabled: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False
)
process_auto_apply_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.90
)
process_conflict_threshold: Mapped[float] = mapped_column(
Float, nullable=False, default=0.50
)
# Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069); # Default = SigLIP 2 (so400m, 512px) for new installs (migration 0069);
# existing libraries keep their stored value until the operator re-embeds. # existing libraries keep their stored value until the operator re-embeds.
embedder_model_version: Mapped[str] = mapped_column( embedder_model_version: Mapped[str] = mapped_column(
+25 -1
View File
@@ -8,7 +8,17 @@ artist-filter queries don't depend on the Source detour).
from datetime import datetime from datetime import datetime
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func from sqlalchemy import (
JSON,
CheckConstraint,
DateTime,
ForeignKey,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from .base import Base from .base import Base
@@ -23,6 +33,10 @@ class Post(Base):
# (created in alembic 0030) covers that case via # (created in alembic 0030) covers that case via
# (artist_id, external_post_id). # (artist_id, external_post_id).
UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"), UniqueConstraint("source_id", "external_post_id", name="uq_post_source_external_id"),
CheckConstraint(
"translation_override IN ('auto', 'force', 'original')",
name="ck_post_translation_override",
),
) )
id: Mapped[int] = mapped_column(Integer, primary_key=True) id: Mapped[int] = mapped_column(Integer, primary_key=True)
@@ -64,6 +78,16 @@ class Post(Base):
translated_at: Mapped[datetime | None] = mapped_column( translated_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True DateTime(timezone=True), nullable=True
) )
# Sticky per-post override of the translation decision (milestone 155):
# 'auto' = the acceptance gate decides; 'force' = always store Interpreter's
# translation even below the confidence floor (rescue a skipped legit-foreign
# title); 'original' = never translate, keep the original (kill a confidently
# mis-flagged one the floor can't catch). The sweep reads this on every run,
# and re-translate leaves 'original' posts alone, so the choice survives a
# Re-translate-all.
translation_override: Mapped[str] = mapped_column(
String(16), nullable=False, default="auto", server_default="auto",
)
downloaded_at: Mapped[datetime] = mapped_column( downloaded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
+15 -7
View File
@@ -1,15 +1,17 @@
"""PresentationReview — an auto-hidden presentation tag that ALSO looked like """PresentationReview — a system-tag the auto-apply sweep applied that ALSO looked
real content, flagged for operator review (milestone 141). like real content, flagged for operator review (milestone 141 + #1464).
When the auto-apply sweep hides an image as chrome (banner / editor screenshot) When a sweep applies a system tag but the image ALSO scores highly on a content
but the image ALSO scores highly on a content head, it still hides it but records head, it still applies the tag but records this row so a review strip can surface
this row so the Hidden view can surface it ("⚠ also looks like <conflict tag>") it ("⚠ also looks like <conflict tag>"). Two modes (#1464): 'chrome' (banner —
for a keep-hidden / un-hide decision. Resolved rows are pruned by retention. image is HIDDEN, review is keep-hidden / un-hide) and 'process' (wip / editor
screenshot — image stays VISIBLE, review is confirm / remove-tag). Resolved rows
are pruned by retention.
""" """
from datetime import datetime from datetime import datetime
from sqlalchemy import DateTime, Float, ForeignKey, func from sqlalchemy import DateTime, Float, ForeignKey, String, func
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from .base import Base from .base import Base
@@ -31,6 +33,12 @@ class PresentationReview(Base):
ForeignKey("tag.id", ondelete="SET NULL"), nullable=True ForeignKey("tag.id", ondelete="SET NULL"), nullable=True
) )
conflict_score: Mapped[float] = mapped_column(Float, nullable=False) conflict_score: Mapped[float] = mapped_column(Float, nullable=False)
# Which sweep flagged this (#1464): 'chrome' (banner, hidden) or 'process'
# (wip / editor screenshot, shown). Drives which review strip surfaces it and
# what "resolve" means (un-hide vs remove-tag). Existing rows backfill 'chrome'.
mode: Mapped[str] = mapped_column(
String(16), nullable=False, default="chrome", server_default="chrome"
)
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now() DateTime(timezone=True), nullable=False, server_default=func.now()
) )
+12 -7
View File
@@ -43,14 +43,19 @@ class TagKind(StrEnum):
# to keep historic tag rows queryable. # to keep historic tag rows queryable.
# The seeded system tags (migration 0075). PRESENTATION tags additionally # The seeded system tags (migration 0075). Two behavior groups (#1464):
# hide from whole-image similarity results — they cluster on UI chrome, not # CHROME (banner): clusters on UI chrome, not content → HIDDEN from the default
# content. `wip` is real art: only the training pipelines exclude it. # gallery + from similarity; auto-applied via the sweep's chrome mode.
# PROCESS (wip, editor screenshot): real-but-unfinished art / program screenshots
# → SHOWN in the gallery (operator 2026-07-12), but excluded from the Explore
# rabbit-hole; auto-applied via the sweep's process mode (provisional source,
# ring-loud review guard).
# ALL three are excluded from OTHER concepts' head/CCIP training (training-hygiene,
# keyed on is_system); a system tag's OWN head trains on them — that's what makes
# auto-flagging work.
SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot") SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot")
PRESENTATION_SYSTEM_TAGS = ("banner", "editor screenshot") CHROME_SYSTEM_TAGS = ("banner",)
# `wip` marks real-but-unfinished art. It's kept in the gallery's own "similar" PROCESS_SYSTEM_TAGS = ("wip", "editor screenshot")
# results (#1274), but the Explore rabbit-hole opts to hide it (exclude_wip) so a
# browse doesn't keep surfacing work-in-progress (operator, 2026-07-08).
WIP_SYSTEM_TAG = "wip" WIP_SYSTEM_TAG = "wip"
image_tag = Table( image_tag = Table(
+51 -15
View File
@@ -31,7 +31,7 @@ from ..models import (
Tag, Tag,
TagPositiveConfirmation, TagPositiveConfirmation,
) )
from ..models.tag import PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG, image_tag from ..models.tag import CHROME_SYSTEM_TAGS, PROCESS_SYSTEM_TAGS, image_tag
from .pagination import decode_cursor, encode_cursor from .pagination import decode_cursor, encode_cursor
from .tag_query import ( from .tag_query import (
fandom_join_alias, fandom_join_alias,
@@ -396,6 +396,25 @@ def _diversify_similar(src, rows, limit, *, dup_threshold=8, lam=0.40):
return [kept[i] for i in order] return [kept[i] for i in order]
def _reach_sample(rows, limit, reach):
"""From a distance-sorted candidate pool (nearest first), pick a spread of ranks
that MIXES near (tag the current cluster) and mid-far (escape it) BEFORE dedup +
MMR — the Explore "reach" dial (#1476).
reach in (0, 1]: the sampled span grows outward from the anchor (0.25→1.0 of the
pool), evenly strided from rank 0 so the nearest are still represented. In a
dense signature the nearest ranks are near-identical, so reaching farther is the
only way to hand MMR genuinely different content — MMR alone can't escape a pool
that's already all-near. reach<=0 or a small pool passes through unchanged."""
n = len(rows)
want = max(limit * 8, 100)
if reach <= 0 or n <= want:
return rows
span = int(min(1.0, 0.25 + 0.75 * reach) * n)
idx = sorted({min(int(i * span / want), n - 1) for i in range(want)})
return [rows[i] for i in idx]
async def _artists_for(session, image_ids: list[int]) -> dict[int, dict]: async def _artists_for(session, image_ids: list[int]) -> dict[int, dict]:
"""Map image_id -> {"name","slug"} via the canonical """Map image_id -> {"name","slug"} via the canonical
image_record.artist_id (FC-2d-vii-c). Bounded by page size.""" image_record.artist_id (FC-2d-vii-c). Bounded by page size."""
@@ -419,16 +438,17 @@ class GalleryService:
async def _hidden_tag_ids( async def _hidden_tag_ids(
self, include_hidden, tag_ids, tag_or_groups, self, include_hidden, tag_ids, tag_or_groups,
) -> list[int] | None: ) -> list[int] | None:
"""Presentation-chrome tag ids to implicitly exclude from a gallery query, """Chrome (banner) tag ids to implicitly exclude from a gallery query, or
or None. None when the caller asked to include hidden, when the operator None. None when the caller asked to include hidden, when the operator is
is explicitly filtering FOR a presentation tag (they clearly want to see explicitly filtering FOR a chrome tag (they clearly want to see it), or when
it), or when no presentation tags exist. (milestone 141)""" no chrome tags exist. (milestone 141; #1464: editor screenshot is now PROCESS
— shown — so only `banner` hides here.)"""
if include_hidden: if include_hidden:
return None return None
rows = await self.session.execute( rows = await self.session.execute(
select(Tag.id).where( select(Tag.id).where(
Tag.is_system.is_(True), Tag.is_system.is_(True),
Tag.name.in_(PRESENTATION_SYSTEM_TAGS), Tag.name.in_(CHROME_SYSTEM_TAGS),
) )
) )
pres = [r[0] for r in rows] pres = [r[0] for r in rows]
@@ -716,6 +736,7 @@ class GalleryService:
untagged: bool = False, no_artist: bool = False, untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None, date_from: datetime | None = None, date_to: datetime | None = None,
exclude_wip: bool = False, exclude_wip: bool = False,
reach: float = 0.0, exclude_ids: list[int] | None = None,
) -> list[GalleryImage] | None: ) -> list[GalleryImage] | None:
"""Visual "more like this": images near `image_id`'s SigLIP embedding """Visual "more like this": images near `image_id`'s SigLIP embedding
(pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result (pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result
@@ -744,20 +765,27 @@ class GalleryService:
# wide pool there's nothing but the near-dupes to choose from. Widened # wide pool there's nothing but the near-dupes to choose from. Widened
# (5×→8×, cap 200→400) so the stronger MMR has genuinely distinct # (5×→8×, cap 200→400) so the stronger MMR has genuinely distinct
# neighbourhoods to reach into for more variance (operator, 2026-07-01). # neighbourhoods to reach into for more variance (operator, 2026-07-01).
pool_n = min(400, max(limit * 8, 100)) # Explore's reach>0 (#1476) widens it a LOT more: in a dense signature the
# nearest few hundred are all near-identical, so far-enough candidates only
# exist deeper in the ranked pool. _reach_sample then strides across them.
if reach > 0:
pool_n = min(1000, max(limit * 25, 100))
else:
pool_n = min(400, max(limit * 8, 100))
distance = ImageRecord.siglip_embedding.cosine_distance(src.siglip_embedding) distance = ImageRecord.siglip_embedding.cosine_distance(src.siglip_embedding)
eff = _effective_date_col() eff = _effective_date_col()
stmt = select(ImageRecord, Post.post_date, eff.label("eff")) stmt = select(ImageRecord, Post.post_date, eff.label("eff"))
stmt = _outer_join_primary_post(stmt) stmt = _outer_join_primary_post(stmt)
# Presentation images (banner / editor-screenshot system tags, #128) # Chrome (banner, #128) clusters on UI rather than content, so near any one
# cluster on UI chrome rather than content, so near any one of them # of them they'd fill the grid → excluded from CANDIDATES always (the anchor
# they'd fill the grid. Excluded from CANDIDATES only — the anchor # itself may be a banner). PROCESS art (wip / editor screenshot) stays
# itself may be a banner. `wip` stays surfaced here by default (real art; # surfaced here by default (real content; only the training pipelines exclude
# only the training pipelines exclude it), but the Explore rabbit-hole # it), but the Explore rabbit-hole passes exclude_wip to also drop the whole
# passes exclude_wip to also drop work-in-progress (operator, 2026-07-08). # process group so a browse doesn't keep surfacing work-in-progress
excluded_system_tags = PRESENTATION_SYSTEM_TAGS # (operator, 2026-07-08; #1464 — editor now rides with wip here).
excluded_system_tags = CHROME_SYSTEM_TAGS
if exclude_wip: if exclude_wip:
excluded_system_tags = (*PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG) excluded_system_tags = (*CHROME_SYSTEM_TAGS, *PROCESS_SYSTEM_TAGS)
presentation = ( presentation = (
select(image_tag.c.image_record_id) select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id) .join(Tag, Tag.id == image_tag.c.tag_id)
@@ -771,6 +799,10 @@ class GalleryService:
ImageRecord.id != image_id, ImageRecord.id != image_id,
ImageRecord.id.not_in(presentation), ImageRecord.id.not_in(presentation),
) )
# Anti-revisit (#1476): the Explore walk passes its breadcrumb so already-
# walked images aren't re-served as neighbours — → can't loop you back in.
if exclude_ids:
stmt = stmt.where(ImageRecord.id.not_in(exclude_ids))
stmt = _apply_scope( stmt = _apply_scope(
stmt, tag_ids=tag_ids, post_id=None, stmt, tag_ids=tag_ids, post_id=None,
artist_id=artist_id, media_type=media_type, artist_id=artist_id, media_type=media_type,
@@ -780,6 +812,10 @@ class GalleryService:
) )
stmt = stmt.order_by(distance.asc()).limit(pool_n) stmt = stmt.order_by(distance.asc()).limit(pool_n)
rows = (await self.session.execute(stmt)).all() rows = (await self.session.execute(stmt)).all()
# Explore reach: stride across an outward-growing distance span so the pool
# handed to MMR spans near→mid-far, not just the tight cluster (#1476).
if reach > 0:
rows = _reach_sample(rows, limit, reach)
rows = _diversify_similar(src, rows, limit) rows = _diversify_similar(src, rows, limit)
artists = await _artists_for(self.session, [r[0].id for r in rows]) artists = await _artists_for(self.session, [r[0].id for r in rows])
return _gallery_images(rows, artists) return _gallery_images(rows, artists)
+65
View File
@@ -47,9 +47,21 @@ from .attachment_store import AttachmentStore
from .audits import single_color from .audits import single_color
from .link_extract import extract_external_links from .link_extract import extract_external_links
from .thumbnailer import Thumbnailer from .thumbnailer import Thumbnailer
from .wip_title import (
WIP_TITLE_SOFT_SOURCE,
WIP_TITLE_SOURCE,
apply_wip_image_tags,
matches_soft_wip_title,
matches_wip_title,
resolve_wip_tag_id,
)
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
# Sentinel for the lazily-resolved wip tag id (distinguishes "not resolved yet"
# from a genuine None = tag absent, so absence is cached and not re-queried).
_UNSET = object()
class SkipReason(StrEnum): class SkipReason(StrEnum):
too_small = "too_small" too_small = "too_small"
@@ -183,6 +195,10 @@ class Importer:
# invalidated mid-Importer (Importer instances are per-task / # invalidated mid-Importer (Importer instances are per-task /
# per-archive-import so cross-instance staleness is harmless). # per-archive-import so cross-instance staleness is harmless).
self._phash_candidates: list[tuple] | None = None self._phash_candidates: list[tuple] | None = None
# Lazily-resolved `wip` system tag id for title-based WIP auto-tagging
# (task #1458). Sentinel _UNSET so a genuine None (tag absent) is cached
# and not re-queried per media. Importer is per-task, so this can't stale.
self._wip_tag_id: int | None = _UNSET
def _phash_candidates_cache(self) -> list[tuple]: def _phash_candidates_cache(self) -> list[tuple]:
"""Cached `(phash, width, height, id)` rows from image_record. """Cached `(phash, width, height, id)` rows from image_record.
@@ -933,6 +949,10 @@ class Importer:
# Thumbnail is queued separately by the calling task; the importer # Thumbnail is queued separately by the calling task; the importer
# does not generate thumbnails inline so the import queue stays moving. # does not generate thumbnails inline so the import queue stays moving.
# Title-based WIP auto-tag (task #1458): fresh import only, after the
# sidecar has linked the post so record.primary_post_id / its title exist.
self._maybe_apply_wip_title(record)
self.session.commit() self.session.commit()
return ImportResult(status="imported", image_id=record.id) return ImportResult(status="imported", image_id=record.id)
@@ -976,6 +996,47 @@ class Importer:
self.session.commit() self.session.commit()
return ImportResult(status="refreshed", image_id=existing.id) return ImportResult(status="refreshed", image_id=existing.id)
def _maybe_apply_wip_title(self, record: ImageRecord) -> None:
"""Auto-apply the `wip` system tag to a FRESHLY-imported image when its
primary post's TITLE explicitly declares work-in-progress (task #1458 —
the artist's own "WIP" / "work in progress" label).
Called ONLY from the two new-record paths (never deep-scan / supersede),
so a manually-removed WIP tag is never re-applied by a routine re-scan —
removal sticks. The existing catalogue is covered separately by the
operator-triggered backfill sweep. Gated by the settings toggle, and
best-effort: any failure is logged, never allowed to fail the import."""
hard_on = self.settings.wip_title_tagging_enabled
soft_on = self.settings.wip_soft_title_tagging_enabled
if not (hard_on or soft_on):
return
if record.primary_post_id is None:
return
try:
title = self.session.execute(
select(Post.post_title).where(Post.id == record.primary_post_id)
).scalar_one_or_none()
# HARD tier ("WIP"/"work in progress") wins — higher precision, and it
# trains the head; SOFT (sketch/doodle, #1474) is the provisional fallback
# that never trains (source wip_title_soft).
if hard_on and matches_wip_title(title):
source = WIP_TITLE_SOURCE
elif soft_on and matches_soft_wip_title(title):
source = WIP_TITLE_SOFT_SOURCE
else:
return
if self._wip_tag_id is _UNSET:
self._wip_tag_id = resolve_wip_tag_id(self.session)
if self._wip_tag_id is None:
return
apply_wip_image_tags(
self.session, [record.id], self._wip_tag_id, source=source
)
except Exception as exc: # noqa: BLE001 — a tag must never fail an import
log.warning(
"wip-title auto-tag failed for image %s: %s", record.id, exc
)
def _apply_post_fields(self, post: Post, sd) -> None: def _apply_post_fields(self, post: Post, sd) -> None:
"""Write a parsed sidecar's post-level fields onto a Post — the SINGLE """Write a parsed sidecar's post-level fields onto a Post — the SINGLE
predicate shared by BOTH ingest paths: the per-media path (_apply_sidecar) predicate shared by BOTH ingest paths: the per-media path (_apply_sidecar)
@@ -1253,6 +1314,10 @@ class Importer:
# per-post Source row. # per-post Source row.
self._apply_sidecar(record, path, artist, explicit_source=source) self._apply_sidecar(record, path, artist, explicit_source=source)
# Title-based WIP auto-tag (task #1458): fresh import only, see the
# matching call in _import_media.
self._maybe_apply_wip_title(record)
self.session.commit() self.session.commit()
return ImportResult(status="imported", image_id=record.id) return ImportResult(status="imported", image_id=record.id)
+7 -3
View File
@@ -96,8 +96,9 @@ def translate(
) -> dict: ) -> dict:
"""Translate a batch. Returns:: """Translate a batch. Returns::
{"translations": [str, ...], # SAME order & length as `texts` {"translations": [str, ...], # SAME order & length as `texts`
"detected_lang": str | None, # aggregate: describes the FIRST item "detected_lang": str | None, # aggregate: describes the FIRST item
"detected_confidence": float | None, # detector's confidence, if given
"engine": str | None, "engine": str | None,
"engine_version": str | None} "engine_version": str | None}
@@ -110,6 +111,7 @@ def translate(
""" """
if not texts: if not texts:
return {"translations": [], "detected_lang": None, return {"translations": [], "detected_lang": None,
"detected_confidence": None,
"engine": None, "engine_version": None} "engine": None, "engine_version": None}
try: try:
r = session.post( r = session.post(
@@ -140,9 +142,11 @@ def translate(
if not isinstance(out, list): if not isinstance(out, list):
out = [out] out = [out]
interp = body.get("interpreter") or {} interp = body.get("interpreter") or {}
detected = body.get("detectedLanguage") or {}
return { return {
"translations": out, "translations": out,
"detected_lang": (body.get("detectedLanguage") or {}).get("language"), "detected_lang": detected.get("language"),
"detected_confidence": detected.get("confidence"),
"engine": interp.get("engine"), "engine": interp.get("engine"),
"engine_version": interp.get("engine_version"), "engine_version": interp.get("engine_version"),
} }
-14
View File
@@ -12,13 +12,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
from ...models import TagSuggestionRejection from ...models import TagSuggestionRejection
from ...models.tag import image_tag from ...models.tag import image_tag
from .aliases import AliasService
class AllowlistService: class AllowlistService:
def __init__(self, session: AsyncSession): def __init__(self, session: AsyncSession):
self.session = session self.session = session
self.aliases = AliasService(session)
async def _apply_image_tag(self, image_id: int, tag_id: int, source: str): async def _apply_image_tag(self, image_id: int, tag_id: int, source: str):
stmt = insert(image_tag).values( stmt = insert(image_tag).values(
@@ -41,18 +39,6 @@ class AllowlistService:
await self._apply_image_tag(image_id, tag_id, source="ml_accepted") await self._apply_image_tag(image_id, tag_id, source="ml_accepted")
await self._clear_rejection(image_id, tag_id) await self._clear_rejection(image_id, tag_id)
async def add_alias_and_accept(
self,
image_id: int,
alias_string: str,
alias_category: str,
canonical_tag_id: int,
) -> None:
await self.aliases.create(
alias_string, alias_category, canonical_tag_id
)
await self.accept(image_id, canonical_tag_id)
async def dismiss(self, image_id: int, tag_id: int) -> None: async def dismiss(self, image_id: int, tag_id: int) -> None:
stmt = insert(TagSuggestionRejection).values( stmt = insert(TagSuggestionRejection).values(
image_record_id=image_id, tag_id=tag_id image_record_id=image_id, tag_id=tag_id
+148 -37
View File
@@ -39,7 +39,7 @@ from ...models import (
TagPositiveConfirmation, TagPositiveConfirmation,
TagSuggestionRejection, TagSuggestionRejection,
) )
from ...models.tag import PRESENTATION_SYSTEM_TAGS, image_tag from ...models.tag import CHROME_SYSTEM_TAGS, PROCESS_SYSTEM_TAGS, image_tag
from .training_data import ( from .training_data import (
_AUTO_SOURCES, _AUTO_SOURCES,
_auto_apply_point, _auto_apply_point,
@@ -500,13 +500,19 @@ async def score_image(
session: AsyncSession, image_id: int, threshold_override: float | None = None, session: AsyncSession, image_id: int, threshold_override: float | None = None,
) -> list[dict]: ) -> list[dict]:
"""Suggestions for one image from the trained heads: [{tag_id, name, """Suggestions for one image from the trained heads: [{tag_id, name,
category, score}], ranked. A concept surfaces when its score clears the category, score, above_threshold, grounding}], ranked. A concept is INCLUDED
head's own suggest_threshold — or, when threshold_override is given (the when its score clears the head's own suggest_threshold — or, when
typed-dropdown "show everything" mode), that flat floor instead (0 → every threshold_override is given (the typed-dropdown "show everything" mode), that
head). System-tag heads (wip/banner/editor) instead use a flat flat floor instead (0 → every head). System-tag heads (wip/banner/editor)
_SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface for rejection instead use a flat _SYSTEM_TAG_SUGGEST_FLOOR so their false positives surface
(still overridden by threshold_override). Empty if the image has no for rejection (still overridden by threshold_override). Empty if the image has
embedding or no heads exist yet. no embedding or no heads exist yet.
``above_threshold`` is reported SEPARATELY from inclusion: it's always whether
the score cleared the head's NATURAL cut (suggest_threshold, or the system
floor), regardless of any override. So the single min=0 fetch returns every
head, and the caller can split panel (above_threshold) from dropdown (all)
without a second request.
MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image MAX-OVER-BAG: the image is scored as a BAG of embeddings — the whole-image
vector PLUS every concept-region crop the agent embedded (same model vector PLUS every concept-region crop the agent embedded (same model
@@ -537,21 +543,25 @@ async def score_image(
winners = probs_bag.argmax(axis=0) # (H,) winners = probs_bag.argmax(axis=0) # (H,)
out = [] out = []
for i, p in enumerate(probs): for i, p in enumerate(probs):
if threshold_override is not None: m = heads["meta"][i]
cut = threshold_override # The head's NATURAL suggest cut — system tags use the flat floor (see
elif heads["meta"][i]["is_system"]: # _SYSTEM_TAG_SUGGEST_FLOOR) so their false positives show up for the
# System tags surface at the flat floor (see _SYSTEM_TAG_SUGGEST_FLOOR) # operator to reject; content heads use their own precision-tuned
# so their false positives show up for the operator to reject. # threshold. This is what "above threshold" means (drives the panel).
cut = _SYSTEM_TAG_SUGGEST_FLOOR natural = (
else: _SYSTEM_TAG_SUGGEST_FLOOR if m["is_system"] else float(heads["thr"][i])
cut = heads["thr"][i] )
# INCLUSION is looser under threshold_override (dropdown show-all,
# override=0): every head comes back so a low-confidence concept can still
# be typed + picked, each carrying its own above_threshold flag.
cut = threshold_override if threshold_override is not None else natural
if p >= cut: if p >= cut:
m = heads["meta"][i]
out.append({ out.append({
"tag_id": m["tag_id"], "tag_id": m["tag_id"],
"name": m["name"], "name": m["name"],
"category": m["category"], "category": m["category"],
"score": float(p), "score": float(p),
"above_threshold": bool(p >= natural),
"grounding": bag_meta[int(winners[i])], "grounding": bag_meta[int(winners[i])],
}) })
out.sort(key=lambda d: d["score"], reverse=True) out.sort(key=lambda d: d["score"], reverse=True)
@@ -749,18 +759,42 @@ def auto_apply_sweep(
_PRESENTATION_SOURCE = "presentation_auto" _PRESENTATION_SOURCE = "presentation_auto"
_PROCESS_SOURCE = "process_auto"
# System-tag auto-apply modes (#1464). Both modes run the identical sweep — apply
# a system tag at a flat threshold with a PROVISIONAL source + a ring-loud review
# guard — and differ ONLY in which tags, which settings knobs, and which
# source/review-mode. 'chrome' (banner) is HIDDEN from the gallery; 'process'
# (wip / editor screenshot) stays VISIBLE (the hide is a gallery-query effect of
# the tag's group membership, not of this sweep).
_SWEEP_MODES = {
"chrome": {
"names": CHROME_SYSTEM_TAGS,
"enabled": "presentation_auto_apply_enabled",
"threshold": "presentation_auto_apply_threshold",
"conflict": "presentation_conflict_threshold",
"source": _PRESENTATION_SOURCE,
},
"process": {
"names": PROCESS_SYSTEM_TAGS,
"enabled": "process_auto_apply_enabled",
"threshold": "process_auto_apply_threshold",
"conflict": "process_conflict_threshold",
"source": _PROCESS_SOURCE,
},
}
def _presentation_heads(session: Session, embedding_version: str): def _system_tag_heads(session: Session, embedding_version: str, names):
"""Trained heads for the presentation chrome tags (banner / editor screenshot). """Trained heads for a system-tag group (chrome banner / process wip+editor).
They fire at the FLAT presentation threshold regardless of graduation — a head They fire at the group's FLAT threshold regardless of graduation — a head
exists once the operator has labelled enough chrome (head_min_positives).""" exists once the operator has labelled enough (head_min_positives)."""
return session.execute( return session.execute(
select(TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias) select(TagHead.tag_id, Tag.name, TagHead.weights, TagHead.bias)
.join(Tag, Tag.id == TagHead.tag_id) .join(Tag, Tag.id == TagHead.tag_id)
.where(TagHead.embedding_version == embedding_version) .where(TagHead.embedding_version == embedding_version)
.where(Tag.is_system.is_(True)) .where(Tag.is_system.is_(True))
.where(Tag.name.in_(PRESENTATION_SYSTEM_TAGS)) .where(Tag.name.in_(names))
).all() ).all()
@@ -792,27 +826,33 @@ def _valued_image_ids(session: Session) -> set[int]:
return {r[0] for r in rows} return {r[0] for r in rows}
def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> dict: def system_tag_auto_apply_sweep(
"""Auto-hide presentation chrome (banner / editor screenshot) at the FLAT session: Session, *, mode: str, dry_run: bool = False
presentation threshold (#141) — NOT the per-head graduated threshold. Two ) -> dict:
guards keep it safe: (1) never hide an image carrying a human/confirmed content """Auto-apply a system-tag group at its FLAT threshold. mode='chrome' (banner,
tag; (2) if an image about to be hidden ALSO scores >= the conflict threshold #141) hides the image; mode='process' (wip / editor screenshot, #1464) keeps it
on a content head, still hide it but flag it (PresentationReview) so the Hidden VISIBLE — the ONLY difference is the tag group's gallery membership, not this
view surfaces "also looks like <X>" for review. No-op unless sweep. Two guards keep it safe: (1) never touch an image carrying a
presentation_auto_apply_enabled. numpy-only (no sklearn). Returns human/confirmed content tag; (2) if the image ALSO scores >= the conflict
{n_applied, n_flagged, concepts}.""" threshold on a content head, still apply but flag it (PresentationReview,
mode=<mode>) so the review strip surfaces "also looks like <X>". The source is
PROVISIONAL so the head never trains on its own output. No-op unless the mode's
enabled flag is set. numpy-only (no sklearn). Returns {n_applied, n_flagged,
concepts}."""
import numpy as np import numpy as np
from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.dialects.postgresql import insert as pg_insert
cfg = _SWEEP_MODES[mode]
settings = _settings(session) settings = _settings(session)
if not dry_run and not settings.presentation_auto_apply_enabled: if not dry_run and not getattr(settings, cfg["enabled"]):
return {"n_applied": 0, "n_flagged": 0, "concepts": []} return {"n_applied": 0, "n_flagged": 0, "concepts": []}
ver = settings.embedder_model_version ver = settings.embedder_model_version
pres = _presentation_heads(session, ver) pres = _system_tag_heads(session, ver, cfg["names"])
if not pres: if not pres:
return {"n_applied": 0, "n_flagged": 0, "concepts": []} return {"n_applied": 0, "n_flagged": 0, "concepts": []}
thr = float(settings.presentation_auto_apply_threshold) thr = float(getattr(settings, cfg["threshold"]))
conflict_thr = float(settings.presentation_conflict_threshold) conflict_thr = float(getattr(settings, cfg["conflict"]))
source = cfg["source"]
Wp = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in pres]) Wp = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in pres])
bp = np.asarray([r.bias for r in pres], dtype=np.float32) bp = np.asarray([r.bias for r in pres], dtype=np.float32)
@@ -874,11 +914,13 @@ def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> di
pg_insert(image_tag) pg_insert(image_tag)
.values( .values(
image_record_id=iid, tag_id=tid, image_record_id=iid, tag_id=tid,
source=_PRESENTATION_SOURCE, source=source,
) )
.on_conflict_do_nothing() .on_conflict_do_nothing()
) )
# Guard 2: also looks like content → hide but flag for review. # Guard 2: also looks like real content → still apply, but flag it
# for the review strip instead of silently marking (chrome hides,
# process stays visible — either way the operator gets a heads-up).
if Wc is not None and float(max_c[idx]) >= conflict_thr: if Wc is not None and float(max_c[idx]) >= conflict_thr:
n_flagged += 1 n_flagged += 1
if not dry_run: if not dry_run:
@@ -888,6 +930,7 @@ def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> di
image_record_id=iid, tag_id=tid, image_record_id=iid, tag_id=tid,
conflict_tag_id=conf_tag_ids[int(arg_c[idx])], conflict_tag_id=conf_tag_ids[int(arg_c[idx])],
conflict_score=float(max_c[idx]), conflict_score=float(max_c[idx]),
mode=mode,
) )
.on_conflict_do_nothing() .on_conflict_do_nothing()
) )
@@ -904,6 +947,74 @@ def presentation_auto_apply_sweep(session: Session, dry_run: bool = False) -> di
} }
def soft_wip_conflict_audit(session: Session, dry_run: bool = False) -> dict:
"""Ring-loud audit for the SOFT WIP-title cohort (#1474). Images auto-tagged
`wip` from a low-precision sketch/doodle title (source='wip_title_soft') that ALSO
score >= the process conflict threshold on a content head are probably FINISHED
art mis-tagged as process — flag them (PresentationReview, mode='process') so the
review strip surfaces them ("also looks like <X>", Keep tag / Remove tag). Does
NOT remove the tag; the operator decides. No-op when there are no content heads.
numpy-only. Returns {n_scanned, n_flagged}."""
import numpy as np
from sqlalchemy.dialects.postgresql import insert as pg_insert
from ..wip_title import WIP_TITLE_SOFT_SOURCE, resolve_wip_tag_id
settings = _settings(session)
ver = settings.embedder_model_version
conflict_thr = float(settings.process_conflict_threshold)
conf = _conflict_heads(session, ver)
wip_id = resolve_wip_tag_id(session)
if not conf or wip_id is None:
return {"n_scanned": 0, "n_flagged": 0}
Wc = np.vstack([np.asarray(r.weights, dtype=np.float32) for r in conf])
bc = np.asarray([r.bias for r in conf], dtype=np.float32)
conf_tag_ids = [r.tag_id for r in conf]
soft_ids = [iid for (iid,) in session.execute(
select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == wip_id)
.where(image_tag.c.source == WIP_TITLE_SOFT_SOURCE)
)]
# Skip images already flagged for this tag (idempotent re-runs).
flagged = {iid for (iid,) in session.execute(
select(PresentationReview.image_record_id)
.where(PresentationReview.tag_id == wip_id)
)}
soft_ids = [i for i in soft_ids if i not in flagged]
n_flagged = 0
scanned = 0
for start in range(0, len(soft_ids), _AUTO_APPLY_CHUNK):
chunk = soft_ids[start:start + _AUTO_APPLY_CHUNK]
emb = _load_embeddings(session, chunk)
cids = [i for i in chunk if i in emb]
if not cids:
continue
scanned += len(cids)
Xn = _l2norm(np.vstack([emb[i] for i in cids]).astype(np.float32), np)
cprobs = 1.0 / (1.0 + np.exp(-(Xn @ Wc.T + bc)))
max_c = cprobs.max(axis=1)
arg_c = cprobs.argmax(axis=1)
for k in range(len(cids)):
if float(max_c[k]) >= conflict_thr:
n_flagged += 1
if not dry_run:
session.execute(
pg_insert(PresentationReview)
.values(
image_record_id=cids[k], tag_id=wip_id,
conflict_tag_id=conf_tag_ids[int(arg_c[k])],
conflict_score=float(max_c[k]),
mode="process",
)
.on_conflict_do_nothing()
)
if not dry_run:
session.commit()
return {"n_scanned": scanned, "n_flagged": n_flagged}
def retract_auto_applied_heads(session: Session) -> int: def retract_auto_applied_heads(session: Session) -> int:
"""Soft auto-apply (milestone 139): re-score every standing source='head_auto' """Soft auto-apply (milestone 139): re-score every standing source='head_auto'
tag against its CURRENT head and REMOVE the ones now BELOW the head's tag against its CURRENT head and REMOVE the ones now BELOW the head's
+17 -17
View File
@@ -22,22 +22,20 @@ from .heads import score_image
@dataclass(frozen=True) @dataclass(frozen=True)
class Suggestion: class Suggestion:
# canonical_tag_id is None when this is a raw Camie tag with no alias and # Every suggestion is a canonical Tag: heads/CCIP only score EXISTING concept
# no existing Tag row — accepting it will create the tag. # tags (tagging-v2, #114). The old raw-model-key / creates-new / alias-remap
canonical_tag_id: int | None # cases are gone — a suggestion always maps to a real tag id.
canonical_tag_id: int
display_name: str display_name: str
category: str category: str
score: float score: float
source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2) source: str # 'head' | 'ccip' | 'both' (Camie tagger/centroid removed in v2)
creates_new_tag: bool # above_threshold = the score cleared the head's own suggest cut (or the
# raw_name = the booru model vocab key behind this suggestion. It's the key # system floor). The Suggestions PANEL shows only these; the typed-tag
# an alias MUST be stored under (resolution looks up the raw key), so the # dropdown fetches ALL suggestions (every head, min=0) and just annotates each
# modal needs it to author an alias correctly. None for centroid-only hits # matching row with its score, so a low-confidence concept can still be typed
# (no underlying prediction → nothing to alias). # and picked. CCIP character matches are always above their match threshold.
raw_name: str | None = None above_threshold: bool
# via_alias = this suggestion was surfaced because an operator alias remapped
# the raw prediction to this canonical tag. Lets the UI mark it + offer undo.
via_alias: bool = False
# rejected = the operator dismissed this tag for this image (a stored # rejected = the operator dismissed this tag for this image (a stored
# TagSuggestionRejection). It stays in the list — flagged, not dropped — so # TagSuggestionRejection). It stays in the list — flagged, not dropped — so
# the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery, # the rejection is VISIBLE and REVERSIBLE in the rail (misclick recovery,
@@ -108,6 +106,7 @@ class SuggestionService:
for h in hits: for h in hits:
merged[(h["category"], h["tag_id"])] = { merged[(h["category"], h["tag_id"])] = {
"name": h["name"], "score": h["score"], "source": "head", "name": h["name"], "score": h["score"], "source": "head",
"above_threshold": h["above_threshold"],
"grounding": h.get("grounding"), "grounding": h.get("grounding"),
} }
for c in ccip_hits: for c in ccip_hits:
@@ -116,12 +115,16 @@ class SuggestionService:
if ex is not None: if ex is not None:
ex["source"] = "both" ex["source"] = "both"
ex["score"] = max(ex["score"], c["score"]) ex["score"] = max(ex["score"], c["score"])
# CCIP only returns matches above its own threshold, so a CCIP
# corroboration always makes the merged suggestion above-threshold.
ex["above_threshold"] = True
# Keep the head's localized crop if it had one; else fall back to # Keep the head's localized crop if it had one; else fall back to
# the CCIP figure so a corroborated character still grounds (#1206). # the CCIP figure so a corroborated character still grounds (#1206).
ex["grounding"] = ex.get("grounding") or c.get("grounding") ex["grounding"] = ex.get("grounding") or c.get("grounding")
else: else:
merged[key] = { merged[key] = {
"name": c["name"], "score": c["score"], "source": "ccip", "name": c["name"], "score": c["score"], "source": "ccip",
"above_threshold": True,
"grounding": c.get("grounding"), "grounding": c.get("grounding"),
} }
@@ -136,7 +139,7 @@ class SuggestionService:
category=cat, category=cat,
score=m["score"], score=m["score"],
source=m["source"], source=m["source"],
creates_new_tag=False, above_threshold=m["above_threshold"],
rejected=tag_id in rejected, rejected=tag_id in rejected,
grounding=m.get("grounding"), grounding=m.get("grounding"),
) )
@@ -157,8 +160,7 @@ class SuggestionService:
was suggested for (or already applied to) >= threshold fraction of was suggested for (or already applied to) >= threshold fraction of
the selection AND was acceptable on >= 1 image. Confidence is the the selection AND was acceptable on >= 1 image. Confidence is the
mean over images where it was suggested. Aggregated by mean over images where it was suggested. Aggregated by
canonical_tag_id; creates-new (no canonical id) suggestions are canonical_tag_id (every suggestion is a canonical tag now)."""
skipped (bulk Accept applies by tag id)."""
if not image_ids: if not image_ids:
return {} return {}
threshold = min(1.0, max(0.0, threshold)) threshold = min(1.0, max(0.0, threshold))
@@ -169,8 +171,6 @@ class SuggestionService:
sl = await self.for_image(image_id) sl = await self.for_image(image_id)
for category, items in sl.by_category.items(): for category, items in sl.by_category.items():
for s in items: for s in items:
if s.canonical_tag_id is None or s.creates_new_tag:
continue
# for_image keeps rejected tags (flagged) for the rail; # for_image keeps rejected tags (flagged) for the rail;
# bulk consensus must still ignore them — a tag dismissed on # bulk consensus must still ignore them — a tag dismissed on
# an image isn't a suggestion for that image. # an image isn't a suggestion for that image.
+9 -1
View File
@@ -29,7 +29,15 @@ from ...models.tag import image_tag
# a CCIP reference) unless the operator confirms them (milestone 139). Keeping # a CCIP reference) unless the operator confirms them (milestone 139). Keeping
# auto-applied predictions out of training is what makes them "soft" — a misfire # auto-applied predictions out of training is what makes them "soft" — a misfire
# can't reinforce itself, so the retraction sweep can actually drop it. # can't reinforce itself, so the retraction sweep can actually drop it.
_AUTO_SOURCES = ("head_auto", "ccip_auto", "ml_auto", "presentation_auto") # `process_auto` (#1464): wip/editor screenshot applied by the process sweep are
# ALSO provisional — the head must learn only from title (`wip_title`) + manual
# labels, never its own auto-applied output, or it would runaway (operator 2026-07-12).
# `wip_title_soft` (#1474): the soft title tier (sketch/doodle) is LOW-precision, so
# it's provisional too — a finished piece titled "sketch" must not train the wip head.
_AUTO_SOURCES = (
"head_auto", "ccip_auto", "ml_auto", "presentation_auto", "process_auto",
"wip_title_soft",
)
def _hygiene_excluded_ids(session: Session) -> set[int]: def _hygiene_excluded_ids(session: Session) -> set[int]:
@@ -397,6 +397,8 @@ class PostFeedService:
"post_title_translated": post.post_title_translated, "post_title_translated": post.post_title_translated,
"description_translated": desc_trans_short, "description_translated": desc_trans_short,
"translated_source_lang": post.translated_source_lang, "translated_source_lang": post.translated_source_lang,
# Sticky per-post translation choice (auto/force/original, #155).
"translation_override": post.translation_override,
"artist": {"id": artist.id, "name": artist.name, "slug": artist.slug}, "artist": {"id": artist.id, "name": artist.name, "slug": artist.slug},
"source": ( "source": (
{"id": source.id, "platform": source.platform} {"id": source.id, "platform": source.platform}
@@ -35,6 +35,7 @@ def _post_dict(p: Post) -> dict:
"title_translated": p.post_title_translated, "title_translated": p.post_title_translated,
"description_translated": p.description_translated, "description_translated": p.description_translated,
"translated_source_lang": p.translated_source_lang, "translated_source_lang": p.translated_source_lang,
"translation_override": p.translation_override,
} }
+125
View File
@@ -0,0 +1,125 @@
"""Title-based WIP auto-tagging (task #1458).
Deterministic heuristic: when a post's TITLE explicitly declares work-in-progress
(the artist's own "WIP" / "work in progress" label), the ``wip`` system tag is
applied to that post's images — a cheap, high-precision complement to the
image-based ML ``wip`` head. WIP images are excluded from the Explore/gallery
browse (see gallery_service ``excluded_system_tags``), so honouring the artist's
own label keeps unfinished pieces out of the main browse right at import.
Precision over recall — a false WIP tag HIDES a finished post — so matching is
token-anchored: ``swipe`` / ``wiped`` / ``wiping`` never trip it (a letter on the
boundary blocks the match).
Sync-only: both consumers (the importer and the backfill Celery task) run on a
sync Session. Application is idempotent-additive (ON CONFLICT DO NOTHING) and
stamps a distinct ``image_tag.source`` so a later pass can tell where a wip tag
came from — the "manual" / "head_auto" / "ccip_auto" / "ml_accepted" provenance
family gains one member.
"""
import re
from sqlalchemy import select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.orm import Session
from ..models.tag import WIP_SYSTEM_TAG, Tag, image_tag
# image_tag.source stamped on title-heuristic WIP tags — distinct from the other
# apply sources so provenance stays legible and a future undo can target only these.
# HARD tier ("WIP"/"work in progress") is high-precision → trains the wip head.
WIP_TITLE_SOURCE = "wip_title"
# SOFT tier (sketch/doodle/scribble, #1474) is LOWER-precision — a finished "sketch"
# is often not WIP. This source is PROVISIONAL (in training_data._AUTO_SOURCES) so it
# NEVER trains the wip head; a soft-tagged image that also looks like real content is
# surfaced by the ring-loud audit for review.
WIP_TITLE_SOFT_SOURCE = "wip_title_soft"
# A standalone "WIP" / "W.I.P" token, or the phrase "work in progress"
# (space/underscore/hyphen separated). The letter-boundary lookarounds are what
# make this precision-first: `s|wip|e`, `|wip|ed`, `|wip|ing` all have a letter
# abutting the token, so they're rejected. A trailing digit is allowed so
# "WIP2" (= WIP part 2) still matches.
_WIP_RE = re.compile(
r"(?<![A-Za-z])(?:w\.?i\.?p\.?|work[\s_-]+in[\s_-]+progress)(?![A-Za-z])",
re.IGNORECASE,
)
# Soft tier: sketch / doodle / scribble (+ plurals), letter-boundary anchored so
# "sketchbook" / "kadoodle" don't trip it. Deliberately conservative — recall is
# secondary because the soft source doesn't train the head and the ring-loud audit
# catches false positives.
_SOFT_WIP_RE = re.compile(
r"(?<![A-Za-z])(?:sketch|sketches|doodle|doodles|scribble|scribbles)(?![A-Za-z])",
re.IGNORECASE,
)
# Coarse SQL prefilters for the backfill sweep — narrow the post scan to rows that
# COULD match before the precise regex confirms. Case-insensitive ILIKE patterns.
# Each MUST stay a SUPERSET of its regex or the sweep would silently miss posts.
WIP_TITLE_SQL_PREFILTER = ("%wip%", "%work%progress%")
SOFT_WIP_TITLE_SQL_PREFILTER = ("%sketch%", "%doodle%", "%scribble%")
# Chunk bulk inserts so a large sweep can't blow past psycopg's 65535-parameter
# ceiling (3 params/row → ~21k rows max; 5k stays comfortably under).
_INSERT_CHUNK = 5000
def matches_wip_title(title: str | None) -> bool:
"""True when a post title explicitly marks it work-in-progress (HARD tier)."""
if not title:
return False
return _WIP_RE.search(title) is not None
def matches_soft_wip_title(title: str | None) -> bool:
"""True when a title carries a SOFT WIP cue (sketch/doodle/scribble, #1474)."""
if not title:
return False
return _SOFT_WIP_RE.search(title) is not None
def resolve_wip_tag_id(session: Session) -> int | None:
"""The seeded ``wip`` system tag's id (migration 0075), or None if absent."""
return session.execute(
select(Tag.id).where(Tag.name == WIP_SYSTEM_TAG, Tag.is_system.is_(True))
).scalar_one_or_none()
def apply_wip_image_tags(
session: Session, image_ids, tag_id: int, *, source: str = WIP_TITLE_SOURCE
) -> int:
"""Attach ``tag_id`` (stamped with ``source``) to each image id, idempotently —
never disturbs an existing tag or its source. Returns the number of image_tag
rows newly inserted. Does NOT commit.
The insert count is computed from a pre-SELECT of already-tagged ids rather
than the statement's ``rowcount``: psycopg reports -1 for a multi-row
ON CONFLICT DO NOTHING insert (it runs via an executemany path), so rowcount
is unusable here. The SELECT is accurate within this single transaction (no
concurrent writer touches these (image, wip) rows); ON CONFLICT DO NOTHING
stays as a race-safety belt so a rare concurrent insert can't error."""
ids = list({int(i) for i in image_ids})
if not ids:
return 0
inserted = 0
for start in range(0, len(ids), _INSERT_CHUNK):
chunk = ids[start:start + _INSERT_CHUNK]
already = set(session.execute(
select(image_tag.c.image_record_id)
.where(image_tag.c.tag_id == tag_id)
.where(image_tag.c.image_record_id.in_(chunk))
).scalars())
to_insert = [iid for iid in chunk if iid not in already]
if not to_insert:
continue
session.execute(
pg_insert(image_tag)
.values([
{"image_record_id": iid, "tag_id": tag_id, "source": source}
for iid in to_insert
])
.on_conflict_do_nothing(index_elements=["image_record_id", "tag_id"])
)
inserted += len(to_insert)
return inserted
+92
View File
@@ -76,6 +76,8 @@ DOWNLOAD_STALL_THRESHOLD_MINUTES = 30
OLD_TASK_DAYS = 7 OLD_TASK_DAYS = 7
PHASH_PAGE = 500 PHASH_PAGE = 500
VERIFY_PAGE = 200 VERIFY_PAGE = 200
# Title-based WIP backfill (task #1458): posts scanned per keyset page.
WIP_BACKFILL_PAGE = 500
FFPROBE_TIMEOUT_SECONDS = 10 FFPROBE_TIMEOUT_SECONDS = 10
TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h
TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days
@@ -1045,6 +1047,96 @@ def cleanup_old_download_events() -> int:
return result.rowcount or 0 return result.rowcount or 0
def _backfill_wip_tier(session, tag_id, prefilter, matcher, source) -> int:
"""One keyset-paginated pass over posts whose title matches a WIP tier, applying
`tag_id` (stamped `source`) to their images. Shared by the hard + soft tiers
(#1458 / #1474). Coarse `prefilter` (ILIKE superset) narrows the scan; the precise
`matcher` confirms. Idempotent-additive (ON CONFLICT DO NOTHING). Returns the row
count newly applied."""
from ..models import Post
from ..models.image_provenance import ImageProvenance
from ..services.wip_title import apply_wip_image_tags
applied = 0
last_id = 0
while True:
rows = session.execute(
select(Post.id, Post.post_title)
.where(Post.id > last_id)
.where(Post.post_title.is_not(None))
.where(or_(*[Post.post_title.ilike(p) for p in prefilter]))
.order_by(Post.id.asc())
.limit(WIP_BACKFILL_PAGE)
).all()
if not rows:
break
last_id = rows[-1][0]
match_ids = [pid for pid, title in rows if matcher(title)]
if match_ids:
image_ids = session.execute(
select(ImageProvenance.image_record_id)
.where(ImageProvenance.post_id.in_(match_ids))
).scalars().all()
applied += apply_wip_image_tags(session, image_ids, tag_id, source=source)
session.commit()
return applied
@celery.task(
name="backend.app.tasks.maintenance.backfill_wip_title_tags",
# Coarse-prefiltered scan over posts; the candidate set is small on a typical
# library, but bound it like the other full-library sweeps.
soft_time_limit=1800, time_limit=2100,
)
def backfill_wip_title_tags() -> int:
"""Scan EXISTING posts for WIP titles and apply the `wip` system tag to their
images — the operator-triggered back-catalogue catch-up (task #1458 hard tier +
#1474 soft tier). New imports are tagged live by the importer; this covers the
existing library.
HARD tier ("WIP"/"work in progress") always runs (the operator triggered the
scan); the SOFT tier (sketch/doodle, provisional source) runs only when
wip_soft_title_tagging_enabled, AFTER hard so a title matching both keeps the
trained hard tag (ON CONFLICT DO NOTHING). Keyset-paginated, restart-safe.
Deliberately NOT scheduled as a beat: a periodic re-run would re-apply to matching
posts and silently undo a manual WIP removal, so it stays an explicit operator
action (Settings → "Scan existing posts for WIP titles"). Returns rows applied.
"""
from ..models import ImportSettings
from ..services.wip_title import (
SOFT_WIP_TITLE_SQL_PREFILTER,
WIP_TITLE_SOFT_SOURCE,
WIP_TITLE_SOURCE,
WIP_TITLE_SQL_PREFILTER,
matches_soft_wip_title,
matches_wip_title,
resolve_wip_tag_id,
)
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
tag_id = resolve_wip_tag_id(session)
if tag_id is None:
log.warning(
"backfill_wip_title_tags: no `wip` system tag present; nothing to do"
)
return 0
settings = ImportSettings.load_sync(session)
applied = _backfill_wip_tier(
session, tag_id, WIP_TITLE_SQL_PREFILTER, matches_wip_title,
WIP_TITLE_SOURCE,
)
if settings.wip_soft_title_tagging_enabled:
applied += _backfill_wip_tier(
session, tag_id, SOFT_WIP_TITLE_SQL_PREFILTER, matches_soft_wip_title,
WIP_TITLE_SOFT_SOURCE,
)
if applied:
log.info("backfill_wip_title_tags: applied wip to %d image(s)", applied)
return applied
@celery.task(name="backend.app.tasks.maintenance.vacuum_analyze") @celery.task(name="backend.app.tasks.maintenance.vacuum_analyze")
def vacuum_analyze() -> dict: def vacuum_analyze() -> dict:
"""Periodic VACUUM (ANALYZE) over the high-churn tables (VACUUM_TABLES) to """Periodic VACUUM (ANALYZE) over the high-churn tables (VACUUM_TABLES) to
+42 -6
View File
@@ -599,18 +599,54 @@ def scheduled_ccip_auto_apply() -> str:
soft_time_limit=1800, time_limit=2100, soft_time_limit=1800, time_limit=2100,
) )
def scheduled_presentation_auto_apply() -> str: def scheduled_presentation_auto_apply() -> str:
"""Auto-hide presentation chrome (banner / editor screenshot) on a daily """Auto-hide presentation chrome (banner) on a daily passive sweep (#141).
passive sweep (#141). No-op unless presentation_auto_apply_enabled. Idempotent No-op unless presentation_auto_apply_enabled. Idempotent — already-tagged images
— already-hidden images are skipped — so an interrupted run simply re-runs next are skipped — so an interrupted run simply re-runs next cycle (that IS the
cycle (that IS the recovery). Wall-clock bounded by the task time limits.""" recovery). Wall-clock bounded by the task time limits."""
from ..services.ml.heads import presentation_auto_apply_sweep from ..services.ml.heads import system_tag_auto_apply_sweep
SessionLocal = _sync_session_factory() SessionLocal = _sync_session_factory()
with SessionLocal() as session: with SessionLocal() as session:
result = presentation_auto_apply_sweep(session) result = system_tag_auto_apply_sweep(session, mode="chrome")
return f"applied={result['n_applied']} flagged={result['n_flagged']}" return f"applied={result['n_applied']} flagged={result['n_flagged']}"
@celery.task(
name="backend.app.tasks.ml.scheduled_process_auto_apply",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_process_auto_apply() -> str:
"""Auto-apply the PROCESS system tags (wip / editor screenshot) on a daily
passive sweep (#1464) — provisional source, ring-loud review guard, image stays
VISIBLE. No-op unless process_auto_apply_enabled (opt-in). Idempotent —
already-tagged/rejected images are skipped — so an interrupted run just re-runs
next cycle (the recovery). Wall-clock bounded by the task time limits."""
from ..services.ml.heads import system_tag_auto_apply_sweep
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
result = system_tag_auto_apply_sweep(session, mode="process")
return f"applied={result['n_applied']} flagged={result['n_flagged']}"
@celery.task(
name="backend.app.tasks.ml.scheduled_soft_wip_conflict_audit",
soft_time_limit=1800, time_limit=2100,
)
def scheduled_soft_wip_conflict_audit() -> str:
"""Ring-loud audit over the SOFT WIP-title cohort (#1474) — flag sketch/doodle
auto-tags that ALSO look like real content for review. No-op when there are no
content heads; idempotent (already-flagged images skipped). Runs regardless of
the process-sweep toggle, since soft-title tags come from the importer, not that
sweep. Wall-clock bounded by the task time limits."""
from ..services.ml.heads import soft_wip_conflict_audit
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
result = soft_wip_conflict_audit(session)
return f"scanned={result['n_scanned']} flagged={result['n_flagged']}"
@celery.task(name="backend.app.tasks.ml.prune_presentation_reviews") @celery.task(name="backend.app.tasks.ml.prune_presentation_reviews")
def prune_presentation_reviews() -> str: def prune_presentation_reviews() -> str:
"""Retention (rule 89): drop RESOLVED presentation-review flags older than 30 """Retention (rule 89): drop RESOLVED presentation-review flags older than 30
+165 -45
View File
@@ -6,9 +6,16 @@ viewing is instant and re-runs are cache-fast. Sync (Celery workers are sync),
same pattern as the other backfill sweeps. No-op unless translation is enabled, same pattern as the other backfill sweeps. No-op unless translation is enabled,
a base URL is set, and the service is healthy. a base URL is set, and the service is healthy.
Curator does NO language detection of its own: each field's translation is
accepted or rejected purely on Interpreter's reported engine + detected language
+ confidence (see ``_accept``) — a short English title Interpreter mis-labels as
a European language scores below the latin-script floor and is kept as-is.
Two entry points: Two entry points:
- ``translate_posts`` — the daily untranslated sweep (translated_source_lang IS - ``translate_posts`` — the periodic untranslated sweep (every 8h;
NULL only); also the Settings "Translate now" button. translated_source_lang IS NULL only), one bounded chunk per fire. The Settings
"Translate now" button calls it with ``drain=True`` to chase the tail
run-until-done and clear the whole backlog in a single press.
- ``retranslate_posts`` — after a model change, resets the stored translation - ``retranslate_posts`` — after a model change, resets the stored translation
columns for a scoped set of posts (all, or a given set of artists) so the columns for a scoped set of posts (all, or a given set of artists) so the
untranslated selection re-runs them, then chases the tail until drained untranslated selection re-runs them, then chases the tail until drained
@@ -43,6 +50,28 @@ _RETRANSLATE_COUNTDOWN = 5
_INTERRUPT_BACKOFF = 60 _INTERRUPT_BACKOFF = 60
_INTERRUPT_BACKOFF_MAX = 900 _INTERRUPT_BACKOFF_MAX = 900
# --- Translation acceptance gate (Scribe rule 133; contract note #1347) -------
# Curator does NO language detection of its own — it accepts or rejects a
# translation purely on Interpreter's reported detection. CJK is script-detected
# (kana/hangul/kanji) and reliably high-confidence, so ja/ko/zh are trusted
# outright (pure-kanji Japanese can even come back labelled zh ~0.75, still
# correctly translated). Latin-script detection is a real statistical
# probability, so a short English title Interpreter mis-labels as a European
# language scores low; require it to clear this floor, else keep the original.
# Calibrated 2026-07-09 against fresh (uncached) probes once Interpreter returned
# real langdetect confidence: genuine German detected at 1.0, while a correctly-
# detected but ambiguous latin string dipped to 0.86 — so a single constant can't win.
# The floor is now operator-tunable (ImportSettings.translation_min_confidence,
# milestone 155) with a stricter 0.90 default; per-post overrides
# (Post.translation_override) rescue the false negatives it skips. Genuine German
# and mis-flagged short English both land ~0.86, and single-word mis-flags sit at
# a confident 1.0 no floor catches. Real langdetect fixed some cases at the
# source (some short English titles now detect as English → passthrough), so this
# floor is a safety net, not the primary fix. Re-tune via the "Test translation"
# box (send fresh text — cache hits report 1.0).
_CJK_LANGS = frozenset({"ja", "ko", "zh"})
_DEFAULT_MIN_LATIN_CONFIDENCE = 0.90
def _interrupt_backoff(retry_after) -> int: def _interrupt_backoff(retry_after) -> int:
"""Seconds to wait before resuming after an Interpreter interruption: the """Seconds to wait before resuming after an Interpreter interruption: the
@@ -56,31 +85,51 @@ def _interrupt_backoff(retry_after) -> int:
name="backend.app.tasks.translation.translate_posts", name="backend.app.tasks.translation.translate_posts",
soft_time_limit=1800, time_limit=2100, soft_time_limit=1800, time_limit=2100,
) )
def translate_posts() -> str: def translate_posts(drain: bool = False) -> str:
"""Translate untranslated non-English post title/description (default target """Translate untranslated non-English post title/description (default target
en) via Interpreter. Per-post [title, description] batch → per-post detected en) via Interpreter. Per-post [title, description] batch → per-post detected
language. Passthrough / already-target posts are marked handled (source == language. Passthrough / already-target posts are marked handled (source ==
target) with no stored translation. 503 or a connection error interrupts the target) with no stored translation. 503 or a connection error interrupts the
run (retry next cycle); 400 stops it (fix config); posts done so far stay run (retry next cycle); 400 stops it (fix config); posts done so far stay
committed. No-op unless configured + healthy. Returns a summary string.""" committed. No-op unless configured + healthy. Returns a summary string.
``drain`` (the Settings "Translate now" button) chases the tail
run-until-done: on a clean chunk with work still remaining it re-enqueues
itself until the untranslated backlog is zero, like ``retranslate_posts``. The
periodic beat leaves it False → one bounded chunk per fire (the 8h cadence
drives the rest, enough for the trickle of newly-imported posts)."""
SessionLocal = _sync_session_factory() SessionLocal = _sync_session_factory()
with SessionLocal() as session: with SessionLocal() as session:
ready = _translation_config(session) ready = _translation_config(session)
if isinstance(ready, str): if isinstance(ready, str):
return ready return ready
base_url, target = ready base_url, target, min_confidence = ready
posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN) posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch( status, translated, retry_after = _translate_batch(
session, posts, base_url, target, session, posts, base_url, target, min_confidence,
) )
# Steady-state daily sweep: one chunk per run on success (the beat drives # Drain mode (manual "Translate now"): chase the tail until the backlog is
# the rest — no tail-chase). But if Interpreter drained mid-chunk, don't # zero, so one press clears the whole pile instead of one 300-chunk.
# make a manual "Translate now" wait a whole day — re-enqueue after its # Termination is guaranteed — every handled post leaves the untranslated
# Retry-After hint (or the default backoff). Self-terminating: once the # set, so the remaining count strictly decreases each chunk.
# service is down the health gate returns "interpreter unavailable" early. if status == "ok" and drain:
if status == "interrupted" and _count_untranslated(session, None): remaining = _count_untranslated(session, None)
if remaining:
translate_posts.apply_async(
(), {"drain": True}, countdown=_RETRANSLATE_COUNTDOWN,
)
log.info(
"translate_posts: draining — %d remaining → re-enqueued",
remaining,
)
# Interpreter drained mid-chunk (graceful restart): don't make a manual
# "Translate now" wait a whole beat cycle — re-enqueue after its
# Retry-After hint (or the default backoff), preserving `drain` so a bulk
# drain resumes cleanly. Self-terminating: once the service is down the
# health gate returns "interpreter unavailable" early.
elif status == "interrupted" and _count_untranslated(session, None):
delay = _interrupt_backoff(retry_after) delay = _interrupt_backoff(retry_after)
translate_posts.apply_async((), {}, countdown=delay) translate_posts.apply_async((), {"drain": drain}, countdown=delay)
log.info( log.info(
"translate_posts: interrupted (service draining) → retry in %ss", "translate_posts: interrupted (service draining) → retry in %ss",
delay, delay,
@@ -106,7 +155,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
ready = _translation_config(session) ready = _translation_config(session)
if isinstance(ready, str): if isinstance(ready, str):
return ready return ready
base_url, target = ready base_url, target, min_confidence = ready
if not _reset_done: if not _reset_done:
n = _reset_translations(session, artist_ids) n = _reset_translations(session, artist_ids)
@@ -117,7 +166,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN) posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN)
status, translated, retry_after = _translate_batch( status, translated, retry_after = _translate_batch(
session, posts, base_url, target, session, posts, base_url, target, min_confidence,
) )
# run-until-done: chase the tail when the chunk finished cleanly. # run-until-done: chase the tail when the chunk finished cleanly.
@@ -155,17 +204,19 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
def _translation_config(session): def _translation_config(session):
"""Resolve (base_url, target) if translation is enabled + healthy, else a """Resolve (base_url, target, min_confidence) if translation is enabled +
short status string ("disabled" / "interpreter unavailable") for the caller healthy, else a short status string ("disabled" / "interpreter unavailable")
to return. Centralises the guard so translate/retranslate agree exactly.""" for the caller to return. Centralises the guard so translate/retranslate agree
exactly, including the operator-tunable acceptance floor."""
cfg = ImportSettings.load_sync(session) cfg = ImportSettings.load_sync(session)
if not cfg.translation_enabled or not cfg.interpreter_base_url.strip(): if not cfg.translation_enabled or not cfg.interpreter_base_url.strip():
return "disabled" return "disabled"
base_url = cfg.interpreter_base_url.strip() base_url = cfg.interpreter_base_url.strip()
target = (cfg.translation_target_lang or "en").strip() or "en" target = (cfg.translation_target_lang or "en").strip() or "en"
min_confidence = cfg.translation_min_confidence
if not ic.health(base_url): if not ic.health(base_url):
return "interpreter unavailable" return "interpreter unavailable"
return base_url, target return base_url, target, min_confidence
def _untranslated_filter(stmt, artist_ids): def _untranslated_filter(stmt, artist_ids):
@@ -198,10 +249,13 @@ def _reset_translations(session, artist_ids) -> int:
"""Clear the stored translation columns for scoped, already-handled posts so """Clear the stored translation columns for scoped, already-handled posts so
the untranslated sweep re-runs them. Only touches rows that were translated the untranslated sweep re-runs them. Only touches rows that were translated
(translated_source_lang IS NOT NULL) — untranslated rows are already NULL. (translated_source_lang IS NOT NULL) — untranslated rows are already NULL.
Returns the row count reset (commits it).""" Skips 'keep original' posts (translation_override = 'original') so that choice
survives a Re-translate-all (milestone 155). Returns the row count reset
(commits it)."""
stmt = ( stmt = (
update(Post) update(Post)
.where(Post.translated_source_lang.is_not(None)) .where(Post.translated_source_lang.is_not(None))
.where(Post.translation_override != "original")
.values( .values(
post_title_translated=None, post_title_translated=None,
description_translated=None, description_translated=None,
@@ -217,7 +271,7 @@ def _reset_translations(session, artist_ids) -> int:
return n return n
def _translate_batch(session, posts, base_url: str, target: str): def _translate_batch(session, posts, base_url: str, target: str, min_confidence: float):
"""Translate each post in the chunk with a per-post commit (an interrupted """Translate each post in the chunk with a per-post commit (an interrupted
run keeps progress). Returns (status, translated, retry_after) where status is run keeps progress). Returns (status, translated, retry_after) where status is
one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout"; one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout";
@@ -226,7 +280,7 @@ def _translate_batch(session, posts, base_url: str, target: str):
translated = 0 translated = 0
for post in posts: for post in posts:
try: try:
translated += _translate_one(session, post, base_url, target) translated += _translate_one(session, post, base_url, target, min_confidence)
session.commit() session.commit()
except ic.InterpreterUnavailable as e: except ic.InterpreterUnavailable as e:
session.rollback() session.rollback()
@@ -250,29 +304,95 @@ def _summary(status: str, translated: int, scanned: int) -> str:
return f"translated={translated} scanned={scanned}" return f"translated={translated} scanned={scanned}"
def _translate_one(session, post, base_url: str, target: str) -> int: def _accept(
"""Translate one post's title/description in place. Returns 1 if it stored a language: str, confidence, min_confidence: float = _DEFAULT_MIN_LATIN_CONFIDENCE,
translation, 0 for passthrough/empty (which still marks the post handled).""" ) -> bool:
"""Should curator store Interpreter's translation, or keep the original?
Consumes ONLY Interpreter's own detection (curator does none, Scribe rule
133): CJK languages are trusted regardless of the reported number; a
latin-script detection must clear ``min_confidence`` (the operator-tunable
ImportSettings.translation_min_confidence). A missing confidence fails OPEN
(accept), so a service that omits the field never silently drops a
translation. Per-post ``force`` overrides bypass this entirely (see
``_translate_one``)."""
if (language or "").split("-", 1)[0].lower() in _CJK_LANGS:
return True
if confidence is None:
return True
return confidence >= min_confidence
def _translate_field(
text: str, base_url: str, target: str, min_confidence: float, force: bool = False,
):
"""Translate ONE field independently. Returns (translated, source_lang,
engine_version), or (None, None, None) when there's nothing to store — empty
text, already the target language, a passthrough (engine "none"), or a
latin-script detection the acceptance gate rejects as too low-confidence (kept
as the original). ``force`` (a per-post 'force' override) bypasses the
confidence gate so a legitimately-foreign title Interpreter is only ~0.86 sure
about is still stored; a genuine passthrough stores nothing regardless (there
is no translation to force). Per-field (not aggregate first-item) detection so
a non-English description still translates when the title is already English."""
if not text:
return None, None, None
res = ic.translate([text], base_url=base_url, target=target)
detected = res["detected_lang"] or target
if detected == target or res["engine"] == "none":
return None, None, None
# Trust only Interpreter's own detection (Scribe rule 133): keep the original
# when a latin-script detection doesn't clear the confidence floor, so a short
# English title mis-labelled as e.g. German isn't rewritten into the archive.
# A 'force' override skips the gate — the operator is overriding a rejection.
if not force and not _accept(
detected, res.get("detected_confidence"), min_confidence,
):
return None, None, None
return res["translations"][0], detected, res["engine_version"]
def _store_translation(post, title_res, desc_res, target: str) -> int:
"""Apply per-field (translated, source_lang, engine_version) results to a post
in place. Returns 1 if a translation was stored, 0 when the post is all
passthrough/empty — in which case the translated columns are CLEARED and the
post is marked handled (translated_source_lang = target). Clearing (not just
leaving) matters on re-apply: a 'keep original' override, or a re-translate
that now rejects a previously-accepted mis-flag, must remove the stale
translation, not keep it."""
title_tr, title_lang, title_ev = title_res
desc_tr, desc_lang, desc_ev = desc_res
if title_tr is None and desc_tr is None:
post.post_title_translated = None
post.description_translated = None
post.translated_source_lang = target
post.translation_engine_version = None
post.translated_at = None
return 0
post.post_title_translated = title_tr
post.description_translated = desc_tr
# Source lang = whichever field was actually non-target (for a mixed post,
# the translated field's language, not the already-English one).
post.translated_source_lang = title_lang or desc_lang
post.translation_engine_version = title_ev or desc_ev
post.translated_at = datetime.now(UTC)
return 1
def _translate_one(
session, post, base_url: str, target: str, min_confidence: float,
) -> int:
"""Translate a post's title/description in place, each field independently,
honoring the per-post override (milestone 155): 'original' keeps the original
(clears any stored translation, no Interpreter call); 'force' translates even
below the confidence floor; 'auto' (default) runs the acceptance gate. Returns
1 if it stored a translation, else 0 (still marks the post handled so the sweep
won't revisit it)."""
if post.translation_override == "original":
return _store_translation(post, (None, None, None), (None, None, None), target)
force = post.translation_override == "force"
title = (post.post_title or "").strip() title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or "" desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip() desc = desc.strip()
fields: list[tuple[str, str]] = [] title_res = _translate_field(title, base_url, target, min_confidence, force=force)
if title: desc_res = _translate_field(desc, base_url, target, min_confidence, force=force)
fields.append(("title", title)) return _store_translation(post, title_res, desc_res, target)
if desc:
fields.append(("desc", desc))
if not fields:
post.translated_source_lang = target # nothing to translate → handled
return 0
res = ic.translate([t for _, t in fields], base_url=base_url, target=target)
detected = res["detected_lang"] or target
if detected == target or res["engine"] == "none":
post.translated_source_lang = target # already target language → handled
return 0
by_field = {f: res["translations"][i] for i, (f, _) in enumerate(fields)}
post.post_title_translated = by_field.get("title")
post.description_translated = by_field.get("desc")
post.translated_source_lang = detected
post.translation_engine_version = res["engine_version"]
post.translated_at = datetime.now(UTC)
return 1
+5 -7
View File
@@ -10,8 +10,7 @@
"dev": "vite", "dev": "vite",
"build": "vite build", "build": "vite build",
"preview": "vite preview", "preview": "vite preview",
"test:unit": "vitest run", "test:unit": "vitest run"
"check": "vue-tsc --noEmit"
}, },
"dependencies": { "dependencies": {
"vue": "^3.4.0", "vue": "^3.4.0",
@@ -21,13 +20,12 @@
"@mdi/font": "^7.4.0" "@mdi/font": "^7.4.0"
}, },
"devDependencies": { "devDependencies": {
"@vitejs/plugin-vue": "^5.0.0", "@vitejs/plugin-vue": "^6.0.0",
"vite": "^5.2.0", "vite": "^8.0.0",
"vue-tsc": "^2.0.0",
"vite-plugin-vuetify": "^2.0.0", "vite-plugin-vuetify": "^2.0.0",
"sass": "^1.71.0", "sass": "^1.71.0",
"vitest": "^2.1.0", "vitest": "^4.0.0",
"@vue/test-utils": "^2.4.0", "@vue/test-utils": "^2.4.0",
"happy-dom": "^15.0.0" "happy-dom": "^20.0.0"
} }
} }
@@ -1,14 +1,15 @@
<template> <template>
<!-- Auto-hidden chrome that ALSO looked like real content surfaced PROACTIVELY <!-- System-tag auto-applies (chrome hides / process WIP tags) that ALSO looked
atop the gallery whenever there's something to review (NOT gated on the like real content surfaced PROACTIVELY atop the gallery whenever there's
Show-hidden toggle, so misfires can't go unnoticed), most-concerning first, something to review (NOT gated on the Show-hidden toggle, so misfires can't
with keep / un-hide (#141). Renders nothing when there's nothing to review. --> go unnoticed), most-concerning first, with keep / remove (#141, #1464).
<section v-if="items.length" class="fc-review" aria-label="Hidden images to review"> Renders nothing when there's nothing to review. -->
<section v-if="items.length" class="fc-review" aria-label="Auto-tagged images to review">
<div class="fc-review__head"> <div class="fc-review__head">
<v-icon size="18" color="warning">mdi-alert-outline</v-icon> <v-icon size="18" color="warning">mdi-alert-outline</v-icon>
<span class="fc-review__title"> <span class="fc-review__title">
{{ items.length }} auto-hidden {{ items.length === 1 ? 'image' : 'images' }} {{ items.length }} auto-tagged {{ items.length === 1 ? 'image' : 'images' }}
may be real content — review before they stay hidden may be real content — review
</span> </span>
</div> </div>
<div class="fc-review__cards"> <div class="fc-review__cards">
@@ -26,16 +27,16 @@
> >
also looks like <strong>{{ it.conflict_name || 'content' }}</strong> also looks like <strong>{{ it.conflict_name || 'content' }}</strong>
</div> </div>
<div class="fc-review-card__tag">hidden as {{ it.tag_name }}</div> <div class="fc-review-card__tag">{{ tagLine(it) }}</div>
<div class="fc-review-card__acts"> <div class="fc-review-card__acts">
<button <button
type="button" class="fc-review-btn fc-review-btn--keep" type="button" class="fc-review-btn fc-review-btn--keep"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')" :disabled="busy.includes(keyOf(it))" @click="resolve(it, 'keep')"
>Keep hidden</button> >{{ keepLabel(it) }}</button>
<button <button
type="button" class="fc-review-btn fc-review-btn--unhide" type="button" class="fc-review-btn fc-review-btn--unhide"
:disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')" :disabled="busy.includes(keyOf(it))" @click="resolve(it, 'unhide')"
>Un-hide</button> >{{ removeLabel(it) }}</button>
</div> </div>
</div> </div>
</div> </div>
@@ -54,6 +55,11 @@ const items = ref([])
const busy = ref([]) const busy = ref([])
function keyOf(it) { return `${it.image_id}:${it.tag_id}` } function keyOf(it) { return `${it.image_id}:${it.tag_id}` }
// Chrome flags hide the image (keep-hidden / un-hide); process flags leave it
// visible and just tagged (keep-tag / remove-tag). Same endpoints, different words.
function tagLine(it) { return (it.mode === 'process' ? 'auto-tagged ' : 'hidden as ') + it.tag_name }
function keepLabel(it) { return it.mode === 'process' ? 'Keep tag' : 'Keep hidden' }
function removeLabel(it) { return it.mode === 'process' ? 'Remove tag' : 'Un-hide' }
async function load() { async function load() {
// Fetched unconditionally on mount — the strip prompts for pending misfires // Fetched unconditionally on mount — the strip prompts for pending misfires
@@ -71,7 +77,8 @@ async function resolve(it, action) {
await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`) await api.post(`/api/gallery/hidden-review/${it.image_id}/${it.tag_id}/${action}`)
items.value = items.value.filter((x) => keyOf(x) !== k) items.value = items.value.filter((x) => keyOf(x) !== k)
if (action === 'unhide') { if (action === 'unhide') {
toast({ text: `Un-hidden — “${it.tag_name}” removed; it'll train the head`, type: 'success' }) const verb = it.mode === 'process' ? 'Removed' : 'Un-hidden'
toast({ text: `${verb} — “${it.tag_name}” removed; it'll train the head`, type: 'success' })
} }
} catch (e) { } catch (e) {
toast({ toast({
@@ -1,72 +0,0 @@
<template>
<v-card>
<v-card-title>Treat as alias for</v-card-title>
<v-card-text>
<p class="text-caption mb-2">
Pick the existing tag the model's prediction should map to. All
future predictions of this name will resolve to your tag.
</p>
<v-autocomplete
v-model="selectedId"
v-model:menu="menuOpen"
:items="results"
:item-title="(t) => t.fandom_name ? `${t.name} — ${t.fandom_name}` : t.name"
:item-value="(t) => t.id"
:loading="loading"
label="Canonical tag"
no-filter clearable density="compact"
@update:search="onSearch"
@keydown.enter.capture="onEnter"
/>
</v-card-text>
<v-card-actions>
<v-spacer />
<v-btn @click="$emit('cancel')">Cancel</v-btn>
<v-btn
color="primary" rounded="pill" :disabled="!selectedId"
@click="$emit('confirm', selectedId)"
>Use this tag</v-btn>
</v-card-actions>
</v-card>
</template>
<script setup>
import { ref } from 'vue'
import { useApi } from '../../composables/useApi.js'
import { useAcceptOnEnter } from '../../composables/useAcceptOnEnter.js'
const props = defineProps({ category: { type: String, required: true } })
const emit = defineEmits(['confirm', 'cancel'])
const api = useApi()
const results = ref([])
const loading = ref(false)
const selectedId = ref(null)
let debounce = null
// Enter on the closed dropdown confirms the selection instead of re-opening it.
const { menuOpen, onEnter } = useAcceptOnEnter(() => {
if (selectedId.value != null) emit('confirm', selectedId.value)
})
function onSearch(q) {
if (debounce) clearTimeout(debounce)
debounce = setTimeout(async () => {
const query = (q || '').trim()
if (!query) { results.value = []; return }
loading.value = true
try {
// Scope the autocomplete to the prediction's category where it
// maps to a tag kind. Only 'character' surfaces as both a
// suggestion category and a tag kind now ('artist' + 'copyright'
// retired); other categories search unscoped.
const kind = props.category === 'character' ? 'character' : null
const params = { q: query, limit: 20 }
if (kind) params.kind = kind
results.value = await api.get('/api/tags/autocomplete', { params })
} finally {
loading.value = false
}
}, 200)
}
</script>
@@ -11,10 +11,6 @@
{{ suggestion.display_name }} {{ suggestion.display_name }}
<span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag" <span v-if="suggestion.rejected" class="fc-suggestion__rejected-tag"
title="You rejected this for this image — un-reject to recover">rejected</span> title="You rejected this for this image — un-reject to recover">rejected</span>
<span v-else-if="suggestion.creates_new_tag" class="fc-suggestion__new"
title="No matching tag yet — accepting creates it">+ new</span>
<span v-else-if="suggestion.via_alias" class="fc-suggestion__alias"
:title="`Mapped from the tagger's “${suggestion.raw_name}” via an alias`">alias</span>
</span> </span>
<span class="fc-suggestion__score">{{ scorePct }}</span> <span class="fc-suggestion__score">{{ scorePct }}</span>
<!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval <!-- Green / red pair (operator-asked 2026-06-28) mirrors the eval
@@ -46,40 +42,14 @@
@click="$emit('dismiss', suggestion)" @click="$emit('dismiss', suggestion)"
><v-icon size="16">mdi-close</v-icon></button> ><v-icon size="16">mdi-close</v-icon></button>
</div> </div>
<!-- Modal-safe kebab is baked into KebabMenu (this row lives in the
teleported image modal — #711). Only rendered when an alias action
applies — dismiss now lives on the red ✗, so a centroid hit with no
alias option has no menu. -->
<KebabMenu
v-if="hasMenu"
class="fc-suggestion__menu" size="small" variant="outlined"
:label="`More actions for ${suggestion.display_name}`"
>
<!-- Alias is a tagger-prediction remap, so only offer it for tagger
suggestions with a raw model key that aren't already aliased.
Centroid hits (raw_name null) have nothing to alias. -->
<v-list-item
v-if="suggestion.raw_name && !suggestion.via_alias"
@click="$emit('alias', suggestion)"
>
<v-list-item-title>Treat as alias for</v-list-item-title>
</v-list-item>
<v-list-item
v-if="suggestion.via_alias"
@click="$emit('remove-alias', suggestion)"
>
<v-list-item-title>Remove alias</v-list-item-title>
</v-list-item>
</KebabMenu>
</div> </div>
</template> </template>
<script setup> <script setup>
import { computed, inject } from 'vue' import { computed, inject } from 'vue'
import KebabMenu from '../common/KebabMenu.vue'
const props = defineProps({ suggestion: { type: Object, required: true } }) const props = defineProps({ suggestion: { type: Object, required: true } })
defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss']) defineEmits(['accept', 'dismiss', 'undismiss'])
// #1206: on hover, tell the image viewer which crop produced this suggestion so // #1206: on hover, tell the image viewer which crop produced this suggestion so
// it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere. // it highlights that region. Provided by ImageViewer/Explore; a no-op elsewhere.
@@ -92,12 +62,6 @@ function onLeave () {
} }
const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`) const scorePct = computed(() => `${Math.round(props.suggestion.score * 100)}%`)
// Kebab now only carries alias actions: show it when this suggestion can be
// aliased (raw model key, not yet aliased) or is already aliased (so it can be
// un-aliased). Centroid hits (no raw_name, no alias) have an empty menu → hide.
const hasMenu = computed(() =>
Boolean(props.suggestion.raw_name) || Boolean(props.suggestion.via_alias)
)
</script> </script>
<style scoped> <style scoped>
@@ -119,26 +83,6 @@ const hasMenu = computed(() =>
color: rgb(var(--v-theme-on-surface)); color: rgb(var(--v-theme-on-surface));
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
} }
.fc-suggestion__new {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-accent));
background: rgba(var(--v-theme-accent), 0.12);
border: 1px solid rgb(var(--v-theme-accent), 0.4);
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__alias {
display: inline-block;
font-size: 10px; font-weight: 600;
color: rgb(var(--v-theme-on-surface-variant));
background: rgb(var(--v-theme-surface-light));
border: 1px solid rgb(var(--v-theme-surface-light));
padding: 1px 6px; border-radius: 999px;
margin-left: 6px;
text-transform: uppercase; letter-spacing: 0.04em;
}
.fc-suggestion__score { .fc-suggestion__score {
flex: 0 0 auto; min-width: 38px; text-align: right; flex: 0 0 auto; min-width: 38px; text-align: right;
font-size: 11px; font-size: 11px;
@@ -166,10 +110,6 @@ const hasMenu = computed(() =>
background: transparent; color: rgb(var(--v-theme-on-surface-variant)); background: transparent; color: rgb(var(--v-theme-on-surface-variant));
border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5); border: 1px solid rgb(var(--v-theme-on-surface-variant), 0.5);
} }
.fc-suggestion__menu {
flex: 0 0 auto;
}
/* Rejected state: the row stays put (recovery), dimmed + red-edged so it /* Rejected state: the row stays put (recovery), dimmed + red-edged so it
reads as "handled, negative" without shouting over live suggestions. */ reads as "handled, negative" without shouting over live suggestions. */
.fc-suggestion--rejected { .fc-suggestion--rejected {
@@ -26,8 +26,6 @@
v-for="(s, i) in items" :key="`${s.display_name}-${i}`" v-for="(s, i) in items" :key="`${s.display_name}-${i}`"
:suggestion="s" :suggestion="s"
@accept="$emit('accept', $event)" @accept="$emit('accept', $event)"
@alias="$emit('alias', $event)"
@remove-alias="$emit('remove-alias', $event)"
@dismiss="$emit('dismiss', $event)" @dismiss="$emit('dismiss', $event)"
@undismiss="$emit('undismiss', $event)" @undismiss="$emit('undismiss', $event)"
/> />
@@ -45,7 +43,7 @@ const props = defineProps({
collapsible: { type: Boolean, default: false }, collapsible: { type: Boolean, default: false },
defaultOpen: { type: Boolean, default: true } defaultOpen: { type: Boolean, default: true }
}) })
defineEmits(['accept', 'alias', 'remove-alias', 'dismiss', 'undismiss', 'reject-all']) defineEmits(['accept', 'dismiss', 'undismiss', 'reject-all'])
// Still-unhandled suggestions (not yet rejected) — how many "Reject rest" clears. // Still-unhandled suggestions (not yet rejected) — how many "Reject rest" clears.
const rejectableCount = computed(() => props.items.filter((s) => !s.rejected).length) const rejectableCount = computed(() => props.items.filter((s) => !s.rejected).length)
@@ -20,48 +20,39 @@
first, so false positives are easy to spot and reject (reject-to-train first, so false positives are easy to spot and reject (reject-to-train
for these heads). Small set collapsible, open by default. --> for these heads). Small set collapsible, open by default. -->
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.byCategory.system && store.byCategory.system.length" v-if="store.aboveByCategory.system && store.aboveByCategory.system.length"
label="System" :items="store.byCategory.system" label="System" :items="store.aboveByCategory.system"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('system')" @reject-all="onRejectAll('system')"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-for="cat in peopleCats" :key="cat" v-for="cat in peopleCats" :key="cat"
v-show="store.byCategory[cat] && store.byCategory[cat].length" v-show="store.aboveByCategory[cat] && store.aboveByCategory[cat].length"
:label="labelFor(cat)" :items="store.byCategory[cat] || []" :label="labelFor(cat)" :items="store.aboveByCategory[cat] || []"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll(cat)" @reject-all="onRejectAll(cat)"
/> />
<SuggestionsCategoryGroup <SuggestionsCategoryGroup
v-if="store.byCategory.general && store.byCategory.general.length" v-if="store.aboveByCategory.general && store.aboveByCategory.general.length"
label="General" :items="store.byCategory.general" label="General" :items="store.aboveByCategory.general"
collapsible :default-open="true" collapsible :default-open="true"
@accept="onAccept" @alias="onAlias" @remove-alias="onRemoveAlias" @accept="onAccept"
@dismiss="onDismiss" @undismiss="onUndismiss" @dismiss="onDismiss" @undismiss="onUndismiss"
@reject-all="onRejectAll('general')" @reject-all="onRejectAll('general')"
/> />
</div> </div>
<v-dialog v-model="aliasDialog" max-width="480">
<AliasPickerDialog
v-if="aliasTarget"
:category="aliasTarget.category"
@confirm="onAliasConfirm" @cancel="aliasDialog = false"
/>
</v-dialog>
</section> </section>
</template> </template>
<script setup> <script setup>
import { toast } from '../../utils/toast.js' import { toast } from '../../utils/toast.js'
import { computed, ref, watch } from 'vue' import { computed, watch } from 'vue'
import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js' import { useSuggestionsStore, CATEGORY_LABELS } from '../../stores/suggestions.js'
import { useModalStore } from '../../stores/modal.js' import { useModalStore } from '../../stores/modal.js'
import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue' import SuggestionsCategoryGroup from './SuggestionsCategoryGroup.vue'
import AliasPickerDialog from './AliasPickerDialog.vue'
const props = defineProps({ const props = defineProps({
imageId: { type: Number, required: true }, imageId: { type: Number, required: true },
@@ -100,13 +91,14 @@ const peopleCats = ['character']
function labelFor(c) { return CATEGORY_LABELS[c] || c } function labelFor(c) { return CATEGORY_LABELS[c] || c }
const isEmpty = computed(() => const isEmpty = computed(() =>
Object.values(store.byCategory).every(list => !list || list.length === 0) Object.values(store.aboveByCategory).every(list => !list || list.length === 0)
) )
watch(() => props.imageId, (id) => { watch(() => props.imageId, (id) => {
if (id == null) return if (id == null) return
store.load(id) // panel: curated, ≥ threshold // One fetch (min=0) backs both surfaces: the panel reads aboveByCategory, the
store.loadAll(id) // dropdown: full prediction set (low-confidence included) // typed tag-input dropdown reads the full set. No second request.
store.load(id)
}, { immediate: true }) }, { immediate: true })
// After a successful accept/alias-accept, refresh the modal's current // After a successful accept/alias-accept, refresh the modal's current
@@ -125,31 +117,6 @@ async function onAccept(s) {
toast({ text: `Accept failed: ${e.message}`, type: 'error' }) toast({ text: `Accept failed: ${e.message}`, type: 'error' })
} }
} }
const aliasDialog = ref(false)
const aliasTarget = ref(null)
function onAlias(s) { aliasTarget.value = s; aliasDialog.value = true }
async function onAliasConfirm(canonicalTagId) {
try {
await store.aliasAccept(aliasTarget.value, canonicalTagId)
aliasDialog.value = false
await host.reloadTags()
emit('accepted')
} catch (e) {
toast({ text: `Alias failed: ${e.message}`, type: 'error' })
}
}
// Undo the model-key→tag mapping behind an aliased suggestion. The store
// reloads suggestions so the prediction reverts to its raw form; the applied
// canonical tag (if any) stays, so no tag-rail reload is needed.
async function onRemoveAlias(s) {
try {
await store.removeAlias(s)
} catch (e) {
toast({ text: `Remove alias failed: ${e.message}`, type: 'error' })
}
}
</script> </script>
<style scoped> <style scoped>
@@ -17,7 +17,14 @@
density="compact" class="fc-tag-autocomplete__list" density="compact" class="fc-tag-autocomplete__list"
> >
<template v-for="(row, idx) in rows" :key="row.key"> <template v-for="(row, idx) in rows" :key="row.key">
<!-- Existing tag match (server autocomplete). --> <!-- One unified row per DB tag (server autocomplete). Every suggestion is
a canonical tag now, so when the model ALSO scored this tag for the
image the row just carries its confidence (🔧 %) in place of the
redundant kind label the coloured leading icon already shows kind.
That means a searched tag that's also a suggestion shows its % on ONE
row, with no dedup and no flicker. Picking it emits pick-existing;
TagPanel.findPending routes a matching suggestion through accept() so
the acceptance is still recorded + it drops from the panel. -->
<v-list-item <v-list-item
v-if="row.type === 'hit'" v-if="row.type === 'hit'"
:active="idx === highlight" @click="onPickRow(row)" :active="idx === highlight" @click="onPickRow(row)"
@@ -32,32 +39,18 @@
<span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span> <span v-if="row.hit.fandom_name" class="text-caption"> {{ row.hit.fandom_name }}</span>
</v-list-item-title> </v-list-item-title>
<template #append> <template #append>
<span class="text-caption">{{ row.hit.kind }}</span> <span
</template> v-if="row.sugg"
</v-list-item> class="fc-tag-autocomplete__sugg-tag"
:class="{ 'fc-tag-autocomplete__sugg-tag--below': !row.sugg.above_threshold }"
<!-- ML suggestion for THIS image that matches the typed query. Picking :title="row.sugg.above_threshold
it routes through the same accept path as the Suggestions panel, so ? 'The model suggests this tag for this image'
it's recorded + drops out of the panel (operator-asked 2026-06-07). --> : 'The model scored this tag below its suggest threshold'"
<v-list-item >
v-else-if="row.type === 'suggestion'"
:active="idx === highlight" @click="onPickRow(row)"
class="fc-tag-autocomplete__sugg"
>
<template #prepend>
<v-icon size="small" :color="store.colorFor(row.sugg.category)">
{{ iconFor(row.sugg.category) }}
</v-icon>
</template>
<v-list-item-title>
{{ row.sugg.display_name }}
<span v-if="row.sugg.creates_new_tag" class="text-caption">— new</span>
</v-list-item-title>
<template #append>
<span class="fc-tag-autocomplete__sugg-tag">
<v-icon size="x-small">mdi-auto-fix</v-icon> <v-icon size="x-small">mdi-auto-fix</v-icon>
{{ scorePct(row.sugg) }} {{ scorePct(row.sugg) }}
</span> </span>
<span v-else class="text-caption">{{ row.hit.kind }}</span>
</template> </template>
</v-list-item> </v-list-item>
@@ -100,11 +93,10 @@ import { useSuggestionsStore } from '../../stores/suggestions.js'
import { useInflightToken } from '../../composables/useInflightToken.js' import { useInflightToken } from '../../composables/useInflightToken.js'
import FandomPicker from './FandomPicker.vue' import FandomPicker from './FandomPicker.vue'
const emit = defineEmits(['pick-existing', 'pick-new', 'accept-suggestion', 'cancel']) const emit = defineEmits(['pick-existing', 'pick-new', 'cancel'])
// The host surface's applied tags (host.current?.tags). Both dropdown sections // The host surface's applied tags (host.current?.tags). The dropdown filters
// filter against this so a just-added tag drops out of the search the moment // against this so a just-added tag drops out of the search the moment the chip
// the chip rail updates, not after the next modal refresh (operator-asked // rail updates, not after the next modal refresh (operator-asked 2026-07-03).
// 2026-07-03).
const props = defineProps({ const props = defineProps({
appliedTags: { type: Array, default: () => [] }, appliedTags: { type: Array, default: () => [] },
}) })
@@ -238,9 +230,6 @@ const createLabel = computed(() =>
function scorePct (s) { return `${Math.round(s.score * 100)}%` } function scorePct (s) { return `${Math.round(s.score * 100)}%` }
const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id))) const appliedIds = computed(() => new Set(props.appliedTags.map(t => t.id)))
const appliedNames = computed(() =>
new Set(props.appliedTags.map(t => `${t.kind}:${(t.name || '').toLowerCase()}`)),
)
// Server matches minus the image's applied tags. allowCreate/sameNameCharExists // Server matches minus the image's applied tags. allowCreate/sameNameCharExists
// keep reading the RAW hits — they reason about the tag universe (does this tag // keep reading the RAW hits — they reason about the tag universe (does this tag
@@ -249,49 +238,29 @@ const visibleHits = computed(() =>
hits.value.filter(h => !appliedIds.value.has(h.id)), hits.value.filter(h => !appliedIds.value.has(h.id)),
) )
// This image's suggestions that match the typed query, minus any the server // This image's suggestions keyed by canonical tag id, so a matching DB-tag row
// autocomplete already returned (same name+kind) so a tag never shows twice. // can show the model's confidence inline. Every suggestion is a canonical tag
// Sources the FULL prediction set (allByCategory, down to the store floor) — NOT // now (tagging-v2), so the id is the join key — no name/kind matching, no dedup,
// the threshold-filtered panel list — so a low-confidence action/feature the // no flicker. Rejected suggestions are excluded: a dismissed tag shouldn't
// model saw can be typed and accepted in canonical formatting instead of being // advertise a score in the type-to-add dropdown (un-reject lives in the panel).
// hand-entered as a custom tag (operator-asked 2026-06-09). The typed query is const suggByTagId = computed(() => {
// the only filter; the threshold no longer hides anything here. const m = new Map()
const suggestionHits = computed(() => { for (const list of Object.values(suggestions.byCategory)) {
const q = parsedName.value.toLowerCase()
if (!q) return []
const seen = new Set(visibleHits.value.map(h => `${h.kind}:${h.name.toLowerCase()}`))
const out = []
for (const list of Object.values(suggestions.allByCategory)) {
for (const s of list || []) { for (const s of list || []) {
// Rejected suggestions now stay in allByCategory (flagged) so the panel
// can show + un-reject them; keep them OUT of the type-to-add dropdown,
// whose job is finding a tag to ADD (un-reject lives in the panel).
if (s.rejected) continue if (s.rejected) continue
// Already on the image (matched by canonical id or name+kind): nothing if (s.canonical_tag_id != null) m.set(s.canonical_tag_id, s)
// to add. Covers the window between an add and the next suggestions
// fetch, where the stale row would otherwise still be pickable.
if (s.canonical_tag_id != null && appliedIds.value.has(s.canonical_tag_id)) continue
const key = `${s.category}:${s.display_name.toLowerCase()}`
if (appliedNames.value.has(key)) continue
if (!s.display_name.toLowerCase().includes(q)) continue
if (seen.has(key)) continue
seen.add(key)
out.push(s)
} }
} }
// Best matches first; cap generously so a specific typed query surfaces its return m
// matches even when many predictions exist, while the list stays scrollable.
out.sort((a, b) => b.score - a.score)
return out.slice(0, 20)
}) })
// One ordered list backing both the rendered dropdown and keyboard nav, so the // One ordered list backing both the rendered dropdown and keyboard nav, so the
// highlight index maps 1:1 to a row regardless of which section it's in. // highlight index maps 1:1 to a row. Each hit is annotated with its suggestion
// (score) when the model scored that tag for this image.
const rows = computed(() => { const rows = computed(() => {
const r = visibleHits.value.map(h => ({ type: 'hit', key: `h${h.id}`, hit: h })) const r = visibleHits.value.map(h => ({
for (const s of suggestionHits.value) { type: 'hit', key: `h${h.id}`, hit: h, sugg: suggByTagId.value.get(h.id) || null,
r.push({ type: 'suggestion', key: `s${s.category}:${s.display_name}`, sugg: s }) }))
}
if (allowCreate.value) r.push({ type: 'create', key: 'create' }) if (allowCreate.value) r.push({ type: 'create', key: 'create' })
return r return r
}) })
@@ -313,16 +282,9 @@ function moveHighlight (delta) {
// Dispatch a chosen dropdown row by its type. // Dispatch a chosen dropdown row by its type.
function onPickRow (row) { function onPickRow (row) {
if (row.type === 'hit') { emit('pick-existing', row.hit); reset() } if (row.type === 'hit') { emit('pick-existing', row.hit); reset() }
else if (row.type === 'suggestion') { onPickSuggestion(row.sugg) }
else { onCreate() } else { onCreate() }
} }
// Picking an ML suggestion hands it to the parent, which runs the same accept
// flow as the Suggestions panel (creates the tag if raw, records acceptance,
// drops it from the panel). reset() clears the query; the panel-drop makes it
// fall out of suggestionHits reactively.
function onPickSuggestion (s) { emit('accept-suggestion', s); reset() }
function onCreate () { function onCreate () {
const name = parsedName.value const name = parsedName.value
const kind = parsedKind.value const kind = parsedKind.value
@@ -389,14 +351,16 @@ function reset () { query.value = ''; hits.value = []; highlight.value = 0 }
border: 1px solid rgb(var(--v-theme-surface-light)); border: 1px solid rgb(var(--v-theme-surface-light));
border-radius: 6px; border-radius: 6px;
} }
/* Mark ML-suggestion rows so they read as distinct from typed/known matches. */ /* The model-confidence badge on a row whose tag the model also scored. Accent
.fc-tag-autocomplete__sugg { when above the head's suggest threshold; muted when below (a low-confidence
border-left: 2px solid rgb(var(--v-theme-accent), 0.5); match you can still type + pick). */
}
.fc-tag-autocomplete__sugg-tag { .fc-tag-autocomplete__sugg-tag {
display: inline-flex; align-items: center; gap: 2px; display: inline-flex; align-items: center; gap: 2px;
font-size: 11px; font-size: 11px;
font-family: 'JetBrains Mono', monospace; font-family: 'JetBrains Mono', monospace;
color: rgb(var(--v-theme-accent)); color: rgb(var(--v-theme-accent));
} }
.fc-tag-autocomplete__sugg-tag--below {
color: rgb(var(--v-theme-on-surface-variant));
}
</style> </style>
+31 -15
View File
@@ -1,7 +1,7 @@
<template> <template>
<span class="fc-tag-chip" @mouseenter="onEnter" @mouseleave="onLeave"> <span class="fc-tag-chip" @mouseenter="onEnter" @mouseleave="onLeave">
<v-chip <v-chip
size="small" :closable="!unconfirmedAuto" size="default" :closable="!unconfirmedAuto"
:color="store.colorFor(tag.kind)" variant="tonal" :color="store.colorFor(tag.kind)" variant="tonal"
class="fc-tag-chip__nav" class="fc-tag-chip__nav"
role="link" role="link"
@@ -9,7 +9,7 @@
@click="$emit('navigate', tag)" @click="$emit('navigate', tag)"
@click:close="$emit('remove', tag.id)" @click:close="$emit('remove', tag.id)"
> >
<v-icon start size="x-small">{{ iconFor(tag.kind) }}</v-icon> <v-icon start size="small">{{ iconFor(tag.kind) }}</v-icon>
<span class="fc-tag-chip__name">{{ tag.name }}</span><v-icon <span class="fc-tag-chip__name">{{ tag.name }}</span><v-icon
v-if="tag.is_system" end size="x-small" class="fc-tag-chip__system" v-if="tag.is_system" end size="x-small" class="fc-tag-chip__system"
title="System tag — tagged items are excluded from training other concepts" title="System tag — tagged items are excluded from training other concepts"
@@ -27,13 +27,13 @@
:title="`Yes — keep “${tag.name}” (trains the model, won't be retracted)`" :title="`Yes — keep “${tag.name}” (trains the model, won't be retracted)`"
:aria-label="`Confirm ${tag.name}`" :aria-label="`Confirm ${tag.name}`"
@click.stop="$emit('confirm', tag)" @click.stop="$emit('confirm', tag)"
><v-icon size="13">mdi-check</v-icon></button> ><v-icon size="15">mdi-check</v-icon></button>
<button <button
type="button" class="fc-tag-chip__no" type="button" class="fc-tag-chip__no"
:title="`No — remove “${tag.name}”`" :title="`No — remove “${tag.name}”`"
:aria-label="`Reject ${tag.name}`" :aria-label="`Reject ${tag.name}`"
@click.stop="$emit('remove', tag.id)" @click.stop="$emit('remove', tag.id)"
><v-icon size="13">mdi-close</v-icon></button> ><v-icon size="15">mdi-close</v-icon></button>
</span> </span>
</v-chip> </v-chip>
<!-- Modal-safe kebab is baked into KebabMenu (this chip lives in the <!-- Modal-safe kebab is baked into KebabMenu (this chip lives in the
@@ -143,7 +143,19 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
chip's hover title. */ chip's hover title. */
.fc-tag-chip { display: inline-flex; align-items: center; gap: 1px; max-width: 100%; min-width: 0; } .fc-tag-chip { display: inline-flex; align-items: center; gap: 1px; max-width: 100%; min-width: 0; }
.fc-tag-chip__nav { max-width: 100%; min-width: 0; } .fc-tag-chip__nav { max-width: 100%; min-width: 0; }
/* Tonal chips (esp. character = info) wash out against the dark rail — the fill
is intentionally faint. Give every tag chip a thin border in its OWN kind
colour (currentColor = the tonal chip's themed foreground) so the edge reads
clearly without touching the fill (operator-asked 2026-07-08). Theme-aware:
currentColor tracks the kind colour in either light or dark. */
.fc-tag-chip__nav { border: thin solid color-mix(in srgb, currentColor 55%, transparent); }
.fc-tag-chip__nav :deep(.v-chip__content) { min-width: 0; overflow: hidden; } .fc-tag-chip__nav :deep(.v-chip__content) { min-width: 0; overflow: hidden; }
/* The bigger size=default chip widens Vuetify's negative start-margin on the
leading kind-icon, pulling it LEFT of the content box above — where that
overflow:hidden then clips the icon's left edge as a vertical slice
(operator-flagged 2026-07-08). Zero the pull so the icon sits inside the clip
box; the chip's own 12px padding keeps a comfortable left inset. */
.fc-tag-chip__nav :deep(.v-icon--start) { margin-inline-start: 0; }
.fc-tag-chip__name { .fc-tag-chip__name {
overflow: hidden; text-overflow: ellipsis; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
min-width: 0; flex: 0 1 auto; min-width: 0; flex: 0 1 auto;
@@ -159,23 +171,27 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
.fc-tag-chip__kebab { opacity: 0.7; } .fc-tag-chip__kebab { opacity: 0.7; }
.fc-tag-chip:hover .fc-tag-chip__kebab { opacity: 1; } .fc-tag-chip:hover .fc-tag-chip__kebab { opacity: 1; }
.fc-tag-chip__fandom { opacity: 0.7; font-size: 0.85em; } .fc-tag-chip__fandom { opacity: 0.7; font-size: 0.85em; }
/* Provisional auto-tag: a compact green ✓ / red ✗ pair in place of the ✕. The /* Provisional auto-tag: a green ✓ / red ✗ pair in place of the ✕. The yes/no is
yes/no is obvious enough on its own, so there's no "auto" label (operator-asked obvious enough on its own, so there's no "auto" label (operator-asked
2026-07-07). flex:0 0 auto keeps it visible while the name ellipsis-truncates. */ 2026-07-07). flex:0 0 auto keeps it visible while the name ellipsis-truncates. */
.fc-tag-chip__verdict { .fc-tag-chip__verdict {
flex: 0 0 auto; display: inline-flex; align-items: center; gap: 1px; flex: 0 0 auto; display: inline-flex; align-items: center; gap: 3px;
margin-left: 3px; margin-left: 5px;
} }
/* Solid-filled (white glyph on a full success/error circle) so accept/reject
stay well-defined even on a muted tonal chip — e.g. character (info) — instead
of a faint icon that only lit up on hover. Mirrors the Suggestions rail's
verdict buttons so the affordance reads identically (operator-asked 2026-07-08). */
.fc-tag-chip__yes, .fc-tag-chip__no { .fc-tag-chip__yes, .fc-tag-chip__no {
display: inline-flex; align-items: center; justify-content: center; display: inline-flex; align-items: center; justify-content: center;
width: 18px; height: 18px; padding: 0; border: none; border-radius: 50%; width: 22px; height: 22px; padding: 0; border: none; border-radius: 50%;
background: transparent; cursor: pointer; transition: background 0.1s; color: #fff; cursor: pointer; opacity: 0.92;
transition: transform 0.1s, opacity 0.1s;
} }
.fc-tag-chip__yes { color: rgb(var(--v-theme-success)); } .fc-tag-chip__yes { background: rgb(var(--v-theme-success)); }
.fc-tag-chip__no { color: rgb(var(--v-theme-error)); } .fc-tag-chip__no { background: rgb(var(--v-theme-error)); }
.fc-tag-chip__yes:hover { background: rgb(var(--v-theme-success), 0.16); } .fc-tag-chip__yes:hover, .fc-tag-chip__no:hover { opacity: 1; transform: scale(1.1); }
.fc-tag-chip__no:hover { background: rgb(var(--v-theme-error), 0.16); }
.fc-tag-chip__yes:focus-visible, .fc-tag-chip__no:focus-visible { .fc-tag-chip__yes:focus-visible, .fc-tag-chip__no:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px; outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 2px;
} }
</style> </style>
@@ -17,7 +17,6 @@
ref="tagInputRef" ref="tagInputRef"
:applied-tags="host.current?.tags || []" :applied-tags="host.current?.tags || []"
@pick-existing="onPickExisting" @pick-new="onPickNew" @pick-existing="onPickExisting" @pick-new="onPickNew"
@accept-suggestion="onAcceptSuggestion"
/> />
<v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable> <v-alert v-if="errorMsg" type="error" variant="tonal" class="mt-2" closable>
@@ -134,7 +133,6 @@ async function onRemove(tagId) {
// until the next modal open (operator-asked 2026-07-03). // until the next modal open (operator-asked 2026-07-03).
if (host.currentImageId != null) { if (host.currentImageId != null) {
suggestions.load(host.currentImageId) suggestions.load(host.currentImageId)
suggestions.loadAll(host.currentImageId)
} }
focusTagInput() focusTagInput()
} }
@@ -178,18 +176,6 @@ async function onPickNew(payload) {
} }
catch (e) { errorMsg.value = e.message } catch (e) { errorMsg.value = e.message }
} }
// A suggestion picked from the autocomplete dropdown runs the SAME path as the
// Suggestions panel's Accept: the store creates the tag if it's raw, records the
// acceptance, and drops it from the panel; then we refresh the chip rail.
async function onAcceptSuggestion(s) {
errorMsg.value = null
try {
await suggestions.accept(s)
await host.reloadTags()
focusTagInput()
} catch (e) { errorMsg.value = e.message }
}
const renameDialog = ref(false) const renameDialog = ref(false)
const renameTarget = ref(null) const renameTarget = ref(null)
@@ -70,6 +70,14 @@
@click.stop="showOriginal = !showOriginal" @click.stop="showOriginal = !showOriginal"
>{{ showOriginal ? 'Show translation' : `Show original${sourceLangLabel}` }}</button> >{{ showOriginal ? 'Show translation' : `Show original${sourceLangLabel}` }}</button>
<!-- Per-post translation override (#155): force a skipped translation on,
or keep the original for a confidently mis-flagged title. Only where
there's text to translate. -->
<PostTranslationControl
v-if="post.post_title || post.description_plain"
:post="post"
/>
<!-- Faithful (semantic) body render once expanded: backend-sanitized <!-- Faithful (semantic) body render once expanded: backend-sanitized
HTML (headings, lists, links, inline images). Collapsed and the HTML (headings, lists, links, inline images). Collapsed and the
no-detail fallback stay plain text. --> no-detail fallback stay plain text. -->
@@ -136,6 +144,7 @@ import { useModalStore } from '../../stores/modal.js'
import { usePostsStore } from '../../stores/posts.js' import { usePostsStore } from '../../stores/posts.js'
import { toPlainText } from '../../utils/htmlSanitize.js' import { toPlainText } from '../../utils/htmlSanitize.js'
import PostSeriesMenu from './PostSeriesMenu.vue' import PostSeriesMenu from './PostSeriesMenu.vue'
import PostTranslationControl from './PostTranslationControl.vue'
const props = defineProps({ const props = defineProps({
post: { type: Object, required: true }, post: { type: Object, required: true },
@@ -0,0 +1,102 @@
<template>
<!-- Sticky per-post translation override (#155). Quiet inline control under the
post title: force a skipped legit-foreign title on, or knock a wrongly
mis-translated one back to the original. The choice sticks through a
Re-translate-all. -->
<div class="fc-post-tx">
<span class="fc-post-tx__label">Translation:</span>
<v-menu :disabled="busy">
<template #activator="{ props: menuProps }">
<button
type="button" class="fc-post-tx__btn" v-bind="menuProps" :disabled="busy"
:aria-label="`Translation handling: ${currentLabel}`"
>
{{ currentLabel }}
<v-icon size="14">mdi-menu-down</v-icon>
</button>
</template>
<v-list density="compact" min-width="240" class="fc-post-tx__list">
<v-list-item
v-for="opt in OPTIONS" :key="opt.value"
:active="opt.value === current" @click="choose(opt.value)"
>
<template #prepend>
<v-icon size="small">{{ opt.icon }}</v-icon>
</template>
<v-list-item-title>{{ opt.label }}</v-list-item-title>
<v-list-item-subtitle>{{ opt.help }}</v-list-item-subtitle>
</v-list-item>
</v-list>
</v-menu>
<v-progress-circular v-if="busy" indeterminate size="13" width="2" />
</div>
</template>
<script setup>
import { computed, ref } from 'vue'
import { usePostsStore } from '../../stores/posts.js'
import { toast } from '../../utils/toast.js'
const props = defineProps({ post: { type: Object, required: true } })
const posts = usePostsStore()
const busy = ref(false)
const OPTIONS = [
{ value: 'auto', label: 'Auto', icon: 'mdi-cog-outline',
help: 'Translate only when confident enough' },
{ value: 'force', label: 'Force translate', icon: 'mdi-translate',
help: 'Always translate, even at low confidence' },
{ value: 'original', label: 'Keep original', icon: 'mdi-translate-off',
help: 'Never translate — keep the original text' },
]
const current = computed(() => props.post.translation_override || 'auto')
const currentLabel = computed(
() => (OPTIONS.find((o) => o.value === current.value) || OPTIONS[0]).label,
)
async function choose (value) {
if (value === current.value || busy.value) return
busy.value = true
try {
const res = await posts.applyTranslationOverride(props.post.id, value)
if (res.applied === 'queued') {
toast({
text: 'Saved — this post will translate on the next sweep (service offline).',
type: 'info',
})
}
} catch (e) {
toast({ text: `Couldn't update translation: ${e.message}`, type: 'error' })
} finally {
busy.value = false
}
}
</script>
<style scoped>
.fc-post-tx {
display: inline-flex; align-items: center; gap: 6px;
margin: 2px 0 6px;
}
.fc-post-tx__label {
font-size: 0.72rem;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn {
display: inline-flex; align-items: center; gap: 1px;
padding: 0; background: none; border: 0; cursor: pointer;
font-size: 0.72rem; font-weight: 700;
color: rgb(var(--v-theme-on-surface-variant));
}
.fc-post-tx__btn:hover { color: rgb(var(--v-theme-accent)); }
.fc-post-tx__btn:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
border-radius: 2px;
}
.fc-post-tx__btn:disabled { cursor: default; opacity: 0.6; }
.fc-post-tx__list :deep(.v-list-item-subtitle) {
font-size: 0.7rem;
}
</style>
+75 -6
View File
@@ -171,13 +171,13 @@
/> />
</div> </div>
<p class="fc-muted text-body-2 mb-3"> <p class="fc-muted text-body-2 mb-3">
Auto-hide banners and editor screenshots from the gallery once a head has Auto-hide <code>banner</code> chrome from the gallery once a head has
learned them ( {{ minPositives }} examples) and clears learned it ( {{ minPositives }} examples) and clears
{{ Math.round((presentationThresholdInput || 0) * 100) }}% confidence. {{ Math.round((presentationThresholdInput || 0) * 100) }}% confidence.
<code>wip</code> is never auto-hidden. If a hidden image also looks like (<code>wip</code> and <code>editor screenshot</code> are handled by the
real content ( {{ Math.round((presentationConflictInput || 0) * 100) }}% process auto-tagger below.) If a hidden image also looks like real content
on a content tag), it's flagged in the Hidden view instead of buried. ( {{ Math.round((presentationConflictInput || 0) * 100) }}% on a content
Every auto-hide is reversible. tag), it's flagged for review instead of buried. Every auto-hide is reversible.
</p> </p>
<div class="d-flex mb-3" style="gap: 12px;"> <div class="d-flex mb-3" style="gap: 12px;">
<v-text-field <v-text-field
@@ -195,6 +195,43 @@
</div> </div>
</div> </div>
<!-- Process auto-tagging (#1464): wip / editor screenshot -->
<div class="fc-auto mt-6">
<div class="d-flex align-center mb-1" style="gap: 10px;">
<v-icon size="18" color="accent">mdi-progress-wrench</v-icon>
<span class="fc-section-h">Auto-tag work-in-progress</span>
<v-switch
v-model="processEnabled" :loading="settingBusy" hide-details
density="compact" color="success" class="ml-auto"
@update:model-value="onToggleProcess"
/>
</div>
<p class="fc-muted text-body-2 mb-3">
Auto-tag <code>wip</code> and <code>editor screenshot</code> process art
once a head has learned them (≥ {{ minPositives }} examples) and clears
{{ Math.round((processThresholdInput || 0) * 100) }}% confidence. These stay
<strong>visible</strong> in the gallery — the tag just keeps them out of
training and the Explore rabbit-hole. Off by default. If a tagged image also
looks like real content (≥ {{ Math.round((processConflictInput || 0) * 100) }}%
on a content tag), it's flagged for review. Learns only from your titles +
manual tags, never its own guesses so it can't run away. Every tag reversible.
</p>
<div class="d-flex mb-3" style="gap: 12px;">
<v-text-field
v-model.number="processThresholdInput" label="Tag confidence"
type="number" min="0.5" max="0.999" step="0.01" density="compact"
hide-details style="max-width: 200px;" :disabled="settingBusy"
@change="onSaveProcess"
/>
<v-text-field
v-model.number="processConflictInput" label="Flag if content ≥"
type="number" min="0" max="1" step="0.05" density="compact"
hide-details style="max-width: 200px;" :disabled="settingBusy"
@change="onSaveProcess"
/>
</div>
</div>
<!-- Performance / tuning --> <!-- Performance / tuning -->
<div v-if="metricsConcepts.length" class="mt-5"> <div v-if="metricsConcepts.length" class="mt-5">
<div class="fc-section-h mb-1">How auto-apply is landing</div> <div class="fc-section-h mb-1">How auto-apply is landing</div>
@@ -258,6 +295,9 @@ let autoTimer = null
const presentationEnabled = ref(true) const presentationEnabled = ref(true)
const presentationThresholdInput = ref(0.90) const presentationThresholdInput = ref(0.90)
const presentationConflictInput = ref(0.50) const presentationConflictInput = ref(0.50)
const processEnabled = ref(false)
const processThresholdInput = ref(0.90)
const processConflictInput = ref(0.50)
const autoRunning = computed(() => autoStatus.value?.running_id != null) const autoRunning = computed(() => autoStatus.value?.running_id != null)
const lastSweep = computed(() => const lastSweep = computed(() =>
@@ -292,6 +332,9 @@ onMounted(async () => {
presentationEnabled.value = s.presentation_auto_apply_enabled ?? true presentationEnabled.value = s.presentation_auto_apply_enabled ?? true
presentationThresholdInput.value = s.presentation_auto_apply_threshold ?? 0.90 presentationThresholdInput.value = s.presentation_auto_apply_threshold ?? 0.90
presentationConflictInput.value = s.presentation_conflict_threshold ?? 0.50 presentationConflictInput.value = s.presentation_conflict_threshold ?? 0.50
processEnabled.value = s.process_auto_apply_enabled ?? false
processThresholdInput.value = s.process_auto_apply_threshold ?? 0.90
processConflictInput.value = s.process_conflict_threshold ?? 0.50
} catch { /* non-fatal */ } } catch { /* non-fatal */ }
await refresh() await refresh()
if (running.value) startPoll() if (running.value) startPoll()
@@ -402,6 +445,32 @@ async function onSavePresentation() {
settingBusy.value = false settingBusy.value = false
} }
} }
async function onToggleProcess(val) {
settingBusy.value = true
try {
await mlSettings.patchSettings({ process_auto_apply_enabled: !!val })
toast({ text: val ? 'WIP auto-tag on' : 'WIP auto-tag off', type: 'success' })
} catch (e) {
processEnabled.value = !val // revert the switch
toast({ text: `Could not update: ${e.message}`, type: 'error' })
} finally {
settingBusy.value = false
}
}
async function onSaveProcess() {
settingBusy.value = true
try {
await mlSettings.patchSettings({
process_auto_apply_threshold: Number(processThresholdInput.value),
process_conflict_threshold: Number(processConflictInput.value),
})
} catch (e) {
toast({ text: `Could not save: ${e.message}`, type: 'error' })
} finally {
settingBusy.value = false
}
}
function onPreview() { startSweep(true) } function onPreview() { startSweep(true) }
function onApplyNow() { startSweep(false) } function onApplyNow() { startSweep(false) }
async function startSweep(dryRun) { async function startSweep(dryRun) {
@@ -79,6 +79,46 @@
</v-col> </v-col>
</v-row> </v-row>
<v-divider class="my-5" />
<!-- Title-based WIP auto-tagging (task #1458). The switch gates the LIVE
import hook; the button runs the one-off back-catalogue scan (an
explicit action it is deliberately not a scheduled sweep so it can't
silently re-apply a WIP tag you removed by hand). -->
<div class="fc-wip">
<div class="fc-wip__title">WIP auto-tagging</div>
<v-switch
v-model="local.wip_title_tagging_enabled"
label="Tag work-in-progress from post titles"
density="compact" hide-details color="primary" @change="save"
/>
<div class="fc-help mb-3">
When a post's title says <strong>WIP</strong> or
<strong>work in progress</strong>, new imports get the
<code>wip</code> tag automatically keeping unfinished pieces out of
the Explore browse. Applies to new imports; run the scan below to catch
posts already in your library.
</div>
<v-switch
v-model="local.wip_soft_title_tagging_enabled"
label="Also tag “sketch” / “doodle” titles (lower precision)"
density="compact" hide-details color="primary" @change="save"
/>
<div class="fc-help mb-3">
Extends the above to softer cues (<code>sketch</code>, <code>doodle</code>,
<code>scribble</code>). These stay <strong>visible</strong> and never train
the tagging model a daily audit flags any that actually look like finished
art for review. Off by default.
</div>
<v-btn
variant="tonal" color="primary" size="small"
:loading="store.wipScanBusy" prepend-icon="mdi-magnify"
@click="store.scanWipTitles()"
>
Scan existing posts for WIP titles
</v-btn>
</div>
<v-alert v-if="store.settingsError" type="error" variant="tonal" class="mt-2" closable> <v-alert v-if="store.settingsError" type="error" variant="tonal" class="mt-2" closable>
{{ store.settingsError }} {{ store.settingsError }}
</v-alert> </v-alert>
@@ -109,6 +149,8 @@ const local = reactive({
skip_transparent: false, transparency_threshold: 0.9, skip_transparent: false, transparency_threshold: 0.9,
skip_single_color: false, single_color_threshold: 0.95, skip_single_color: false, single_color_threshold: 0.95,
phash_threshold: 10, phash_threshold: 10,
wip_title_tagging_enabled: true,
wip_soft_title_tagging_enabled: false,
}) })
watch(() => store.settings, (s) => { if (s) Object.assign(local, s) }, { immediate: true }) watch(() => store.settings, (s) => { if (s) Object.assign(local, s) }, { immediate: true })
@@ -124,11 +166,18 @@ async function save() {
color: rgb(var(--v-theme-on-surface-variant)); color: rgb(var(--v-theme-on-surface-variant));
margin-top: 2px; margin-top: 2px;
} }
.fc-phash__title { .fc-phash__title,
.fc-wip__title {
font-size: 0.95rem; font-size: 0.95rem;
font-weight: 600; font-weight: 600;
color: rgb(var(--v-theme-on-surface)); color: rgb(var(--v-theme-on-surface));
} }
.fc-wip code {
font-size: 0.85em;
padding: 1px 4px;
border-radius: 3px;
background: rgb(var(--v-theme-surface-variant));
}
/* Headroom so the tick labels (Exact/Strict/Default/Loose) aren't clipped. */ /* Headroom so the tick labels (Exact/Strict/Default/Loose) aren't clipped. */
.fc-phash__slider { margin-bottom: 18px; } .fc-phash__slider { margin-bottom: 18px; }
</style> </style>
@@ -28,6 +28,21 @@
@change="onSave" @change="onSave"
/> />
<!-- Acceptance floor (#155): latin-script translations below this Interpreter
confidence are kept as the original. Per-post overrides handle exceptions. -->
<v-text-field
v-model.number="minConfidence" label="Acceptance confidence"
type="number" min="0" max="1" step="0.01" density="compact"
hide-details style="max-width: 200px;" class="mb-1" :disabled="busy"
@change="onSaveConfidence"
/>
<div class="fc-muted text-caption mb-3">
Latin-script titles Interpreter is less than this sure about (01) are kept
as the original; CJK is always translated. Default 0.90 — raise it to reject
more mis-detections, lower it to translate more. Per-post controls in the
posts feed override this either way.
</div>
<div class="d-flex align-center flex-wrap mb-3" style="gap: 8px;"> <div class="d-flex align-center flex-wrap mb-3" style="gap: 8px;">
<v-icon size="12" :color="statusColor">mdi-circle</v-icon> <v-icon size="12" :color="statusColor">mdi-circle</v-icon>
<span class="fc-muted text-body-2">{{ statusText }}</span> <span class="fc-muted text-body-2">{{ statusText }}</span>
@@ -52,11 +67,25 @@
prepend-icon="mdi-refresh" :loading="retranslating" prepend-icon="mdi-refresh" :loading="retranslating"
:disabled="!enabled || !baseUrl" @click="confirmAll = true" :disabled="!enabled || !baseUrl" @click="confirmAll = true"
>Re-translate all</v-btn> >Re-translate all</v-btn>
<span v-if="status" class="fc-muted text-caption"> <span
v-if="status && status.active"
class="fc-muted text-caption d-inline-flex align-center" style="gap: 6px;"
>
<v-progress-circular indeterminate size="12" width="2" color="accent" />
Translating… {{ status.untranslated_count }} remaining
</span>
<span v-else-if="status" class="fc-muted text-caption">
{{ status.untranslated_count }} {{ status.untranslated_count }}
post{{ status.untranslated_count === 1 ? '' : 's' }} awaiting translation post{{ status.untranslated_count === 1 ? '' : 's' }} awaiting translation
</span> </span>
</div> </div>
<p
v-if="status && status.last_run && ['error', 'timeout'].includes(status.last_run.status)"
class="text-caption text-error mt-1 mb-0"
>
Last translation run ended with “{{ status.last_run.status }}”. It resumes on
the next sweep; check the Interpreter connection if it persists.
</p>
<p class="fc-muted text-caption mt-2 mb-0"> <p class="fc-muted text-caption mt-2 mb-0">
Use <strong>Re-translate all</strong> after switching the Interpreter model — Use <strong>Re-translate all</strong> after switching the Interpreter model —
it clears every stored translation and re-runs it through the new model it clears every stored translation and re-runs it through the new model
@@ -64,6 +93,61 @@
artist, use the Re-translate button on that artist's page. artist, use the Re-translate button on that artist's page.
</p> </p>
<v-divider class="my-3" />
<!-- Diagnostic probe (task #1376): paste text → what Interpreter DETECTS
(language + confidence) and returns, without saving. Used to see why a
short/abbreviation-heavy English title gets mis-detected, and to pick a
detection-confidence / min-length guard from real numbers. -->
<div class="mb-1">
<span class="fc-section-h">Test translation</span>
<p class="fc-muted text-caption mt-1 mb-2">
Paste a title or snippet to see what Interpreter detects and returns —
handy for checking why a short string is mis-detected. Nothing is saved.
</p>
</div>
<v-textarea
v-model="probeText" label="Text to test" rows="2" auto-grow
density="compact" hide-details class="mb-2" :disabled="!baseUrl"
/>
<div class="d-flex align-center flex-wrap mb-2" style="gap: 8px;">
<v-btn
size="small" variant="tonal" rounded="pill" prepend-icon="mdi-magnify"
:loading="probing" :disabled="!baseUrl || !probeText.trim()"
@click="onProbe"
>Test translation</v-btn>
</div>
<v-sheet
v-if="probeResult" rounded class="pa-3 mb-1 text-body-2"
color="rgba(var(--v-theme-accent), 0.06)"
>
<div class="d-flex flex-wrap" style="gap: 4px 16px;">
<span>
<span class="fc-muted">Detected:</span>
<strong>{{ probeResult.detected_lang || '' }}</strong>
</span>
<span v-if="probeResult.detected_confidence != null">
<span class="fc-muted">Confidence:</span>
<strong>{{ fmtConf(probeResult.detected_confidence) }}</strong>
</span>
<span>
<span class="fc-muted">Engine:</span>
{{ probeResult.engine || '' }}<template
v-if="probeResult.engine_version"
> ({{ probeResult.engine_version }})</template>
</span>
<span><span class="fc-muted">Target:</span> {{ probeResult.target }}</span>
</div>
<v-divider class="my-2" />
<div class="fc-muted mb-1">
Result<template v-if="probeResult.detected_lang === probeResult.target">
— already {{ probeResult.target }}, so the sweep leaves it as-is</template>:
</div>
<p class="mb-0" style="white-space: pre-wrap;">{{
probeResult.translated || '(no change)'
}}</p>
</v-sheet>
<v-alert <v-alert
v-if="err" type="error" variant="tonal" density="compact" v-if="err" type="error" variant="tonal" density="compact"
class="mt-3" closable @click:close="err = null" class="mt-3" closable @click:close="err = null"
@@ -94,7 +178,7 @@
</template> </template>
<script setup> <script setup>
import { computed, onMounted, ref } from 'vue' import { computed, onMounted, onUnmounted, ref } from 'vue'
import { toast } from '../../utils/toast.js' import { toast } from '../../utils/toast.js'
import { useApi } from '../../composables/useApi.js' import { useApi } from '../../composables/useApi.js'
import { useImportStore } from '../../stores/import.js' import { useImportStore } from '../../stores/import.js'
@@ -106,6 +190,7 @@ const store = useImportStore()
const enabled = ref(false) const enabled = ref(false)
const baseUrl = ref('') const baseUrl = ref('')
const targetLang = ref('en') const targetLang = ref('en')
const minConfidence = ref(0.9)
const busy = ref(false) const busy = ref(false)
const running = ref(false) const running = ref(false)
const retranslating = ref(false) const retranslating = ref(false)
@@ -115,6 +200,17 @@ const testResult = ref(null) // null = not tested this session; true/false = l
const status = ref(null) const status = ref(null)
const err = ref(null) const err = ref(null)
// #1376 "Test translation" probe: paste text → detected lang/confidence + result.
const probeText = ref('')
const probing = ref(false)
const probeResult = ref(null)
// Confidence scale is Interpreter-defined (0100 or 01) — show it as-is, just
// trimmed to 2 decimals, so the operator reads the true number.
function fmtConf (c) {
return c == null ? '' : Math.round(c * 100) / 100
}
const statusColor = computed(() => { const statusColor = computed(() => {
if (!enabled.value || !baseUrl.value) return 'grey' if (!enabled.value || !baseUrl.value) return 'grey'
return status.value?.healthy ? 'success' : 'error' return status.value?.healthy ? 'success' : 'error'
@@ -126,10 +222,16 @@ const statusText = computed(() => {
return status.value.healthy ? 'Interpreter reachable' : 'Interpreter unreachable' return status.value.healthy ? 'Interpreter reachable' : 'Interpreter unreachable'
}) })
let pollTimer = null
async function loadStatus() { async function loadStatus() {
try { status.value = await api.get('/api/settings/translation/status') } try { status.value = await api.get('/api/settings/translation/status') }
catch { status.value = null } catch { status.value = null }
// Poll live while a sweep is running so the remaining count ticks down; stop
// when idle (a fresh trigger restarts it via its own setTimeout(loadStatus)).
clearTimeout(pollTimer)
if (status.value?.active) pollTimer = setTimeout(loadStatus, 3000)
} }
onUnmounted(() => clearTimeout(pollTimer))
async function onTest() { async function onTest() {
// Ping /v1/health for the CURRENTLY-typed URL (not the saved one), so a new // Ping /v1/health for the CURRENTLY-typed URL (not the saved one), so a new
@@ -150,6 +252,21 @@ async function onTest() {
} }
} }
async function onProbe () {
probing.value = true
err.value = null
probeResult.value = null
try {
probeResult.value = await api.post('/api/settings/translation/probe', {
body: { text: probeText.value.trim() },
})
} catch (e) {
err.value = e.message
} finally {
probing.value = false
}
}
onMounted(async () => { onMounted(async () => {
try { try {
await store.loadSettings() await store.loadSettings()
@@ -157,6 +274,7 @@ onMounted(async () => {
enabled.value = !!s.translation_enabled enabled.value = !!s.translation_enabled
baseUrl.value = s.interpreter_base_url || '' baseUrl.value = s.interpreter_base_url || ''
targetLang.value = s.translation_target_lang || 'en' targetLang.value = s.translation_target_lang || 'en'
minConfidence.value = s.translation_min_confidence ?? 0.9
} catch { /* non-fatal */ } } catch { /* non-fatal */ }
await loadStatus() await loadStatus()
}) })
@@ -181,6 +299,14 @@ async function onSave() {
}) })
await loadStatus() await loadStatus()
} }
async function onSaveConfidence() {
// Clamp to [0, 1] so a stray value never bounces off the API 400.
let v = Number(minConfidence.value)
if (!Number.isFinite(v)) v = 0.9
v = Math.min(1, Math.max(0, v))
minConfidence.value = v
await save({ translation_min_confidence: v })
}
async function onRun() { async function onRun() {
running.value = true running.value = true
err.value = null err.value = null
+21 -3
View File
@@ -25,6 +25,10 @@ export const useExploreStore = defineStore('explore', () => {
const cursor = ref(-1) const cursor = ref(-1)
const loading = ref(false) const loading = ref(false)
const error = ref(null) const error = ref(null)
// Reach (#1476): how far the walk reaches past the anchor's immediate cluster.
// 0 = nearest (can get stuck in a dense signature); ~0.4 default mixes in
// mid-far escape routes so the walk diversifies without hitting "Random image".
const reach = ref(0.4)
const inflight = useInflightToken() const inflight = useInflightToken()
@@ -46,7 +50,13 @@ export const useExploreStore = defineStore('explore', () => {
const body = await api.get('/api/gallery/similar', { const body = await api.get('/api/gallery/similar', {
// exclude_wip: keep work-in-progress out of the Explore rabbit-hole // exclude_wip: keep work-in-progress out of the Explore rabbit-hole
// (the gallery's own "similar" button still shows it) — operator 2026-07-08. // (the gallery's own "similar" button still shows it) — operator 2026-07-08.
params: { similar_to: numId, limit: NEIGHBOR_LIMIT, exclude_wip: 1 }, // reach + exclude_ids (#1476): reach past the dense cluster + never re-serve
// an already-walked image, so the walk keeps moving instead of getting stuck.
params: {
similar_to: numId, limit: NEIGHBOR_LIMIT, exclude_wip: 1,
reach: reach.value,
exclude_ids: breadcrumb.value.map((c) => c.id).join(','),
},
}) })
if (!t.isCurrent()) return if (!t.isCurrent()) return
neighbors.value = body.images || [] neighbors.value = body.images || []
@@ -113,6 +123,14 @@ export const useExploreStore = defineStore('explore', () => {
loading.value = false loading.value = false
} }
// Change how far the walk reaches and re-fetch the current anchor's neighbours
// with the new setting (the anchor + trail are unchanged — only the grid varies).
function setReach (v) {
reach.value = Math.max(0, Math.min(1, Number(v)))
const id = anchor.value?.id
if (id != null) anchorOn(id)
}
// --- TagPanel "host" surface --------------------------------------------- // --- TagPanel "host" surface ---------------------------------------------
// The anchor IS the current image (same /api/gallery/image/<id> payload the // The anchor IS the current image (same /api/gallery/image/<id> payload the
// modal uses), so these mirror the modal store's tag-CRUD, targeting the // modal uses), so these mirror the modal store's tag-CRUD, targeting the
@@ -189,8 +207,8 @@ export const useExploreStore = defineStore('explore', () => {
function close () {} function close () {}
return { return {
anchor, neighbors, breadcrumb, cursor, loading, error, NEIGHBOR_LIMIT, anchor, neighbors, breadcrumb, cursor, loading, error, NEIGHBOR_LIMIT, reach,
anchorOn, reset, backTarget, forwardTarget, anchorOn, reset, backTarget, forwardTarget, setReach,
// host surface // host surface
current, currentImageId, current, currentImageId,
reloadTags, addExistingTag, removeTag, createAndAdd, close, reloadTags, addExistingTag, removeTag, createAndAdd, close,
+20 -2
View File
@@ -16,6 +16,7 @@ export const useImportStore = defineStore('import', () => {
const settings = ref(null) const settings = ref(null)
const settingsLoading = ref(false) const settingsLoading = ref(false)
const settingsError = ref(null) const settingsError = ref(null)
const wipScanBusy = ref(false)
async function loadSettings() { async function loadSettings() {
settingsLoading.value = true settingsLoading.value = true
@@ -40,8 +41,25 @@ export const useImportStore = defineStore('import', () => {
} }
} }
// Enqueue the back-catalogue WIP-title scan (task #1458). New imports are
// tagged live; this catches posts already in the library. Fire-and-forget —
// the sweep runs on the maintenance worker and its run shows in Activity.
async function scanWipTitles() {
wipScanBusy.value = true
try {
const r = await api.post('/api/settings/wip-title/scan')
toast({ text: 'Scanning existing posts for WIP titles…', type: 'success' })
return r
} catch (e) {
toast({ text: `WIP scan failed: ${e.message}`, type: 'error' })
throw e
} finally {
wipScanBusy.value = false
}
}
return { return {
settings, settingsLoading, settingsError, settings, settingsLoading, settingsError, wipScanBusy,
loadSettings, patchSettings, loadSettings, patchSettings, scanWipTitles,
} }
}) })
+19 -1
View File
@@ -71,6 +71,24 @@ export const usePostsStore = defineStore('posts', () => {
return await api.get(`/api/posts/${id}`) return await api.get(`/api/posts/${id}`)
} }
// Set a post's sticky translation override (auto/force/original, #155) and
// patch the loaded item in place with the endpoint's result (it translates /
// clears immediately when the service is up), so the card reflects the change
// without a feed reload.
async function applyTranslationOverride(postId, override) {
const res = await api.post(`/api/posts/${postId}/translation-override`, {
body: { override },
})
const item = items.value.find((p) => p.id === postId)
if (item) {
item.translation_override = res.translation_override
item.post_title_translated = res.post_title_translated
item.description_translated = res.description_translated
item.translated_source_lang = res.translated_source_lang
}
return res
}
// Filter overlay for the around/older/newer (in-context anchored) // Filter overlay for the around/older/newer (in-context anchored)
// path. Keep this distinct from `filters.value` (the down-only feed) // path. Keep this distinct from `filters.value` (the down-only feed)
// so a normal-feed filter change doesn't leak into an active anchored // so a normal-feed filter change doesn't leak into an active anchored
@@ -168,7 +186,7 @@ export const usePostsStore = defineStore('posts', () => {
return { return {
items, cursor, loading, done, error, filters, items, cursor, loading, done, error, filters,
cursorOlder, cursorNewer, doneOlder, doneNewer, anchorId, cursorOlder, cursorNewer, doneOlder, doneNewer, anchorId,
loadInitial, loadMore, getPostFull, loadInitial, loadMore, getPostFull, applyTranslationOverride,
loadAround, loadOlder, loadNewer, loadAround, loadOlder, loadNewer,
} }
}) })
+73 -152
View File
@@ -1,6 +1,6 @@
import { defineStore } from 'pinia' import { defineStore } from 'pinia'
import { toast } from '../utils/toast.js' import { toast } from '../utils/toast.js'
import { ref } from 'vue' import { computed, ref } from 'vue'
import { useApi } from '../composables/useApi.js' import { useApi } from '../composables/useApi.js'
import { useAsyncAction } from '../composables/useAsyncAction.js' import { useAsyncAction } from '../composables/useAsyncAction.js'
import { useInflightToken } from '../composables/useInflightToken.js' import { useInflightToken } from '../composables/useInflightToken.js'
@@ -18,182 +18,112 @@ export const CATEGORY_LABELS = {
export const useSuggestionsStore = defineStore('suggestions', () => { export const useSuggestionsStore = defineStore('suggestions', () => {
const api = useApi() const api = useApi()
const byCategory = ref({}) // { category: [suggestion, ...] } — panel (≥ threshold) // ONE source of truth per image: every trained head scored for this image
// The typed tag-input dropdown searches the model's FULL prediction set for // (min=0), each row carrying above_threshold. The Suggestions PANEL renders
// the image (min=0, down to the store floor) so low-confidence actions/ // aboveByCategory; the typed tag-input dropdown reads the full set and
// features can be picked in canonical formatting (operator-asked 2026-06-09). // annotates matching DB-tag rows with their score. A single fetch backs both —
const allByCategory = ref({}) // every suggestion is a canonical tag now (tagging-v2, #114), so there is no
// raw-prediction / create-new / alias-remap path to reconcile.
const byCategory = ref({})
const { loading, error, run } = useAsyncAction({ errorAs: 'message' }) const { loading, error, run } = useAsyncAction({ errorAs: 'message' })
let currentImageId = null let currentImageId = null
const inflightAll = useInflightToken() // Audit 2026-06-02: without an inflight guard a late /suggestions response from
// Audit 2026-06-02: this store had no inflight guard — a late // a prior image could overwrite byCategory while currentImageId points at a new
// /suggestions response from a prior image could overwrite // one, and accept() could apply image A's tag to image B. Both guarded below by
// byCategory while currentImageId pointed at a new one, and // capturing imageId at call-time and gating writes on the token.
// accept() dereferenced currentImageId AFTER an awaited POST so
// the subsequent /suggestions/accept could apply A's chosen tag
// to image B (and push it to the allowlist). Both fixed below
// by capturing imageId at call-time and gating writes on the token.
const inflight = useInflightToken() const inflight = useInflightToken()
// The curated above-threshold subset the Suggestions panel shows. A rejected
// row that still scores above its cut stays (flagged) so the rejection is
// visible + reversible; below-threshold rows live only in the dropdown.
const aboveByCategory = computed(() => {
const out = {}
for (const [cat, list] of Object.entries(byCategory.value)) {
const kept = (list || []).filter(s => s.above_threshold)
if (kept.length) out[cat] = kept
}
return out
})
async function load(imageId) { async function load(imageId) {
// Cancel any in-flight load from the previous image so its late // Cancel any in-flight load from the previous image so its late response
// response can't overwrite this image's byCategory. // can't overwrite this image's list.
inflight.cancel() inflight.cancel()
currentImageId = imageId currentImageId = imageId
byCategory.value = {} // cleared upfront so it stays empty on error byCategory.value = {} // cleared upfront so it stays empty on error
const t = inflight.claim() const t = inflight.claim()
await run(async () => { await run(async () => {
const body = await api.get(`/api/images/${imageId}/suggestions`) // min=0 → every head, each flagged above_threshold; the panel + the typed
// dropdown are both derived from this one payload.
const body = await api.get(`/api/images/${imageId}/suggestions`, {
params: { min: 0 },
})
if (!t.isCurrent()) return if (!t.isCurrent()) return
byCategory.value = body.by_category || {} byCategory.value = body.by_category || {}
}) })
} }
// Load the full prediction set for the dropdown (separate inflight guard so a // A stable identity for a suggestion across the panel + dropdown surfaces:
// late response from a prior image can't overwrite the current one's list). // its canonical tag id (every suggestion has one now).
async function loadAll(imageId) {
inflightAll.cancel()
allByCategory.value = {}
const t = inflightAll.claim()
try {
const body = await api.get(`/api/images/${imageId}/suggestions`, {
params: { min: 0 },
})
if (!t.isCurrent()) return
allByCategory.value = body.by_category || {}
} catch {
// Dropdown is best-effort — the panel surfaces load errors.
}
}
// A stable identity for a suggestion across the two lists (panel vs dropdown
// are separate fetches, so object identity differs): tag id when known, else
// the raw display key.
function _keyOf(s) { function _keyOf(s) {
return s.canonical_tag_id != null return `id:${s.canonical_tag_id}`
? `id:${s.canonical_tag_id}`
: `raw:${s.category}:${(s.display_name || '').toLowerCase()}`
} }
// Drop an accepted/dismissed suggestion from BOTH the panel list and the // Drop an accepted suggestion from the list so it can't reappear.
// dropdown's full list so it can't reappear in either surface.
function _dropEverywhere(suggestion) { function _dropEverywhere(suggestion) {
const key = _keyOf(suggestion) const key = _keyOf(suggestion)
const cat = suggestion.category const list = byCategory.value[suggestion.category]
for (const map of [byCategory.value, allByCategory.value]) { if (list) byCategory.value[suggestion.category] = list.filter(s => _keyOf(s) !== key)
const list = map[cat]
if (list) map[cat] = list.filter(s => _keyOf(s) !== key)
}
} }
// Flip the `rejected` flag on the matching suggestion in BOTH lists in place, // Flip the `rejected` flag on the matching suggestion in place, so a
// so a reject/un-reject shows immediately without dropping the row (visible, // reject/un-reject shows immediately without dropping the row (visible,
// reversible rejection — misclick recovery, operator-asked 2026-06-27). // reversible rejection — misclick recovery, operator-asked 2026-06-27).
function _setRejectedEverywhere(suggestion, value) { function _setRejectedEverywhere(suggestion, value) {
const key = _keyOf(suggestion) const key = _keyOf(suggestion)
const cat = suggestion.category for (const s of byCategory.value[suggestion.category] || []) {
for (const map of [byCategory.value, allByCategory.value]) { if (_keyOf(s) === key) s.rejected = value
for (const s of map[cat] || []) {
if (_keyOf(s) === key) s.rejected = value
}
} }
} }
// opts.tagId: the tag's known id, when the caller already created/resolved // opts.tagId: the tag's known id when the caller already resolved it (TagPanel's
// it (TagPanel's manual-add-matches-suggestion path). Passed separately — // manual-add-matches-suggestion path). Passed separately — NOT spread onto the
// NOT spread onto the suggestion — because _dropEverywhere keys off the // suggestion — because _dropEverywhere keys off the suggestion object.
// suggestion object, and mutating canonical_tag_id on a raw suggestion
// would stop it matching its own row in the lists.
async function accept(suggestion, { tagId: knownTagId = null } = {}) { async function accept(suggestion, { tagId: knownTagId = null } = {}) {
// Capture imageId so a mid-flight prev/next can't reroute the // Capture imageId so a mid-flight prev/next can't reroute the accept POST to
// accept POST to a different image AND push the tag to that // a different image.
// image's allowlist.
const imageId = currentImageId const imageId = currentImageId
if (imageId == null) return if (imageId == null) return
// Raw tags (creates_new_tag) have no canonical_tag_id; the backend's const tagId = knownTagId ?? suggestion.canonical_tag_id
// accept endpoint needs a tag_id, so for raw tags we create the tag
// first via the existing /api/tags endpoint, then accept by id.
let tagId = knownTagId ?? suggestion.canonical_tag_id
if (tagId == null) {
const created = await api.post('/api/tags', {
body: { name: suggestion.display_name, kind: suggestion.category }
})
tagId = created.id
}
await api.post(`/api/images/${imageId}/suggestions/accept`, { await api.post(`/api/images/${imageId}/suggestions/accept`, {
body: { tag_id: tagId } body: { tag_id: tagId }
}) })
// Only drop from THIS image's category list — if the user navigated, // Only drop from THIS image's list — if the user navigated, the new image
// the new image has its own suggestions and this drop would corrupt them. // has its own suggestions and this drop would corrupt them.
if (currentImageId === imageId) { if (currentImageId === imageId) {
_dropEverywhere(suggestion) _dropEverywhere(suggestion)
} }
_acceptToast('Tagged', suggestion.display_name) _acceptToast('Tagged', suggestion.display_name)
} }
// One non-blocking toast for accept/alias. The accepted tag is applied to this // One non-blocking toast for accept. The accepted tag is applied to this image
// image and feeds head training; head auto-apply handles propagation (earned), // and feeds head training; head auto-apply handles propagation (earned).
// so there's no instant fan-out to project.
function _acceptToast(verb, displayName) { function _acceptToast(verb, displayName) {
toast({ text: `${verb}: ${displayName}`, type: 'success' }) toast({ text: `${verb}: ${displayName}`, type: 'success' })
} }
async function aliasAccept(suggestion, canonicalTagId) { // Find a live (non-rejected) suggestion matching a manually-picked tag — by
const imageId = currentImageId // canonical id when known, else by (kind, name). Manually adding a tag the
if (imageId == null) return // model also suggested must be indistinguishable from accepting the suggestion
// The alias MUST be stored under the raw model key — resolution looks up the // (recorded + dropped from the panel), so TagPanel routes matches through
// raw prediction key, not the normalized display name. Sending display_name // accept() (operator-asked 2026-07-03).
// (the old bug) stored an alias that never resolved, so the prediction kept
// reappearing unaliased. raw_name is null only for centroid hits, which
// can't be aliased (the UI hides the action for them).
const aliasString = suggestion.raw_name ?? suggestion.display_name
await api.post(`/api/images/${imageId}/suggestions/alias`, {
body: {
alias_string: aliasString,
alias_category: suggestion.category,
canonical_tag_id: canonicalTagId
}
})
if (currentImageId === imageId) {
_dropEverywhere(suggestion)
}
_acceptToast('Aliased & tagged', suggestion.display_name)
}
// Remove the alias behind an aliased suggestion (the raw prediction reverts to
// its unaliased form on reload). The canonical tag stays applied if it was
// accepted — this only undoes the model-key→tag mapping.
async function removeAlias(suggestion) {
const imageId = currentImageId
if (imageId == null || suggestion.raw_name == null) return
await api.delete(
`/api/aliases/${encodeURIComponent(suggestion.raw_name)}/${encodeURIComponent(suggestion.category)}`
)
if (currentImageId === imageId) {
await load(imageId)
await loadAll(imageId)
}
toast({ text: `Alias removed: ${suggestion.display_name}`, type: 'success' })
}
// Find a live (non-rejected) pending suggestion matching a manually-picked
// tag — by canonical id when known, else by (kind, name). Manually adding a
// tag the model also suggested must be indistinguishable from accepting the
// suggestion (recorded + dropped from the panel), so TagPanel routes matches
// through accept() (operator-asked 2026-07-03). Searches the full dropdown
// set first, then the panel list (loadAll is best-effort and can be empty).
function findPending(kind, name, tagId = null) { function findPending(kind, name, tagId = null) {
const lname = (name || '').toLowerCase() const lname = (name || '').toLowerCase()
for (const map of [allByCategory.value, byCategory.value]) { for (const list of Object.values(byCategory.value)) {
for (const list of Object.values(map)) { for (const s of list || []) {
for (const s of list || []) { if (s.rejected) continue
if (s.rejected) continue if (tagId != null && s.canonical_tag_id === tagId) return s
if (tagId != null && s.canonical_tag_id === tagId) return s if (s.category === kind && (s.display_name || '').toLowerCase() === lname) return s
if (
s.category === kind &&
(s.display_name || '').toLowerCase() === lname
) return s
}
} }
} }
return null return null
@@ -202,14 +132,8 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
async function dismiss(suggestion) { async function dismiss(suggestion) {
const imageId = currentImageId const imageId = currentImageId
if (imageId == null) return if (imageId == null) return
// Dismiss needs a tag_id. Raw tags (creates_new_tag) have none, so there's
// nothing to persist a rejection against — drop them client-side as before.
// Canonical tags persist a rejection and STAY in the list flagged rejected, // Canonical tags persist a rejection and STAY in the list flagged rejected,
// so the operator can see it and one-click un-reject (misclick recovery). // so the operator can see it and one-click un-reject (misclick recovery).
if (suggestion.canonical_tag_id == null) {
if (currentImageId === imageId) _dropEverywhere(suggestion)
return
}
await api.post(`/api/images/${imageId}/suggestions/dismiss`, { await api.post(`/api/images/${imageId}/suggestions/dismiss`, {
body: { tag_id: suggestion.canonical_tag_id } body: { tag_id: suggestion.canonical_tag_id }
}) })
@@ -218,33 +142,31 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
} }
} }
// Reject every still-unhandled suggestion in a category in one go ("confirm // Reject every still-unhandled VISIBLE (above-threshold) suggestion in a
// the good ones, reject the rest" — operator-asked 2026-07-06). Canonical tags // category in one go ("confirm the good ones, reject the rest" — operator-asked
// persist a rejection and STAY flagged rejected (reversible, one-click // 2026-07-06). Only touches what the panel shows, not the low-confidence tail
// un-reject); raw creates-new-tag rows drop client-side. Dispatched in parallel // the dropdown carries. Dispatched in parallel so a big section clears fast.
// so a big section clears fast.
async function dismissRemaining(category) { async function dismissRemaining(category) {
const imageId = currentImageId const imageId = currentImageId
if (imageId == null) return if (imageId == null) return
const targets = (byCategory.value[category] || []).filter((s) => !s.rejected) const targets = (byCategory.value[category] || []).filter(
(s) => s.above_threshold && !s.rejected
)
if (!targets.length) return if (!targets.length) return
const canon = targets.filter((s) => s.canonical_tag_id != null) await Promise.all(targets.map((s) =>
const raw = targets.filter((s) => s.canonical_tag_id == null)
await Promise.all(canon.map((s) =>
api.post(`/api/images/${imageId}/suggestions/dismiss`, { api.post(`/api/images/${imageId}/suggestions/dismiss`, {
body: { tag_id: s.canonical_tag_id }, body: { tag_id: s.canonical_tag_id },
}) })
)) ))
if (currentImageId === imageId) { if (currentImageId === imageId) {
canon.forEach((s) => _setRejectedEverywhere(s, true)) targets.forEach((s) => _setRejectedEverywhere(s, true))
raw.forEach((s) => _dropEverywhere(s))
} }
} }
// Undo a per-image dismissal — the suggestion reverts to a live row. // Undo a per-image dismissal — the suggestion reverts to a live row.
async function undismiss(suggestion) { async function undismiss(suggestion) {
const imageId = currentImageId const imageId = currentImageId
if (imageId == null || suggestion.canonical_tag_id == null) return if (imageId == null) return
await api.post(`/api/images/${imageId}/suggestions/undismiss`, { await api.post(`/api/images/${imageId}/suggestions/undismiss`, {
body: { tag_id: suggestion.canonical_tag_id } body: { tag_id: suggestion.canonical_tag_id }
}) })
@@ -254,8 +176,7 @@ export const useSuggestionsStore = defineStore('suggestions', () => {
} }
return { return {
byCategory, allByCategory, loading, error, byCategory, aboveByCategory, loading, error,
load, loadAll, accept, aliasAccept, removeAlias, dismiss, dismissRemaining, load, accept, dismiss, dismissRemaining, undismiss, findPending
undismiss, findPending
} }
}) })
+28
View File
@@ -31,6 +31,20 @@
<img :src="c.thumbnail_url" alt="" loading="lazy" /> <img :src="c.thumbnail_url" alt="" loading="lazy" />
</button> </button>
<div class="fc-ex__trail-actions"> <div class="fc-ex__trail-actions">
<!-- Reach (#1476): how far each step reaches past the anchor's cluster.
Raise it to break out of a dense signature without hitting Random. -->
<div
class="fc-ex__reach"
title="How far each step reaches — raise it to escape a dense cluster without going fully random"
>
<v-icon size="16" color="accent">mdi-map-marker-distance</v-icon>
<v-slider
:model-value="store.reach" @end="store.setReach"
:min="0" :max="1" :step="0.2" hide-details density="compact"
color="accent" class="fc-ex__reach-slider"
/>
<span class="fc-muted fc-ex__reach-label">{{ reachLabel }}</span>
</div>
<!-- Active retrain right where you tag: fold the +/- you just gave <!-- Active retrain right where you tag: fold the +/- you just gave
into the heads without a trip to Settings (the nightly beat is the into the heads without a trip to Settings (the nightly beat is the
passive cadence). --> passive cadence). -->
@@ -153,6 +167,12 @@ const modal = useModalStore()
const anchorId = computed(() => route.params.imageId || null) const anchorId = computed(() => route.params.imageId || null)
const isVideo = computed(() => !!store.anchor?.mime?.startsWith('video/')) const isVideo = computed(() => !!store.anchor?.mime?.startsWith('video/'))
const reachLabel = computed(() => {
const r = store.reach
if (r <= 0.15) return 'Near'
if (r <= 0.55) return 'Varied'
return 'Far'
})
// #1206: hovering a suggestion in the rail highlights the crop it came from on // #1206: hovering a suggestion in the rail highlights the crop it came from on
// the anchor image (same provide/inject as the modal viewer). // the anchor image (same provide/inject as the modal viewer).
@@ -296,6 +316,14 @@ onUnmounted(() => {
.fc-ex__trail-actions { .fc-ex__trail-actions {
margin-left: auto; display: flex; align-items: center; gap: 4px; flex: 0 0 auto; margin-left: auto; display: flex; align-items: center; gap: 4px; flex: 0 0 auto;
} }
.fc-ex__reach {
display: flex; align-items: center; gap: 6px;
margin-right: 8px;
}
.fc-ex__reach-slider { width: 96px; }
.fc-ex__reach-label {
font-size: 12px; min-width: 44px; text-align: left;
}
/* The three panes fill the remaining height; each scrolls on its own. /* The three panes fill the remaining height; each scrolls on its own.
grid-template-rows: minmax(0, 1fr) BOUNDS the single row to the container grid-template-rows: minmax(0, 1fr) BOUNDS the single row to the container
+36 -27
View File
@@ -16,25 +16,44 @@ function stubFetch(handler) {
}) })
} }
// Every suggestion is a canonical DB tag now (tagging-v2): a real id, flagged
// above/below its head's suggest threshold. No raw / creates-new / alias cases.
const sugg = (over = {}) => ({ const sugg = (over = {}) => ({
canonical_tag_id: 7, canonical_tag_id: 7,
display_name: 'Ichigo', display_name: 'Ichigo',
category: 'character', category: 'character',
score: 0.9, score: 0.9,
source: 'head', source: 'head',
creates_new_tag: false, above_threshold: true,
rejected: false, rejected: false,
...over, ...over,
}) })
// Seed the store through its own load/loadAll so currentImageId is set the // Seed the store through its single load() so currentImageId is set the way the
// way the panel sets it (accept() derives the image from it). // panel sets it (accept() derives the image from it).
async function seed(store, byCategory) { async function seed(store, byCategory) {
stubFetch(() => ({ status: 200, body: { by_category: byCategory } })) stubFetch(() => ({ status: 200, body: { by_category: byCategory } }))
await store.load(1) await store.load(1)
await store.loadAll(1)
} }
describe('suggestions store: load derives aboveByCategory', () => {
beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks())
it('one fetch backs both surfaces: byCategory is the full set, aboveByCategory the panel subset', async () => {
const s = useSuggestionsStore()
await seed(s, {
general: [
sugg({ canonical_tag_id: 1, display_name: 'sword', category: 'general' }),
sugg({ canonical_tag_id: 2, display_name: 'faint', category: 'general', above_threshold: false }),
],
})
expect(s.byCategory.general).toHaveLength(2) // dropdown sees all
expect(s.aboveByCategory.general).toHaveLength(1) // panel sees above-threshold only
expect(s.aboveByCategory.general[0].display_name).toBe('sword')
})
})
describe('suggestions store: findPending (manual add == accept, 2026-07-03)', () => { describe('suggestions store: findPending (manual add == accept, 2026-07-03)', () => {
beforeEach(() => setActivePinia(createPinia())) beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks()) afterEach(() => vi.restoreAllMocks())
@@ -45,15 +64,12 @@ describe('suggestions store: findPending (manual add == accept, 2026-07-03)', ()
expect(s.findPending('general', 'unrelated', 7)?.display_name).toBe('Ichigo') expect(s.findPending('general', 'unrelated', 7)?.display_name).toBe('Ichigo')
}) })
it('matches raw suggestions by kind + name, case-insensitively', async () => { it('matches by kind + name, case-insensitively', async () => {
const s = useSuggestionsStore() const s = useSuggestionsStore()
await seed(s, { await seed(s, {
general: [sugg({ general: [sugg({ canonical_tag_id: 12, display_name: 'Holding Sword', category: 'general' })],
canonical_tag_id: null, display_name: 'Holding Sword',
category: 'general', creates_new_tag: true,
})],
}) })
expect(s.findPending('general', 'holding sword')).toBeTruthy() expect(s.findPending('general', 'holding sword')?.canonical_tag_id).toBe(12)
// Kind must match: same name under another kind is a different tag. // Kind must match: same name under another kind is a different tag.
expect(s.findPending('character', 'holding sword')).toBeNull() expect(s.findPending('character', 'holding sword')).toBeNull()
}) })
@@ -66,37 +82,31 @@ describe('suggestions store: findPending (manual add == accept, 2026-07-03)', ()
}) })
}) })
describe('suggestions store: accept with a known tag id', () => { describe('suggestions store: accept', () => {
beforeEach(() => setActivePinia(createPinia())) beforeEach(() => setActivePinia(createPinia()))
afterEach(() => vi.restoreAllMocks()) afterEach(() => vi.restoreAllMocks())
it('POSTs only the accept endpoint (no tag create) and drops the row', async () => { it('POSTs only the accept endpoint (canonical id) and drops the row', async () => {
const s = useSuggestionsStore() const s = useSuggestionsStore()
await seed(s, { character: [sugg()] }) await seed(s, { character: [sugg()] })
const calls = [] const calls = []
stubFetch((url, init) => { stubFetch((url, init) => {
calls.push({ url, method: init?.method }) calls.push({ url, method: init?.method, body: init?.body })
return { status: 200, body: { accepted: true, tag_id: 7 } } return { status: 200, body: { accepted: true, tag_id: 7 } }
}) })
await s.accept(s.byCategory.character[0], { tagId: 7 }) await s.accept(s.byCategory.character[0])
expect(calls).toHaveLength(1) expect(calls).toHaveLength(1)
expect(calls[0].url).toContain('/api/images/1/suggestions/accept') expect(calls[0].url).toContain('/api/images/1/suggestions/accept')
expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 7 })
expect(s.byCategory.character).toHaveLength(0) expect(s.byCategory.character).toHaveLength(0)
expect(s.allByCategory.character).toHaveLength(0)
}) })
it('a RAW suggestion accepted with a known tagId skips create AND still drops its row', async () => { it('honors a known tagId and still drops the row by the suggestion identity', async () => {
// TagPanel's manual-create path: the tag was just created by // TagPanel's manual-add-matches-suggestion path passes the resolved tag id;
// host.createAndAdd, so accept must not create again — and the drop must // accept sends exactly it and drops the original suggestion row (which keys
// key off the original raw suggestion object, not the resolved id // off the suggestion object, not the passed id).
// (regression: spreading canonical_tag_id onto the suggestion changed its
// identity key and left the panel row behind).
const s = useSuggestionsStore() const s = useSuggestionsStore()
const raw = sugg({ await seed(s, { general: [sugg({ canonical_tag_id: 12, display_name: 'Holding Sword', category: 'general' })] })
canonical_tag_id: null, display_name: 'Holding Sword',
category: 'general', creates_new_tag: true,
})
await seed(s, { general: [raw] })
const calls = [] const calls = []
stubFetch((url, init) => { stubFetch((url, init) => {
calls.push({ url, body: init?.body }) calls.push({ url, body: init?.body })
@@ -107,6 +117,5 @@ describe('suggestions store: accept with a known tag id', () => {
expect(calls[0].url).toContain('/api/images/1/suggestions/accept') expect(calls[0].url).toContain('/api/images/1/suggestions/accept')
expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 42 }) expect(JSON.parse(calls[0].body)).toEqual({ tag_id: 42 })
expect(s.byCategory.general).toHaveLength(0) expect(s.byCategory.general).toHaveLength(0)
expect(s.allByCategory.general).toHaveLength(0)
}) })
}) })
+19
View File
@@ -0,0 +1,19 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:recommended"
],
"packageRules": [
{
"description": "Bundle the interlocking frontend build/test toolchain into ONE PR. vite, vitest, @vitejs/plugin-vue, happy-dom and @vue/test-utils are version-coupled — a lone major bump red-fails the build/tests until its peers land, so they must move together.",
"matchPackageNames": [
"vite",
"vitest",
"@vitejs/plugin-vue",
"happy-dom",
"@vue/test-utils"
],
"groupName": "frontend build/test toolchain"
}
]
}
+5 -5
View File
@@ -7,14 +7,14 @@ sqlalchemy[asyncio]>=2.0,<2.1
asyncpg>=0.31,<0.32 asyncpg>=0.31,<0.32
psycopg[binary]>=3.3,<3.4 psycopg[binary]>=3.3,<3.4
alembic>=1.18,<1.19 alembic>=1.18,<1.19
pgvector>=0.4,<0.5 pgvector>=0.5,<0.6
# Task queue # Task queue
celery>=5.6,<5.7 celery>=5.6,<5.7
redis>=7.4,<8.0 redis>=7.4,<8.0
# Crypto for credential storage (lands in FC-3, but pinned now for stability) # Crypto for credential storage (lands in FC-3, but pinned now for stability)
cryptography>=48,<49 cryptography>=49,<50
# Image handling (lands in FC-2) # Image handling (lands in FC-2)
pillow>=12,<13 pillow>=12,<13
@@ -30,10 +30,10 @@ yt-dlp>=2025.1
# Utilities # Utilities
python-dotenv>=1.2,<2.0 python-dotenv>=1.2,<2.0
structlog>=25.5,<26.0 structlog>=26.1,<26.2
# HTML sanitization for scraped post descriptions (FC-2d provenance) # HTML sanitization for scraped post descriptions (FC-2d provenance)
nh3>=0.2,<0.3 nh3>=0.3,<0.4
# Archive extraction (FC-2d-iii filesystem-import aid) # Archive extraction (FC-2d-iii filesystem-import aid)
rarfile>=4.2,<5 rarfile>=4.2,<5
@@ -42,4 +42,4 @@ py7zr>=1,<2
# Google Drive fetcher for off-platform file-host links (external downloads, # Google Drive fetcher for off-platform file-host links (external downloads,
# #830). Handles Drive's confirm-token + virus-scan interstitial. mega.nz uses # #830). Handles Drive's confirm-token + virus-scan interstitial. mega.nz uses
# the `megatools` binary instead (Debian apt pkg in the runtime image, not pip). # the `megatools` binary instead (Debian apt pkg in the runtime image, not pip).
gdown>=5,<6 gdown>=6,<7
+62
View File
@@ -184,6 +184,68 @@ async def test_rejects_bad_direction(client):
assert (await resp.get_json())["error"] == "invalid_direction" assert (await resp.get_json())["error"] == "invalid_direction"
@pytest.mark.asyncio
async def test_translation_override_rejects_bad_value(client, seeded_post):
_, _, post = seeded_post
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "nope"}
)
assert resp.status_code == 400
assert (await resp.get_json())["error"] == "invalid_override"
@pytest.mark.asyncio
async def test_translation_override_404_for_unknown(client):
resp = await client.post(
"/api/posts/999999/translation-override", json={"override": "original"}
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_translation_override_original_clears_and_keeps(client, db, seeded_post):
# 'keep original' clears a stored translation immediately (no Interpreter) and
# marks the post handled (== target). Asserts on the response, which reflects
# the endpoint's own committed session.
_, _, post = seeded_post
post.post_title_translated = "STALE"
post.description_translated = "STALE"
post.translated_source_lang = "de"
await db.commit()
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "original"}
)
assert resp.status_code == 200
body = await resp.get_json()
assert body["translation_override"] == "original"
assert body["applied"] == "cleared"
assert body["post_title_translated"] is None
assert body["translated_source_lang"] == "en"
@pytest.mark.asyncio
async def test_translation_override_force_queues_when_disabled(client, seeded_post):
# Translation disabled (default) → can't translate inline; the override is
# saved and the post is queued (columns NULLed) for the next sweep.
_, _, post = seeded_post
resp = await client.post(
f"/api/posts/{post.id}/translation-override", json={"override": "force"}
)
assert resp.status_code == 200
body = await resp.get_json()
assert body["translation_override"] == "force"
assert body["applied"] == "queued"
assert body["translated_source_lang"] is None
@pytest.mark.asyncio
async def test_list_exposes_translation_override(client, seeded_post):
resp = await client.get("/api/posts")
assert resp.status_code == 200
body = await resp.get_json()
assert body["items"][0]["translation_override"] == "auto" # default
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_detail_returns_uncapped_thumbnails(client, db): async def test_detail_returns_uncapped_thumbnails(client, db):
"""Feed query caps thumbnails at 6 for previews; detail endpoint """Feed query caps thumbnails at 6 for previews; detail endpoint
+90 -1
View File
@@ -53,6 +53,26 @@ async def test_translation_settings_defaults_and_patch(client):
"/api/settings/import", json={"interpreter_base_url": 123})).status_code == 400 "/api/settings/import", json={"interpreter_base_url": 123})).status_code == 400
@pytest.mark.asyncio
async def test_translation_min_confidence_default_and_patch(client):
# #155: the latin-script acceptance floor is an operator-tunable setting,
# default 0.9 (stricter than the old hardcoded 0.8).
body = await (await client.get("/api/settings/import")).get_json()
assert body["translation_min_confidence"] == 0.9
ok = await client.patch(
"/api/settings/import", json={"translation_min_confidence": 0.95}
)
assert ok.status_code == 200
assert (await ok.get_json())["translation_min_confidence"] == 0.95
# Out-of-range + wrong-type are rejected.
assert (await client.patch(
"/api/settings/import", json={"translation_min_confidence": 1.5})).status_code == 400
assert (await client.patch(
"/api/settings/import", json={"translation_min_confidence": "high"})).status_code == 400
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_translation_status_defaults(client): async def test_translation_status_defaults(client):
# Off + no URL → no network health call, healthy False (#143). # Off + no URL → no network health call, healthy False (#143).
@@ -61,6 +81,34 @@ async def test_translation_status_defaults(client):
assert body["base_url_set"] is False assert body["base_url_set"] is False
assert body["healthy"] is False assert body["healthy"] is False
assert "untranslated_count" in body assert "untranslated_count" in body
assert body["active"] is False # no sweep running
assert body["last_run"] is None
@pytest.mark.asyncio
async def test_translation_status_reports_active_and_last_run(client, db):
# A running translate/retranslate TaskRun → active True; the most recent
# finished one → last_run (task basename + status).
from datetime import UTC, datetime
from backend.app.models import TaskRun
db.add(TaskRun(
celery_task_id="r-run", queue="maintenance_long",
task_name="backend.app.tasks.translation.retranslate_posts",
started_at=datetime.now(UTC), status="running",
))
db.add(TaskRun(
celery_task_id="r-done", queue="maintenance_long",
task_name="backend.app.tasks.translation.translate_posts",
started_at=datetime.now(UTC), finished_at=datetime.now(UTC),
status="success",
))
await db.commit()
body = await (await client.get("/api/settings/translation/status")).get_json()
assert body["active"] is True
assert body["last_run"]["task"] == "translate_posts"
assert body["last_run"]["status"] == "success"
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -85,9 +133,10 @@ async def test_translation_test_connection(client, monkeypatch):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_translation_run_enqueues_when_configured(client, monkeypatch): async def test_translation_run_enqueues_when_configured(client, monkeypatch):
sink = {}
monkeypatch.setattr( monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.delay", "backend.app.tasks.translation.translate_posts.delay",
lambda *a, **k: type("R", (), {"id": "t1"})(), lambda *a, **k: sink.update(k) or type("R", (), {"id": "t1"})(),
) )
await client.patch("/api/settings/import", json={ await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan", "translation_enabled": True, "interpreter_base_url": "http://i.lan",
@@ -95,6 +144,46 @@ async def test_translation_run_enqueues_when_configured(client, monkeypatch):
resp = await client.post("/api/settings/translation/run") resp = await client.post("/api/settings/translation/run")
assert resp.status_code == 202 assert resp.status_code == 202
assert (await resp.get_json())["celery_task_id"] == "t1" assert (await resp.get_json())["celery_task_id"] == "t1"
assert sink["drain"] is True # "Translate now" chases the whole backlog
@pytest.mark.asyncio
async def test_translation_probe_requires_text(client):
resp = await client.post(
"/api/settings/translation/probe", json={"text": " "})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_requires_base_url(client):
# Text given but no Interpreter URL saved → nothing to probe against.
resp = await client.post(
"/api/settings/translation/probe", json={"text": "hello"})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_returns_detection(client, monkeypatch):
# The diagnostic surfaces detected language + confidence + result, unsaved.
monkeypatch.setattr(
"backend.app.api.settings.ic.translate",
lambda texts, **k: {
"translations": ["Work in Progress"],
"detected_lang": "de", "detected_confidence": 71.5,
"engine": "llm", "engine_version": "v1",
},
)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post(
"/api/settings/translation/probe", json={"text": "WIP"})
assert resp.status_code == 200
body = await resp.get_json()
assert body["detected_lang"] == "de"
assert body["detected_confidence"] == 71.5
assert body["translated"] == "Work in Progress"
assert body["target"] == "en"
def _fake_retranslate(monkeypatch, sink): def _fake_retranslate(monkeypatch, sink):
+1 -9
View File
@@ -52,6 +52,7 @@ async def test_get_suggestions(client, db):
general = body["by_category"].get("general", []) general = body["by_category"].get("general", [])
s2 = next(x for x in general if x["canonical_tag_id"] == tag.id) s2 = next(x for x in general if x["canonical_tag_id"] == tag.id)
assert s2["source"] == "head" assert s2["source"] == "head"
assert s2["above_threshold"] is True # ~0.73 clears the 0.5 suggest cut
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -114,12 +115,3 @@ async def test_undismiss_reverses_rejection(client, db):
f"/api/images/{img.id}/suggestions/undismiss", json={"tag_id": tag.id} f"/api/images/{img.id}/suggestions/undismiss", json={"tag_id": tag.id}
) )
assert resp2.status_code == 204 assert resp2.status_code == 204
@pytest.mark.asyncio
async def test_alias_requires_fields(client, db):
img = await _img(db)
resp = await client.post(
f"/api/images/{img.id}/suggestions/alias", json={"alias_string": "x"}
)
assert resp.status_code == 400
+9 -5
View File
@@ -82,24 +82,28 @@ async def _system_tag(db, name):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_scroll_hides_presentation_chrome_by_default(db): async def test_scroll_hides_presentation_chrome_by_default(db):
# banner / editor screenshot (presentation system tags) are hidden from the # banner (chrome) is hidden from the default gallery; wip AND editor screenshot
# default gallery; wip (also a system tag) is NOT — it's real, in-progress # (the PROCESS system tags) are NOT — they're real art / process shots that stay
# art (milestone 141). Seeded system tags survive the harness TRUNCATE. # visible (milestone 141 + #1464). Seeded system tags survive the harness TRUNCATE.
imgs = await _seed_images(db, 3, sha_prefix="p") imgs = await _seed_images(db, 4, sha_prefix="p")
banner = await _system_tag(db, "banner") banner = await _system_tag(db, "banner")
wip = await _system_tag(db, "wip") wip = await _system_tag(db, "wip")
editor = await _system_tag(db, "editor screenshot")
await db.execute(image_tag.insert().values( await db.execute(image_tag.insert().values(
image_record_id=imgs[0].id, tag_id=banner.id, source="manual")) image_record_id=imgs[0].id, tag_id=banner.id, source="manual"))
await db.execute(image_tag.insert().values( await db.execute(image_tag.insert().values(
image_record_id=imgs[1].id, tag_id=wip.id, source="manual")) image_record_id=imgs[1].id, tag_id=wip.id, source="manual"))
await db.execute(image_tag.insert().values(
image_record_id=imgs[3].id, tag_id=editor.id, source="manual"))
await db.flush() await db.flush()
svc = GalleryService(db) svc = GalleryService(db)
# Default: the banner image is hidden; the wip image + the plain image stay. # Default: only the banner image is hidden; wip + editor + plain all stay.
default_ids = {i.id for i in (await svc.scroll(cursor=None, limit=10)).images} default_ids = {i.id for i in (await svc.scroll(cursor=None, limit=10)).images}
assert imgs[0].id not in default_ids # banner hidden assert imgs[0].id not in default_ids # banner hidden
assert imgs[1].id in default_ids # wip visible assert imgs[1].id in default_ids # wip visible
assert imgs[2].id in default_ids # plain visible assert imgs[2].id in default_ids # plain visible
assert imgs[3].id in default_ids # editor screenshot visible (#1464)
# include_hidden surfaces the banner image (the Hidden view). # include_hidden surfaces the banner image (the Hidden view).
shown = {i.id for i in ( shown = {i.id for i in (
+17
View File
@@ -170,6 +170,23 @@ async def test_similar_respects_limit(db):
assert len(res) == 2 assert len(res) == 2
@pytest.mark.asyncio
async def test_similar_exclude_ids_drops_walked(db):
"""Explore passes its breadcrumb as exclude_ids so already-walked images
aren't re-served as neighbours (#1476). reach>0 runs cleanly too (small pool
→ the sampler passes through)."""
src = await _img(db, 1, _vec(1, 0))
walked = await _img(db, 2, _vec(1, 0.05))
fresh = await _img(db, 3, _vec(1, 0.3))
svc = GalleryService(db)
res = await svc.similar(src.id, limit=10, exclude_ids=[walked.id])
ids = {i.id for i in res}
assert walked.id not in ids
assert fresh.id in ids
res_reach = await svc.similar(src.id, limit=10, reach=1.0)
assert fresh.id in {i.id for i in res_reach}
# --- API --- # --- API ---
@pytest.mark.asyncio @pytest.mark.asyncio
+4
View File
@@ -34,9 +34,13 @@ def test_translate_maps_batch_in_order(monkeypatch):
out = ic.translate(["ねこが可愛い", "金髪ギャル!"], base_url="http://i.lan") out = ic.translate(["ねこが可愛い", "金髪ギャル!"], base_url="http://i.lan")
assert out["translations"] == ["The cat is cute", "Blonde gal!"] assert out["translations"] == ["The cat is cute", "Blonde gal!"]
assert out["detected_lang"] == "ja" assert out["detected_lang"] == "ja"
assert out["detected_confidence"] == 0.98 # surfaced for the detection guard
assert out["engine_version"] == "ollama:x:12b" assert out["engine_version"] == "ollama:x:12b"
assert captured["json"]["q"] == ["ねこが可愛い", "金髪ギャル!"] assert captured["json"]["q"] == ["ねこが可愛い", "金髪ギャル!"]
assert captured["json"]["target"] == "en" assert captured["json"]["target"] == "en"
# Source MUST stay "auto": an explicit source makes Interpreter skip detection
# and report confidence 1.0, which would defeat the acceptance gate.
assert captured["json"]["source"] == "auto"
def test_translate_passthrough_unchanged(monkeypatch): def test_translate_passthrough_unchanged(monkeypatch):
+6 -3
View File
@@ -62,9 +62,8 @@ async def test_head_suggestion_surfaces_for_matching_image(db):
s = general[0] s = general[0]
assert s.canonical_tag_id == tag.id assert s.canonical_tag_id == tag.id
assert s.source == "head" assert s.source == "head"
assert s.creates_new_tag is False
assert s.via_alias is False and s.raw_name is None
assert s.score > 0.5 assert s.score > 0.5
assert s.above_threshold is True # ~0.73 clears the 0.5 suggest cut
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -109,7 +108,10 @@ async def test_threshold_override_surfaces_below_cut(db):
svc = SuggestionService(db) svc = SuggestionService(db)
assert svc and not (await svc.for_image(img.id)).by_category.get("general") assert svc and not (await svc.for_image(img.id)).by_category.get("general")
flooded = await svc.for_image(img.id, threshold_override=0.0) flooded = await svc.for_image(img.id, threshold_override=0.0)
assert any(s.canonical_tag_id == tag.id for s in flooded.by_category["general"]) s = next(s for s in flooded.by_category["general"] if s.canonical_tag_id == tag.id)
# Included by the floor, but flagged below its own cut (0.5 < 0.6) — this is
# what lets the dropdown show it while the panel hides it.
assert s.above_threshold is False
@pytest.mark.asyncio @pytest.mark.asyncio
@@ -281,6 +283,7 @@ async def test_ccip_character_surfaces_in_rail(db):
if c.canonical_tag_id == raven.id if c.canonical_tag_id == raven.id
) )
assert m.source == "ccip" assert m.source == "ccip"
assert m.above_threshold is True # CCIP only returns matches above its cut
# --- #1206 Step 4: on-demand grounding for ALREADY-APPLIED tag chips --------- # --- #1206 Step 4: on-demand grounding for ALREADY-APPLIED tag chips ---------
+6 -6
View File
@@ -15,7 +15,7 @@ from backend.app.models import (
from backend.app.models.tag import image_tag from backend.app.models.tag import image_tag
from backend.app.services.ml.heads import ( from backend.app.services.ml.heads import (
auto_apply_sweep, auto_apply_sweep,
presentation_auto_apply_sweep, system_tag_auto_apply_sweep,
) )
pytestmark = pytest.mark.integration pytestmark = pytest.mark.integration
@@ -70,7 +70,7 @@ def test_presentation_sweep_hides_chrome(db_sync):
_head(db_sync, banner.id, 0, weight=3.0) _head(db_sync, banner.id, 0, weight=3.0)
img = _img(db_sync, "a" * 64, _emb(0)) img = _img(db_sync, "a" * 64, _emb(0))
db_sync.commit() db_sync.commit()
res = presentation_auto_apply_sweep(db_sync) res = system_tag_auto_apply_sweep(db_sync, mode="chrome")
assert res["n_applied"] == 1 assert res["n_applied"] == 1
assert _source(db_sync, img.id, banner.id) == "presentation_auto" assert _source(db_sync, img.id, banner.id) == "presentation_auto"
@@ -86,7 +86,7 @@ def test_presentation_sweep_hard_skips_valued_image(db_sync):
db_sync.execute(image_tag.insert().values( db_sync.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=content.id, source="manual")) image_record_id=img.id, tag_id=content.id, source="manual"))
db_sync.commit() db_sync.commit()
res = presentation_auto_apply_sweep(db_sync) res = system_tag_auto_apply_sweep(db_sync, mode="chrome")
assert res["n_applied"] == 0 assert res["n_applied"] == 0
assert _source(db_sync, img.id, banner.id) is None assert _source(db_sync, img.id, banner.id) is None
@@ -102,7 +102,7 @@ def test_presentation_sweep_flags_conflict(db_sync):
_head(db_sync, content.id, 0, weight=1.0) # content head also fires _head(db_sync, content.id, 0, weight=1.0) # content head also fires
img = _img(db_sync, "c" * 64, _emb(0)) img = _img(db_sync, "c" * 64, _emb(0))
db_sync.commit() db_sync.commit()
res = presentation_auto_apply_sweep(db_sync) res = system_tag_auto_apply_sweep(db_sync, mode="chrome")
assert res["n_applied"] == 1 assert res["n_applied"] == 1
assert res["n_flagged"] == 1 assert res["n_flagged"] == 1
assert _source(db_sync, img.id, banner.id) == "presentation_auto" assert _source(db_sync, img.id, banner.id) == "presentation_auto"
@@ -123,7 +123,7 @@ def test_presentation_sweep_disabled_is_noop(db_sync):
_head(db_sync, banner.id, 0, weight=3.0) _head(db_sync, banner.id, 0, weight=3.0)
img = _img(db_sync, "d" * 64, _emb(0)) img = _img(db_sync, "d" * 64, _emb(0))
db_sync.commit() db_sync.commit()
res = presentation_auto_apply_sweep(db_sync) res = system_tag_auto_apply_sweep(db_sync, mode="chrome")
assert res["n_applied"] == 0 assert res["n_applied"] == 0
assert _source(db_sync, img.id, banner.id) is None assert _source(db_sync, img.id, banner.id) is None
@@ -134,7 +134,7 @@ def test_presentation_sweep_ignores_wip(db_sync):
_head(db_sync, wip.id, 0, weight=3.0) _head(db_sync, wip.id, 0, weight=3.0)
img = _img(db_sync, "e" * 64, _emb(0)) img = _img(db_sync, "e" * 64, _emb(0))
db_sync.commit() db_sync.commit()
res = presentation_auto_apply_sweep(db_sync) res = system_tag_auto_apply_sweep(db_sync, mode="chrome")
assert res["n_applied"] == 0 assert res["n_applied"] == 0
assert _source(db_sync, img.id, wip.id) is None assert _source(db_sync, img.id, wip.id) is None
+188
View File
@@ -0,0 +1,188 @@
"""Process-group auto-apply sweep (#1464): wip / editor screenshot auto-tag at a
flat threshold with a PROVISIONAL source (`process_auto`) so the head never trains
on its own output, and stay VISIBLE (unlike chrome). Mirrors the chrome guards.
numpy-only (no sklearn), tested directly via the sync session."""
import pytest
from sqlalchemy import select
from backend.app.models import (
ImageRecord,
MLSettings,
PresentationReview,
Tag,
TagHead,
TagKind,
)
from backend.app.models.tag import image_tag
from backend.app.services.ml.heads import system_tag_auto_apply_sweep
from backend.app.services.ml.training_data import _ids_with_tag
pytestmark = pytest.mark.integration
def _emb(slot: int) -> list[float]:
v = [0.0] * 1152
v[slot] = 3.0
return v
def _img(db, sha: str, emb) -> ImageRecord:
img = ImageRecord(
path=f"/images/{sha}.jpg", sha256=sha, size_bytes=1, mime="image/jpeg",
width=1, height=1, origin="imported_filesystem",
integrity_status="unknown", siglip_embedding=emb,
)
db.add(img)
db.flush()
return img
def _head(db, tag_id: int, slot: int, *, weight=1.0):
s = db.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
w = [0.0] * 1152
w[slot] = weight
db.add(TagHead(
tag_id=tag_id, embedding_version=s.embedder_model_version,
weights=w, bias=0.0, suggest_threshold=0.5, auto_apply_threshold=0.5,
n_pos=60, n_neg=90, ap=0.9, precision_cv=0.98, recall=0.7,
))
def _system_tag(db, name):
return db.execute(
select(Tag).where(Tag.is_system.is_(True), Tag.name == name)
).scalar_one()
def _enable_process(db):
# process auto-apply is opt-in (default False) — turn it on for these tests.
db.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one().process_auto_apply_enabled = True
def _source(db, image_id, tag_id):
return db.execute(
select(image_tag.c.source)
.where(image_tag.c.image_record_id == image_id)
.where(image_tag.c.tag_id == tag_id)
).scalar_one_or_none()
def test_process_sweep_applies_wip_and_editor(db_sync):
_enable_process(db_sync)
wip = _system_tag(db_sync, "wip")
editor = _system_tag(db_sync, "editor screenshot")
_head(db_sync, wip.id, 0, weight=3.0)
_head(db_sync, editor.id, 1, weight=3.0)
w_img = _img(db_sync, "a" * 64, _emb(0))
e_img = _img(db_sync, "b" * 64, _emb(1))
db_sync.commit()
res = system_tag_auto_apply_sweep(db_sync, mode="process")
assert res["n_applied"] == 2
assert _source(db_sync, w_img.id, wip.id) == "process_auto"
assert _source(db_sync, e_img.id, editor.id) == "process_auto"
def test_process_sweep_disabled_by_default_is_noop(db_sync):
# process_auto_apply_enabled defaults False (opt-in) — no enable = no-op.
wip = _system_tag(db_sync, "wip")
_head(db_sync, wip.id, 0, weight=3.0)
img = _img(db_sync, "c" * 64, _emb(0))
db_sync.commit()
res = system_tag_auto_apply_sweep(db_sync, mode="process")
assert res["n_applied"] == 0
assert _source(db_sync, img.id, wip.id) is None
def test_process_sweep_skips_valued_image(db_sync):
# Guard 1: never auto-apply to an image the operator already content-tagged.
_enable_process(db_sync)
wip = _system_tag(db_sync, "wip")
_head(db_sync, wip.id, 0, weight=3.0)
content = Tag(name="mychar", kind=TagKind.character)
db_sync.add(content)
db_sync.flush()
img = _img(db_sync, "d" * 64, _emb(0))
db_sync.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=content.id, source="manual"))
db_sync.commit()
res = system_tag_auto_apply_sweep(db_sync, mode="process")
assert res["n_applied"] == 0
assert _source(db_sync, img.id, wip.id) is None
def test_process_sweep_flags_conflict_with_process_mode(db_sync):
# Guard 2: also scores high on a content head → still applied, but flagged
# for review with mode='process' (the ring-loud guard).
_enable_process(db_sync)
wip = _system_tag(db_sync, "wip")
_head(db_sync, wip.id, 0, weight=3.0)
content = Tag(name="looksreal", kind=TagKind.general)
db_sync.add(content)
db_sync.flush()
_head(db_sync, content.id, 0, weight=1.0)
img = _img(db_sync, "e" * 64, _emb(0))
db_sync.commit()
res = system_tag_auto_apply_sweep(db_sync, mode="process")
assert res["n_applied"] == 1
assert res["n_flagged"] == 1
flag = db_sync.execute(
select(PresentationReview).where(
PresentationReview.image_record_id == img.id,
PresentationReview.tag_id == wip.id,
)
).scalar_one()
assert flag.mode == "process"
assert flag.conflict_tag_id == content.id
def test_process_auto_source_never_trains_head(db_sync):
# The runaway break: provisional wip tags (process sweep 'process_auto', soft
# title 'wip_title_soft') are NOT training positives; a HARD title-heuristic /
# manual one IS. So the head learns only from trusted labels, never its own
# output or the low-precision sketch/doodle tier (#1464 + #1474).
wip = _system_tag(db_sync, "wip")
auto_img = _img(db_sync, "f" * 64, _emb(0))
soft_img = _img(db_sync, "9" * 64, _emb(2))
title_img = _img(db_sync, "0" * 64, _emb(1))
db_sync.execute(image_tag.insert().values(
image_record_id=auto_img.id, tag_id=wip.id, source="process_auto"))
db_sync.execute(image_tag.insert().values(
image_record_id=soft_img.id, tag_id=wip.id, source="wip_title_soft"))
db_sync.execute(image_tag.insert().values(
image_record_id=title_img.id, tag_id=wip.id, source="wip_title"))
db_sync.commit()
positives = set(_ids_with_tag(db_sync, wip.id))
assert title_img.id in positives # trusted HARD label trains the head
assert auto_img.id not in positives # its own auto-applied output does NOT
assert soft_img.id not in positives # low-precision soft tier does NOT
def test_soft_wip_conflict_audit_flags_ring_loud(db_sync):
# A soft-tagged image (sketch/doodle title) that ALSO scores high on a content
# head is probably finished art mis-tagged — flagged for review; a quiet one is not.
from backend.app.services.ml.heads import soft_wip_conflict_audit
s = db_sync.execute(select(MLSettings).where(MLSettings.id == 1)).scalar_one()
s.process_conflict_threshold = 0.6
wip = _system_tag(db_sync, "wip")
content = Tag(name="looksreal", kind=TagKind.general)
db_sync.add(content)
db_sync.flush()
_head(db_sync, content.id, 0, weight=1.0) # sigmoid(1)=0.73 > 0.6 conflict
ring = _img(db_sync, "1" * 64, _emb(0)) # scores on the content head
quiet = _img(db_sync, "2" * 64, _emb(5)) # orthogonal → 0.5 < 0.6
for img in (ring, quiet):
db_sync.execute(image_tag.insert().values(
image_record_id=img.id, tag_id=wip.id, source="wip_title_soft"))
db_sync.commit()
res = soft_wip_conflict_audit(db_sync)
assert res["n_flagged"] == 1
flag = db_sync.execute(
select(PresentationReview).where(PresentationReview.image_record_id == ring.id)
).scalar_one()
assert flag.mode == "process"
assert flag.conflict_tag_id == content.id
assert db_sync.execute(
select(PresentationReview).where(PresentationReview.image_record_id == quiet.id)
).scalar_one_or_none() is None
+32
View File
@@ -0,0 +1,32 @@
"""Explore reach sampler (#1476) — pure unit tests for `_reach_sample`, which picks
a distance-rank spread so the neighbour pool handed to MMR spans near→mid-far and
the walk can escape a dense cluster. No DB. `_reach_sample` only indexes rows, so
plain ints stand in for the (ImageRecord, ...) tuples."""
from backend.app.services.gallery_service import _reach_sample
def test_reach_zero_or_negative_passes_through():
rows = list(range(1000))
assert _reach_sample(rows, 40, 0.0) is rows
assert _reach_sample(rows, 40, -1.0) is rows
def test_small_pool_passes_through():
# n <= want (limit*8 = 320) → nothing to reach into.
rows = list(range(50))
assert _reach_sample(rows, 40, 1.0) is rows
def test_higher_reach_reaches_deeper_ranks():
rows = list(range(1000))
near = _reach_sample(rows, 40, 0.2)
far = _reach_sample(rows, 40, 1.0)
# Both keep the nearest rank (stride starts at 0) so you can still tag the cluster.
assert near[0] == 0
assert far[0] == 0
# But higher reach samples genuinely farther ranks.
assert max(far) > max(near)
assert max(far) >= 900 # reach=1 spans (almost) the whole pool
assert max(near) <= 550 # reach=0.2 stays in the near half
# Never runs past the pool.
assert max(far) <= len(rows) - 1
+207 -1
View File
@@ -5,7 +5,11 @@ from sqlalchemy import select
from backend.app.models import Artist, ImportSettings, Post from backend.app.models import Artist, ImportSettings, Post
from backend.app.services import interpreter_client as ic from backend.app.services import interpreter_client as ic
from backend.app.tasks.translation import retranslate_posts, translate_posts from backend.app.tasks.translation import (
_RETRANSLATE_COUNTDOWN,
retranslate_posts,
translate_posts,
)
pytestmark = pytest.mark.integration pytestmark = pytest.mark.integration
@@ -115,6 +119,35 @@ def test_translate_posts_passthrough_english_marks_handled(db_sync, monkeypatch)
assert p.post_title_translated is None # ...but nothing to show assert p.post_title_translated is None # ...but nothing to show
def test_translate_posts_mixed_language_translates_nonenglish_field(db_sync, monkeypatch):
# English title + Japanese description: the description is still translated
# even though the title is already English (per-field detection, not the old
# aggregate first-item bail). Source lang comes from the translated field.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def fake_translate(texts, **k):
t = texts[0]
if t.isascii(): # already English → passthrough
return {"translations": [t], "detected_lang": "en",
"engine": "none", "engine_version": None}
return {"translations": [f"EN:{t}"], "detected_lang": "ja",
"engine": "llm", "engine_version": "v9"}
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", fake_translate)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="hello", desc="ねこ")
_enable(db_sync)
db_sync.commit()
assert "translated=1" in translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # English title untouched
assert p.description_translated == "EN:ねこ" # Japanese desc translated
assert p.translated_source_lang == "ja" # from the translated field
assert p.translation_engine_version == "v9"
def test_translate_posts_disabled_is_noop(db_sync, monkeypatch): def test_translate_posts_disabled_is_noop(db_sync, monkeypatch):
_patch(monkeypatch, db_sync) _patch(monkeypatch, db_sync)
a = _artist(db_sync) a = _artist(db_sync)
@@ -162,6 +195,160 @@ def test_translate_posts_interrupt_reenqueues_after_backoff(db_sync, monkeypatch
assert calls[0][1]["countdown"] == 30 assert calls[0][1]["countdown"] == 30
def _mock_ok(monkeypatch):
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": "ja", "engine": "llm", "engine_version": "v1",
},
)
def test_translate_posts_drain_chases_tail(db_sync, monkeypatch):
# "Translate now" (drain=True): with the chunk capped at 1 and 2 untranslated
# posts, the first chunk re-enqueues itself — drain preserved — to finish the
# backlog rather than waiting for the next beat.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts(drain=True)
assert len(calls) == 1 # one re-enqueue for the tail
args, kw = calls[0]
assert args[1] == {"drain": True} # drain flag carried forward
assert kw["countdown"] == _RETRANSLATE_COUNTDOWN
def test_translate_posts_beat_single_chunk_no_tail_chase(db_sync, monkeypatch):
# The periodic beat (drain defaults False) does exactly ONE chunk and never
# re-enqueues — the 8h cadence drives the rest.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts() # drain=False
assert calls == [] # no self-re-enqueue
def _mock_lang(monkeypatch, *, lang, conf, ver="v1"):
# Interpreter is up and returns a translation with the given detected language
# + confidence, so the acceptance gate can be exercised end-to-end.
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": lang, "detected_confidence": conf,
"engine": "llm", "engine_version": ver,
},
)
def test_translate_posts_rejects_low_confidence_latin(db_sync, monkeypatch):
# A short English title Interpreter mis-labels as German with LOW confidence is
# NOT stored — the original is kept, and the post is marked handled (source ==
# target) so the sweep won't revisit it.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.42)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Nami Heroines - WIP Part 1", ext="p1")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # mis-flag rejected, kept
assert p.translated_source_lang == "en" # marked handled (== target)
def test_translate_posts_accepts_high_confidence_latin(db_sync, monkeypatch):
# A confident latin-script detection (real German) clears the floor → stored.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.98)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Das Mädchen mit dem Perlenohrring", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:Das Mädchen mit dem Perlenohrring"
assert p.translated_source_lang == "de"
def test_translate_posts_accepts_low_confidence_cjk(db_sync, monkeypatch):
# CJK is script-detected and trusted regardless of the reported number — a
# zh 0.75 (e.g. pure-kanji Japanese) is still translated, not rejected.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="zh", conf=0.75)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="猫が可愛い", ext="p3")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:猫が可愛い"
assert p.translated_source_lang == "zh"
def test_translate_posts_force_override_translates_below_floor(db_sync, monkeypatch):
# override='force' stores the translation even below the confidence floor —
# rescuing a legitimately-foreign title the gate would otherwise skip.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.42) # below the 0.90 default floor
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Kurz", ext="p1")
p.translation_override = "force"
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:Kurz" # forced through despite 0.42
assert p.translated_source_lang == "de"
def test_translate_posts_original_override_keeps_original(db_sync, monkeypatch):
# override='original' never translates — even a confident (0.98) detection is
# kept as the original. The sweep marks it handled (== target) and stores
# nothing; Interpreter isn't consulted for the decision.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.98) # would be accepted under auto
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Das Mädchen", ext="p1")
p.translation_override = "original"
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # kept original
assert p.translated_source_lang == "en" # marked handled
def _mock_new_model(monkeypatch, *, ver="v2"): def _mock_new_model(monkeypatch, *, ver="v2"):
"""Interpreter is up and translates with a NEW engine version.""" """Interpreter is up and translates with a NEW engine version."""
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True) monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
@@ -192,6 +379,25 @@ def test_retranslate_resets_and_reruns(db_sync, monkeypatch):
assert p.translation_engine_version == "v2" assert p.translation_engine_version == "v2"
def test_retranslate_leaves_keep_original_posts(db_sync, monkeypatch):
# A 'keep original' post survives a Re-translate-all: _reset_translations skips
# override='original', so it stays handled + untranslated instead of being
# re-run (which would re-introduce the mis-flag the operator killed).
_patch(monkeypatch, db_sync)
_mock_new_model(monkeypatch)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
p.translation_override = "original"
p.translated_source_lang = "en" # handled by a prior 'keep original' apply
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
db_sync.refresh(p)
assert p.translated_source_lang == "en" # not reset
assert p.post_title_translated is None # not re-translated
def test_retranslate_artist_scope_leaves_others(db_sync, monkeypatch): def test_retranslate_artist_scope_leaves_others(db_sync, monkeypatch):
# Scoping to one artist must not touch another artist's translations. # Scoping to one artist must not touch another artist's translations.
_patch(monkeypatch, db_sync) _patch(monkeypatch, db_sync)
+48
View File
@@ -0,0 +1,48 @@
"""Translation acceptance gate (#1376, #155) — pure unit tests for ``_accept``,
which decides whether curator stores Interpreter's translation or keeps the
original. Curator does no detection of its own; it only thresholds Interpreter's
reported detected language + confidence against the operator-tunable floor
(Scribe rule 133). No DB, no network."""
from backend.app.tasks.translation import _DEFAULT_MIN_LATIN_CONFIDENCE, _accept
def test_accept_trusts_cjk_regardless_of_confidence_or_floor():
# ja/ko/zh are script-detected — accepted even at a low reported number and
# even against a high floor (pure-kanji Japanese can come back labelled zh
# ~0.75, still translated).
assert _accept("ja", None) is True
assert _accept("ja", 0.10, 0.99) is True
assert _accept("ko", 0.99) is True
assert _accept("zh", 0.75) is True
assert _accept("zh-CN", 0.20) is True # region suffix is still CJK
def test_accept_default_floor_is_strict():
# Default floor is 0.90 (#155): the ~0.86 band — where genuine German AND
# mis-flagged short English collide — is now REJECTED (the whole point of the
# stricter default). Genuine high-confidence non-English (~1.0) still passes.
assert _accept("de", 0.98) is True
assert _accept("de", 0.86) is False # the collision band, now cut
assert _accept("de", _DEFAULT_MIN_LATIN_CONFIDENCE) is True # at floor
assert _accept("de", _DEFAULT_MIN_LATIN_CONFIDENCE - 0.01) is False
assert _accept("fr", 0.40) is False # short mis-flag
def test_accept_honors_an_explicit_floor():
# The floor is a parameter now (fed from ImportSettings). A looser floor keeps
# the 0.86 band; a stricter one rejects even a fairly confident detection.
assert _accept("de", 0.86, 0.80) is True
assert _accept("de", 0.86, 0.95) is False
assert _accept("de", 0.96, 0.95) is True
def test_accept_is_case_insensitive():
assert _accept("JA", None) is True
assert _accept("DE", 0.40) is False
def test_accept_missing_confidence_fails_open():
# No confidence reported → don't silently drop a translation.
assert _accept("de", None) is True
assert _accept("", None) is True
assert _accept("de", None, 0.99) is True # fail-open beats a high floor
+76
View File
@@ -0,0 +1,76 @@
"""Title-based WIP matcher (task #1458) — pure unit tests for
``matches_wip_title``, the precision-first heuristic that decides whether a post
title explicitly declares work-in-progress. No DB, no network.
Precision matters more than recall here: a false positive applies the ``wip``
system tag, which HIDES a finished post from the Explore browse — so the
negative cases (substrings like ``swipe`` / ``wiped``) are the load-bearing ones.
"""
import pytest
from backend.app.services.wip_title import matches_soft_wip_title, matches_wip_title
@pytest.mark.parametrize("title", [
"WIP",
"wip",
"WiP",
"Nami Heroines - WIP Part 1", # the real-world example from the gate work
"sketch (WIP)",
"[WIP] new piece",
"WIP: colour test",
"cool art WIP2", # trailing digit = "WIP part 2", still WIP
"W.I.P.",
"W.I.P",
"work in progress",
"Work In Progress",
"commission work-in-progress",
"big_project_work_in_progress",
"final touches, wip",
])
def test_matches_positive(title):
assert matches_wip_title(title) is True
@pytest.mark.parametrize("title", [
None,
"",
"a quick swipe left", # 'wip' inside swipe — must NOT match
"she wiped the counter", # wiped
"wiping down the desk", # wiping
"unwiped surface",
"progressive rock cover", # 'progress' inside progressive, no 'work in'
"workin on it", # not the full phrase
"finished at last",
"Kawips diner", # 'wip' mid-word
"swipright",
"quick sketch of Nami", # soft cue — NOT a HARD WIP match
])
def test_matches_negative(title):
assert matches_wip_title(title) is False
@pytest.mark.parametrize("title", [
"quick sketch",
"Nami sketch",
"morning doodle",
"some doodles",
"sketches from today",
"a little scribble",
"SKETCH",
])
def test_soft_matches_positive(title):
assert matches_soft_wip_title(title) is True
@pytest.mark.parametrize("title", [
None,
"",
"sketchbook tour", # 'sketch' inside sketchbook — must NOT match
"kadoodle mascot", # 'doodle' mid-word
"the final piece",
"WIP", # a HARD cue is not a SOFT cue
"prescribed colours", # 'scrib' inside prescribed — must NOT match
])
def test_soft_matches_negative(title):
assert matches_soft_wip_title(title) is False
+121
View File
@@ -0,0 +1,121 @@
"""Title-based WIP auto-tagging (task #1458) — integration tests for the apply
helpers and the operator-triggered backfill sweep against a real DB. The
importer's live hook is a thin call to these same tested pieces (matcher +
apply), so the DB-facing behaviour is covered here.
"""
import pytest
from sqlalchemy import select
from backend.app.celery_app import celery
from backend.app.models import Artist, ImageProvenance, ImageRecord, Post, Source
from backend.app.models.tag import image_tag
from backend.app.services.wip_title import (
WIP_TITLE_SOFT_SOURCE,
apply_wip_image_tags,
resolve_wip_tag_id,
)
pytestmark = pytest.mark.integration
_N = 0
@pytest.fixture(autouse=True)
def eager():
celery.conf.task_always_eager = True
yield
celery.conf.task_always_eager = False
def _img(db_sync):
global _N
_N += 1
rec = ImageRecord(
path=f"/images/w/{_N}.jpg", sha256=f"w{_N:063d}",
size_bytes=1, mime="image/jpeg", width=1, height=1,
origin="imported_filesystem", integrity_status="unknown",
)
db_sync.add(rec)
db_sync.flush()
return rec
def _post(db_sync, *, title, slug, ext):
a = Artist(name=slug.upper(), slug=slug)
db_sync.add(a)
db_sync.flush()
s = Source(artist_id=a.id, platform="patreon", url=f"https://patreon.test/{slug}")
db_sync.add(s)
db_sync.flush()
p = Post(source_id=s.id, artist_id=a.id, external_post_id=ext, post_title=title)
db_sync.add(p)
db_sync.flush()
return s, p
def _link(db_sync, rec, post, source):
db_sync.add(ImageProvenance(
image_record_id=rec.id, post_id=post.id, source_id=source.id,
))
db_sync.flush()
def _wip_source(db_sync, image_id, tag_id):
return db_sync.execute(
select(image_tag.c.source).where(
image_tag.c.image_record_id == image_id,
image_tag.c.tag_id == tag_id,
)
).scalar_one_or_none()
def test_resolve_wip_tag_id_present(db_sync):
# The `wip` system tag is seeded by migration 0075 and restored between tests.
assert resolve_wip_tag_id(db_sync) is not None
def test_apply_is_idempotent_and_stamps_source(db_sync):
tag_id = resolve_wip_tag_id(db_sync)
rec = _img(db_sync)
db_sync.commit()
assert apply_wip_image_tags(db_sync, [rec.id], tag_id) == 1
# Second apply is a no-op (ON CONFLICT DO NOTHING) — never disturbs the row.
assert apply_wip_image_tags(db_sync, [rec.id], tag_id) == 0
assert _wip_source(db_sync, rec.id, tag_id) == "wip_title"
def test_backfill_tags_only_wip_titled_posts(db_sync):
from backend.app.tasks.maintenance import backfill_wip_title_tags
tag_id = resolve_wip_tag_id(db_sync)
wip_rec = _img(db_sync)
plain_rec = _img(db_sync)
swipe_rec = _img(db_sync)
s1, wip_post = _post(db_sync, title="Cool piece - WIP Part 1", slug="wipy", ext="1")
s2, plain_post = _post(db_sync, title="Finished commission", slug="doney", ext="2")
s3, swipe_post = _post(db_sync, title="a quick swipe", slug="swipey", ext="3")
_link(db_sync, wip_rec, wip_post, s1)
_link(db_sync, plain_rec, plain_post, s2)
_link(db_sync, swipe_rec, swipe_post, s3) # 'swipe' must NOT trip the matcher
db_sync.commit()
assert backfill_wip_title_tags.apply().get() == 1
assert _wip_source(db_sync, wip_rec.id, tag_id) == "wip_title"
assert _wip_source(db_sync, plain_rec.id, tag_id) is None
assert _wip_source(db_sync, swipe_rec.id, tag_id) is None
# Idempotent: a second sweep finds the tag already present and applies nothing.
assert backfill_wip_title_tags.apply().get() == 0
def test_apply_soft_source_stamps_wip_title_soft(db_sync):
# The soft tier (#1474) stamps a distinct provisional source.
tag_id = resolve_wip_tag_id(db_sync)
rec = _img(db_sync)
db_sync.commit()
assert apply_wip_image_tags(
db_sync, [rec.id], tag_id, source=WIP_TITLE_SOFT_SOURCE
) == 1
assert _wip_source(db_sync, rec.id, tag_id) == "wip_title_soft"