feat(translation): accept/reject gate on Interpreter's detection confidence (#1376)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m46s

Interpreter now returns a real per-detection confidence (source stays "auto"),
so curator can reject the mis-detections it was blindly storing — e.g. a short
English title mis-labelled as German and rewritten into the archive.

The gate consumes ONLY Interpreter's own reported detection — curator does no
language detection of its own (Scribe rule 133): a field is stored when the
engine actually translated it AND either the detected language is CJK
(script-detected, reliably high — ja/ko/zh trusted outright, incl. pure-kanji
Japanese that lands as zh ~0.75) or the reported confidence clears a
latin-script floor (_MIN_LATIN_CONFIDENCE = 0.90). A latin detection below the
floor keeps the original and marks the post handled; a missing confidence fails
open. The client already sent source="auto" and parsed confidence, so this is
purely the gate + tests.

Tests: pinned interpreter-client test now asserts source stays "auto"; new
pure-unit gate tests (CJK trusted / latin floor / case-insensitive / fail-open)
in the fast lane; end-to-end reject-low-latin, accept-high-latin,
accept-low-cjk sweeps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
This commit is contained in:
2026-07-09 16:17:56 -04:00
parent ffdbdbaf07
commit 7ddf94220d
4 changed files with 139 additions and 3 deletions
+62
View File
@@ -252,6 +252,68 @@ def test_translate_posts_beat_single_chunk_no_tail_chase(db_sync, monkeypatch):
assert calls == [] # no self-re-enqueue
def _mock_lang(monkeypatch, *, lang, conf, ver="v1"):
# Interpreter is up and returns a translation with the given detected language
# + confidence, so the acceptance gate can be exercised end-to-end.
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": lang, "detected_confidence": conf,
"engine": "llm", "engine_version": ver,
},
)
def test_translate_posts_rejects_low_confidence_latin(db_sync, monkeypatch):
# A short English title Interpreter mis-labels as German with LOW confidence is
# NOT stored — the original is kept, and the post is marked handled (source ==
# target) so the sweep won't revisit it.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.42)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Nami Heroines - WIP Part 1", ext="p1")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # mis-flag rejected, kept
assert p.translated_source_lang == "en" # marked handled (== target)
def test_translate_posts_accepts_high_confidence_latin(db_sync, monkeypatch):
# A confident latin-script detection (real German) clears the floor → stored.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="de", conf=0.98)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="Das Mädchen mit dem Perlenohrring", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:Das Mädchen mit dem Perlenohrring"
assert p.translated_source_lang == "de"
def test_translate_posts_accepts_low_confidence_cjk(db_sync, monkeypatch):
# CJK is script-detected and trusted regardless of the reported number — a
# zh 0.75 (e.g. pure-kanji Japanese) is still translated, not rejected.
_patch(monkeypatch, db_sync)
_mock_lang(monkeypatch, lang="zh", conf=0.75)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="猫が可愛い", ext="p3")
_enable(db_sync)
db_sync.commit()
translate_posts()
db_sync.refresh(p)
assert p.post_title_translated == "EN:猫が可愛い"
assert p.translated_source_lang == "zh"
def _mock_new_model(monkeypatch, *, ver="v2"):
"""Interpreter is up and translates with a NEW engine version."""
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)