feat(translation): tunable acceptance floor (0.90) + per-post sticky override (#155)

The gate at a fixed 0.80 couldn't catch the real pain: Interpreter (fresh ==
cached, verified by probe) confidently mis-detects short ASCII English like
"... WIP Part 1" as German at 0.86 — above the floor — so it was accepted and a
re-translate reproduced it. Confidence alone can't separate the 0.86 collision
(genuine German lands there too), and single-word mis-flags sit at a confident
1.0 no floor catches.

Two operator-approved levers:

- Acceptance floor is now a live Settings value (ImportSettings.
  translation_min_confidence, default 0.90; surfaced in the Translation card), so
  it's tunable without a redeploy. _accept takes the threshold as a parameter.

- Per-post sticky override (Post.translation_override: auto/force/original).
  'force' stores a translation even below the floor (rescue a skipped
  legit-foreign title); 'original' keeps the original and clears any stored
  translation (kill a confident mis-flag no floor catches). The sweep honors it
  on every run and _reset_translations skips 'original', so the choice survives a
  Re-translate-all. POST /api/posts/<id>/translation-override applies it
  immediately (translate now when the service is up, else queue for the sweep).
  UI: PostTranslationControl on the posts-feed card.

Migration 0084 (both columns + a CHECK on the override). The feed + provenance
serializers expose translation_override.

With a stricter floor the rollback finally works: raise it -> Re-translate all ->
the 0.86 mis-flags are rejected and restored to the original; force /
keep-original handle the residual either way.

Tests: gate thresholds against the param (0.86 rejected at 0.90, explicit-floor
cases); sweep force/original + re-translate-skips-original; override endpoint
(validation, original clears, force queues when disabled, feed exposes it);
settings min_confidence default/save/validate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy

backend/app/tasks/translation.py

@@ -60,14 +60,17 @@ _INTERRUPT_BACKOFF_MAX = 900
 # language scores low; require it to clear this floor, else keep the original.
 # Calibrated 2026-07-09 against fresh (uncached) probes once Interpreter returned
 # real langdetect confidence: genuine German detected at 1.0, while a correctly-
-# detected but ambiguous latin string dipped to 0.86 — so the floor sits at 0.80,
-# below that band, to keep legitimate ambiguous non-English while still rejecting
-# genuinely-unsure guesses. Real langdetect also fixed the original mis-flag at
-# the source (short English titles now detect as English → passthrough), so this
+# detected but ambiguous latin string dipped to 0.86 — so a single constant can't win.
+# The floor is now operator-tunable (ImportSettings.translation_min_confidence,
+# milestone 155) with a stricter 0.90 default; per-post overrides
+# (Post.translation_override) rescue the false negatives it skips. Genuine German
+# and mis-flagged short English both land ~0.86, and single-word mis-flags sit at
+# a confident 1.0 no floor catches. Real langdetect fixed some cases at the
+# source (some short English titles now detect as English → passthrough), so this
 # floor is a safety net, not the primary fix. Re-tune via the "Test translation"
 # box (send fresh text — cache hits report 1.0).
 _CJK_LANGS = frozenset({"ja", "ko", "zh"})
-_MIN_LATIN_CONFIDENCE = 0.80
+_DEFAULT_MIN_LATIN_CONFIDENCE = 0.90
 
 
 def _interrupt_backoff(retry_after) -> int:
@@ -100,10 +103,10 @@ def translate_posts(drain: bool = False) -> str:
         ready = _translation_config(session)
         if isinstance(ready, str):
             return ready
-        base_url, target = ready
+        base_url, target, min_confidence = ready
         posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN)
         status, translated, retry_after = _translate_batch(
-            session, posts, base_url, target,
+            session, posts, base_url, target, min_confidence,
         )
         # Drain mode (manual "Translate now"): chase the tail until the backlog is
         # zero, so one press clears the whole pile instead of one 300-chunk.
@@ -152,7 +155,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
         ready = _translation_config(session)
         if isinstance(ready, str):
             return ready
-        base_url, target = ready
+        base_url, target, min_confidence = ready
 
         if not _reset_done:
             n = _reset_translations(session, artist_ids)
@@ -163,7 +166,7 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
 
         posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN)
         status, translated, retry_after = _translate_batch(
-            session, posts, base_url, target,
+            session, posts, base_url, target, min_confidence,
         )
 
         # run-until-done: chase the tail when the chunk finished cleanly.
@@ -201,17 +204,19 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
 
 
 def _translation_config(session):
-    """Resolve (base_url, target) if translation is enabled + healthy, else a
-    short status string ("disabled" / "interpreter unavailable") for the caller
-    to return. Centralises the guard so translate/retranslate agree exactly."""
+    """Resolve (base_url, target, min_confidence) if translation is enabled +
+    healthy, else a short status string ("disabled" / "interpreter unavailable")
+    for the caller to return. Centralises the guard so translate/retranslate agree
+    exactly, including the operator-tunable acceptance floor."""
     cfg = ImportSettings.load_sync(session)
     if not cfg.translation_enabled or not cfg.interpreter_base_url.strip():
         return "disabled"
     base_url = cfg.interpreter_base_url.strip()
     target = (cfg.translation_target_lang or "en").strip() or "en"
+    min_confidence = cfg.translation_min_confidence
     if not ic.health(base_url):
         return "interpreter unavailable"
-    return base_url, target
+    return base_url, target, min_confidence
 
 
 def _untranslated_filter(stmt, artist_ids):
@@ -244,10 +249,13 @@ def _reset_translations(session, artist_ids) -> int:
     """Clear the stored translation columns for scoped, already-handled posts so
     the untranslated sweep re-runs them. Only touches rows that were translated
     (translated_source_lang IS NOT NULL) — untranslated rows are already NULL.
-    Returns the row count reset (commits it)."""
+    Skips 'keep original' posts (translation_override = 'original') so that choice
+    survives a Re-translate-all (milestone 155). Returns the row count reset
+    (commits it)."""
     stmt = (
         update(Post)
         .where(Post.translated_source_lang.is_not(None))
+        .where(Post.translation_override != "original")
         .values(
             post_title_translated=None,
             description_translated=None,
@@ -263,7 +271,7 @@ def _reset_translations(session, artist_ids) -> int:
     return n
 
 
-def _translate_batch(session, posts, base_url: str, target: str):
+def _translate_batch(session, posts, base_url: str, target: str, min_confidence: float):
     """Translate each post in the chunk with a per-post commit (an interrupted
     run keeps progress). Returns (status, translated, retry_after) where status is
     one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout";
@@ -272,7 +280,7 @@ def _translate_batch(session, posts, base_url: str, target: str):
     translated = 0
     for post in posts:
         try:
-            translated += _translate_one(session, post, base_url, target)
+            translated += _translate_one(session, post, base_url, target, min_confidence)
             session.commit()
         except ic.InterpreterUnavailable as e:
             session.rollback()
@@ -296,28 +304,36 @@ def _summary(status: str, translated: int, scanned: int) -> str:
     return f"translated={translated} scanned={scanned}"
 
 
-def _accept(language: str, confidence) -> bool:
+def _accept(
+    language: str, confidence, min_confidence: float = _DEFAULT_MIN_LATIN_CONFIDENCE,
+) -> bool:
     """Should curator store Interpreter's translation, or keep the original?
     Consumes ONLY Interpreter's own detection (curator does none, Scribe rule
     133): CJK languages are trusted regardless of the reported number; a
-    latin-script detection must clear ``_MIN_LATIN_CONFIDENCE``. A missing
-    confidence fails OPEN (accept), so a service that omits the field never causes
-    translations to be silently dropped."""
+    latin-script detection must clear ``min_confidence`` (the operator-tunable
+    ImportSettings.translation_min_confidence). A missing confidence fails OPEN
+    (accept), so a service that omits the field never silently drops a
+    translation. Per-post ``force`` overrides bypass this entirely (see
+    ``_translate_one``)."""
     if (language or "").split("-", 1)[0].lower() in _CJK_LANGS:
         return True
     if confidence is None:
         return True
-    return confidence >= _MIN_LATIN_CONFIDENCE
+    return confidence >= min_confidence
 
 
-def _translate_field(text: str, base_url: str, target: str):
+def _translate_field(
+    text: str, base_url: str, target: str, min_confidence: float, force: bool = False,
+):
     """Translate ONE field independently. Returns (translated, source_lang,
     engine_version), or (None, None, None) when there's nothing to store — empty
     text, already the target language, a passthrough (engine "none"), or a
     latin-script detection the acceptance gate rejects as too low-confidence (kept
-    as the original). Per-field (not aggregate first-item) detection so a
-    non-English description still gets translated when the title is already
-    English (mixed-language posts)."""
+    as the original). ``force`` (a per-post 'force' override) bypasses the
+    confidence gate so a legitimately-foreign title Interpreter is only ~0.86 sure
+    about is still stored; a genuine passthrough stores nothing regardless (there
+    is no translation to force). Per-field (not aggregate first-item) detection so
+    a non-English description still translates when the title is already English."""
     if not text:
         return None, None, None
     res = ic.translate([text], base_url=base_url, target=target)
@@ -327,23 +343,30 @@ def _translate_field(text: str, base_url: str, target: str):
     # Trust only Interpreter's own detection (Scribe rule 133): keep the original
     # when a latin-script detection doesn't clear the confidence floor, so a short
     # English title mis-labelled as e.g. German isn't rewritten into the archive.
-    if not _accept(detected, res.get("detected_confidence")):
+    # A 'force' override skips the gate — the operator is overriding a rejection.
+    if not force and not _accept(
+        detected, res.get("detected_confidence"), min_confidence,
+    ):
         return None, None, None
     return res["translations"][0], detected, res["engine_version"]
 
 
-def _translate_one(session, post, base_url: str, target: str) -> int:
-    """Translate a post's title/description in place, each field independently.
-    Returns 1 if it stored at least one translation, 0 when the post is all
-    passthrough/empty (still marks it handled so the sweep won't revisit it)."""
-    title = (post.post_title or "").strip()
-    desc = (html_to_plain(post.description) if post.description else "") or ""
-    desc = desc.strip()
-    title_tr, title_lang, title_ev = _translate_field(title, base_url, target)
-    desc_tr, desc_lang, desc_ev = _translate_field(desc, base_url, target)
+def _store_translation(post, title_res, desc_res, target: str) -> int:
+    """Apply per-field (translated, source_lang, engine_version) results to a post
+    in place. Returns 1 if a translation was stored, 0 when the post is all
+    passthrough/empty — in which case the translated columns are CLEARED and the
+    post is marked handled (translated_source_lang = target). Clearing (not just
+    leaving) matters on re-apply: a 'keep original' override, or a re-translate
+    that now rejects a previously-accepted mis-flag, must remove the stale
+    translation, not keep it."""
+    title_tr, title_lang, title_ev = title_res
+    desc_tr, desc_lang, desc_ev = desc_res
     if title_tr is None and desc_tr is None:
-        # Nothing non-target (or nothing to translate) → handled, store nothing.
+        post.post_title_translated = None
+        post.description_translated = None
         post.translated_source_lang = target
+        post.translation_engine_version = None
+        post.translated_at = None
         return 0
     post.post_title_translated = title_tr
     post.description_translated = desc_tr
@@ -353,3 +376,23 @@ def _translate_one(session, post, base_url: str, target: str) -> int:
     post.translation_engine_version = title_ev or desc_ev
     post.translated_at = datetime.now(UTC)
     return 1
+
+
+def _translate_one(
+    session, post, base_url: str, target: str, min_confidence: float,
+) -> int:
+    """Translate a post's title/description in place, each field independently,
+    honoring the per-post override (milestone 155): 'original' keeps the original
+    (clears any stored translation, no Interpreter call); 'force' translates even
+    below the confidence floor; 'auto' (default) runs the acceptance gate. Returns
+    1 if it stored a translation, else 0 (still marks the post handled so the sweep
+    won't revisit it)."""
+    if post.translation_override == "original":
+        return _store_translation(post, (None, None, None), (None, None, None), target)
+    force = post.translation_override == "force"
+    title = (post.post_title or "").strip()
+    desc = (html_to_plain(post.description) if post.description else "") or ""
+    desc = desc.strip()
+    title_res = _translate_field(title, base_url, target, min_confidence, force=force)
+    desc_res = _translate_field(desc, base_url, target, min_confidence, force=force)
+    return _store_translation(post, title_res, desc_res, target)