Compare commits

..

12 Commits

Author SHA1 Message Date
bvandeusen ffdbdbaf07 feat(translation): 8h sweep + drain button + detection probe (#1376)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 46s
CI / integration (push) Successful in 3m55s
Throughput: translate_posts now runs every 8h (was daily) as the
steady-state cadence for newly-imported posts, and the Settings
"Translate now" button runs it in drain mode (run-until-done, no reset)
so one press clears the whole untranslated backlog instead of a single
300-post chunk. The interrupt/backoff re-enqueue now preserves the drain
flag so a bulk drain resumes cleanly after an Interpreter restart.

Misdetection groundwork: surface the detector's confidence from the
Interpreter client (it was in the detectedLanguage payload but discarded)
and add a read-only "Test translation" box — POST /settings/translation/
probe + TranslationCard UI — that shows detected language + confidence +
engine + result for pasted text, without saving. Lets the operator see
why a short/abbreviation-heavy English title gets mis-detected so the
detection guard (min-length + confidence floor) can be tuned from real
numbers. The guard itself follows once the mis-detected cases are probed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CgZP9v2otxVJymiYsnVuMy
2026-07-08 20:13:49 -04:00
bvandeusen a017771621 feat(ui): thin kind-coloured border on tag chips so they don't wash out
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 32s
CI / integration (push) Successful in 3m49s
The tonal fill (esp. character = info) is intentionally faint and blends into
the dark tag rail. Add a thin border in each chip's own kind colour via
color-mix on currentColor (the tonal chip's themed foreground), defining the
edge without changing the fill. Theme-aware in both light and dark.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:31:44 -04:00
bvandeusen 25e555cab6 fix(ui): stop leading tag-chip icon clipping on the left edge
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m50s
The larger size=default chip widened Vuetify's negative start-margin on the
leading kind-icon, placing it left of the .v-chip__content box whose
overflow:hidden (the name-truncation guard) then clipped its left edge.
Zero the icon's negative inline-start margin so it sits inside the clip box;
the chip's 12px padding keeps a comfortable left inset.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 10:29:28 -04:00
bvandeusen 3c7ab44e74 feat(translation): per-field language detection for mixed-language posts
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m43s
_translate_one translated [title, description] in ONE Interpreter call and keyed
the whole-post passthrough on the aggregate detected_lang (the FIRST item). So an
English title + non-English description detected "en" and marked the post handled,
leaving the description untranslated. Now each field is translated independently
(its own detected_lang / passthrough) and the non-target field is stored on its
own; translated_source_lang reflects the translated field's language.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:31:04 -04:00
bvandeusen 06f98acf3e feat(tagging): bolder auto-tag accept/reject + larger tag chips in the rail
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m45s
The in-pill ✓/✗ on unconfirmed auto-tags read muted — a faint colored icon on a
tonal chip (worst on character tags, kind=info) that only lit up on hover. Make
them solid green/red circles with a white glyph (22px, icon 15), mirroring the
Suggestions rail's verdict buttons so accept/reject read identically. Also bump
the applied-tag chips from size=small to default and the leading kind icon to
match — bigger, clearer tags throughout the rail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 23:17:18 -04:00
bvandeusen 4371ddb7e7 feat(translation): live re-translate progress in the Settings card
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m45s
translation/status now reports `active` (a translate/retranslate sweep is
running, from the TaskRun table) and `last_run` (the most recent finished run's
task + status). The Settings card polls live while a sweep runs, showing a
spinner + "Translating… N remaining" that ticks down, and flags a last run that
ended in error/timeout. No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:20:32 -04:00
bvandeusen 40cc11be5b feat(deploy): container healthchecks + Swarm rolling-update auto-rollback
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 29s
CI / integration (push) Successful in 3m45s
web gets a /api/health liveness check; workers a lenient celery-ping check. A
shared deploy policy (update_config order=start-first, failure_action=rollback,
monitor 90s; rollback_config; restart_policy) means a bad image that never goes
healthy is rolled back automatically instead of taking the service down. Ignored
by plain `docker compose up` (deploy: is swarm-only), so the dev override is
unaffected. Assumes prod deploys from this file via docker stack deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 0b78264d62 feat(maintenance): daily janitor for orphaned .part/.partial staging files
Downloads/imports stage into <name>.part / <name>.partial then os.replace() into
place, so a kill mid-write leaves a discardable temp — never a corrupt final.
cleanup_orphaned_temp_files sweeps ones left behind under the images root, only
older than 6h so an in-flight download's staging file is never removed. Daily beat.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:15:33 -04:00
bvandeusen 9eae636047 feat(translation): pooled Interpreter session + manual sweep resumes after drain
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m45s
- interpreter_client: shared requests.Session with a connect-only retry
  (connect=2, no status retries — we map 429/5xx ourselves) so a proxy reload
  is smoothed and the keep-alive connection is pooled across the sweep.
- translate_posts: on an interrupt (drain), re-enqueue after the Retry-After
  hint / default backoff instead of waiting for the daily beat; self-terminates
  via the health gate. Steady-state one-chunk-per-run on success is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen f8105046dc feat(explore): exclude WIP-tagged work from the Explore rabbit-hole
Explore's neighbour grid (/api/gallery/similar → gallery_service.similar) now
takes an Explore-only exclude_wip flag that drops `wip` system-tagged images
from the candidates, alongside the banner/editor presentation tags. The
gallery's own "similar" button is unchanged (keeps wip, #1274) — only the
Explore store passes exclude_wip=1. The anchor itself may still be a WIP; only
neighbours are filtered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 22:10:05 -04:00
bvandeusen d631ed023c style(translation): use datetime.UTC alias (ruff UP017)
CI / lint (push) Successful in 4s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m45s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:04:17 -04:00
bvandeusen c64261593d feat(ops): graceful shutdown — worker stop-grace + Interpreter drain resilience
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 35s
CI / integration (push) Successful in 3m42s
Deploys (docker SIGTERM→SIGKILL, default 10s) were killing Celery jobs
mid-flight. Give in-flight work room to drain and make interrupted work
resume cleanly instead of stalling.

- docker-compose.yml: stop_grace_period per lane (web 30s / worker 90s /
  scheduler 60s / maintenance-long 180s / ml-worker 120s) so warm shutdown
  can actually drain before SIGKILL.
- celery_app.py: task_reject_on_worker_lost=True — a task killed past the
  grace window is re-queued (safe: idempotent + chunked, recovery sweeps
  re-drive stragglers).
- interpreter_client.py: map 429/5xx (502/503/504) → InterpreterUnavailable
  and parse Retry-After (delta-seconds or HTTP-date); a draining Interpreter
  behind a reverse proxy no longer raises an opaque HTTPError.
- translation.py: thread retry_after out of _translate_batch; retranslate_posts
  resumes after the Retry-After hint (or 60s default, capped 900s) on an
  interrupt with _reset_done=True, self-terminating via the health gate.
- tests: 429/5xx mapping + Retry-After parse; interrupt-resume + default backoff.

No migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 21:01:00 -04:00
17 changed files with 867 additions and 85 deletions
+5 -1
View File
@@ -145,6 +145,9 @@ async def similar():
filters, _sort = _parse_filters()
except (KeyError, ValueError):
return jsonify({"error": "similar_to query param required"}), 400
# Explore passes exclude_wip=1 to also drop work-in-progress from the
# rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274).
exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True")
# post_id is the exclusive post-detail view — not a similarity scope.
# include_hidden is a gallery-browse flag; similar() has its OWN presentation
# exclusion (a similarity-quality concern, #1274), so drop it here (#141).
@@ -154,7 +157,8 @@ async def similar():
async with get_session() as session:
svc = GalleryService(session)
try:
images = await svc.similar(image_id=similar_to, limit=limit, **scope)
images = await svc.similar(
image_id=similar_to, limit=limit, exclude_wip=exclude_wip, **scope)
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
if images is None:
+68 -2
View File
@@ -16,6 +16,7 @@ from ..models import (
ImportTask,
Post,
Tag,
TaskRun,
)
from ..services import interpreter_client as ic
@@ -306,6 +307,10 @@ async def translation_status():
"""For the Settings card: is it on, is a URL set, is the service reachable,
and how many posts still await translation. Health runs the sync client in a
thread so the event loop isn't blocked."""
translation_tasks = (
"backend.app.tasks.translation.translate_posts",
"backend.app.tasks.translation.retranslate_posts",
)
async with get_session() as session:
cfg = await ImportSettings.load(session)
untranslated = (await session.execute(
@@ -315,6 +320,21 @@ async def translation_status():
Post.post_title.is_not(None), Post.description.is_not(None),
))
)).scalar_one()
# Live progress: is a sweep running now, and what did the last one do?
# (run-until-done re-enqueues itself, so `active` stays true across a
# bulk re-translate; `last_run` surfaces a completed run's outcome.)
active = (await session.execute(
select(func.count(TaskRun.id))
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.status == "running")
)).scalar_one()
last = (await session.execute(
select(TaskRun.task_name, TaskRun.status, TaskRun.finished_at)
.where(TaskRun.task_name.in_(translation_tasks))
.where(TaskRun.finished_at.is_not(None))
.order_by(TaskRun.finished_at.desc())
.limit(1)
)).first()
base_url = (cfg.interpreter_base_url or "").strip()
healthy = await asyncio.to_thread(ic.health, base_url) if base_url else False
return jsonify({
@@ -322,6 +342,12 @@ async def translation_status():
"base_url_set": bool(base_url),
"healthy": healthy,
"untranslated_count": int(untranslated),
"active": int(active) > 0,
"last_run": {
"task": last[0].rsplit(".", 1)[-1],
"status": last[1],
"finished_at": last[2].isoformat() if last[2] else None,
} if last else None,
})
@@ -338,9 +364,49 @@ async def translation_test():
return jsonify({"healthy": healthy})
@settings_bp.route("/settings/translation/probe", methods=["POST"])
async def translation_probe():
"""Diagnostic for the Settings 'Test translation' box: translate a pasted
snippet WITHOUT saving anything, returning what Interpreter *detected*
(language + confidence) alongside the result. Lets the operator see why a
given string was (mis-)detected — e.g. a short English title flagged as
another language — so a detection guard can be tuned from real numbers.
Read-only: no post is touched. Uses the currently-saved base URL + target."""
body = await request.get_json(silent=True)
body = body if isinstance(body, dict) else {}
text = (body.get("text") or "").strip()
if not text:
return jsonify({"error": "provide text to translate"}), 400
async with get_session() as session:
cfg = await ImportSettings.load(session)
base_url = (cfg.interpreter_base_url or "").strip()
if not base_url:
return jsonify({"error": "no Interpreter base URL is set"}), 400
target = (cfg.translation_target_lang or "en").strip() or "en"
try:
res = await asyncio.to_thread(
ic.translate, [text], base_url=base_url, target=target,
)
except ic.InterpreterUnavailable as e:
return jsonify({"error": f"Interpreter unavailable: {e}"}), 503
except ic.InterpreterBadRequest as e:
return jsonify({"error": f"Interpreter rejected the request: {e}"}), 400
translations = res.get("translations") or []
return jsonify({
"target": target,
"detected_lang": res.get("detected_lang"),
"detected_confidence": res.get("detected_confidence"),
"engine": res.get("engine"),
"engine_version": res.get("engine_version"),
"translated": translations[0] if translations else None,
})
@settings_bp.route("/settings/translation/run", methods=["POST"])
async def translation_run():
"""Enqueue the translate sweep now (the Settings 'Translate now' button)."""
"""Enqueue the translate sweep now (the Settings 'Translate now' button).
Runs in drain mode — run-until-done — so one press chases the whole
untranslated backlog to zero rather than a single 300-post chunk."""
async with get_session() as session:
cfg = await ImportSettings.load(session)
if not cfg.translation_enabled or not (cfg.interpreter_base_url or "").strip():
@@ -349,7 +415,7 @@ async def translation_run():
), 400
from ..tasks.translation import translate_posts
r = translate_posts.delay()
r = translate_posts.delay(drain=True)
return jsonify({"celery_task_id": r.id}), 202
+19 -2
View File
@@ -70,6 +70,14 @@ def make_celery() -> Celery:
},
# Heavy ML tasks need fair dispatch — see ImageRepo's precedent.
task_acks_late=True,
# Deploy graceful-shutdown safety: with acks_late, a task killed because
# it outran the container's stop-grace window (SIGKILL) is re-queued
# rather than silently lost. Safe because our long tasks are idempotent +
# chunked (translation per-post commit, downloads terminal-status, audits
# chunk) and the 5-min recovery sweeps re-drive anything left non-terminal
# — a re-run resumes cleanly and never corrupts. No redeliver-loop risk:
# heavy GPU work is tombstoned via gpu_queue, not run inline in a worker.
task_reject_on_worker_lost=True,
worker_prefetch_multiplier=1,
# Broker resilience (2026-06-24): a swarm overlay-network blip after a
# redeploy left Redis healthy but transiently unreachable, and a worker
@@ -107,6 +115,11 @@ def make_celery() -> Celery:
"task": "backend.app.tasks.maintenance.cleanup_old_tasks",
"schedule": 86400.0, # daily
},
"cleanup-orphaned-temp-files": {
"task": "backend.app.tasks.maintenance.cleanup_orphaned_temp_files",
"schedule": 86400.0, # daily — sweep .part/.partial left by a
# download/import killed mid-write (graceful-shutdown fallout)
},
"train-heads-nightly": {
"task": "backend.app.tasks.ml.scheduled_train_heads",
"schedule": 86400.0, # passive cadence; manual retrain stays available
@@ -165,9 +178,13 @@ def make_celery() -> Celery:
"task": "backend.app.tasks.ml.prune_presentation_reviews",
"schedule": 86400.0, # retention: drop resolved review flags >30d
},
"translate-posts-daily": {
"translate-posts-8h": {
"task": "backend.app.tasks.translation.translate_posts",
"schedule": 86400.0, # no-op unless translation configured + healthy
"schedule": 28800.0, # every 8h: steady-state cadence for the
# trickle of newly-imported posts (no-op unless translation
# configured + healthy). One bounded 300-chunk per fire — the
# one-time backlog drains via the "Translate now" button
# (drain=True, run-until-done), not this sweep.
},
"snapshot-head-metrics-daily": {
"task": "backend.app.tasks.maintenance.snapshot_head_metrics",
+4
View File
@@ -48,6 +48,10 @@ class TagKind(StrEnum):
# content. `wip` is real art: only the training pipelines exclude it.
SYSTEM_TAG_NAMES = ("wip", "banner", "editor screenshot")
PRESENTATION_SYSTEM_TAGS = ("banner", "editor screenshot")
# `wip` marks real-but-unfinished art. It's kept in the gallery's own "similar"
# results (#1274), but the Explore rabbit-hole opts to hide it (exclude_wip) so a
# browse doesn't keep surfacing work-in-progress (operator, 2026-07-08).
WIP_SYSTEM_TAG = "wip"
image_tag = Table(
"image_tag",
+9 -4
View File
@@ -31,7 +31,7 @@ from ..models import (
Tag,
TagPositiveConfirmation,
)
from ..models.tag import PRESENTATION_SYSTEM_TAGS, image_tag
from ..models.tag import PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG, image_tag
from .pagination import decode_cursor, encode_cursor
from .tag_query import (
fandom_join_alias,
@@ -715,6 +715,7 @@ class GalleryService:
platform: str | None = None,
untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None,
exclude_wip: bool = False,
) -> list[GalleryImage] | None:
"""Visual "more like this": images near `image_id`'s SigLIP embedding
(pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result
@@ -751,14 +752,18 @@ class GalleryService:
# Presentation images (banner / editor-screenshot system tags, #128)
# cluster on UI chrome rather than content, so near any one of them
# they'd fill the grid. Excluded from CANDIDATES only — the anchor
# itself may be a banner, and `wip` stays surfaced (real art; only
# the training pipelines exclude it).
# itself may be a banner. `wip` stays surfaced here by default (real art;
# only the training pipelines exclude it), but the Explore rabbit-hole
# passes exclude_wip to also drop work-in-progress (operator, 2026-07-08).
excluded_system_tags = PRESENTATION_SYSTEM_TAGS
if exclude_wip:
excluded_system_tags = (*PRESENTATION_SYSTEM_TAGS, WIP_SYSTEM_TAG)
presentation = (
select(image_tag.c.image_record_id)
.join(Tag, Tag.id == image_tag.c.tag_id)
.where(
Tag.is_system.is_(True),
Tag.name.in_(PRESENTATION_SYSTEM_TAGS),
Tag.name.in_(excluded_system_tags),
)
)
stmt = stmt.where(
+75 -11
View File
@@ -8,12 +8,24 @@ behind a reverse proxy). Full API contract: Scribe note #1347.
"""
from __future__ import annotations
from datetime import UTC, datetime
from email.utils import parsedate_to_datetime
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
class InterpreterUnavailable(Exception):
"""The translation engine is down / unreachable (HTTP 503 or a connection
error) — retry later, don't drop the item."""
"""The translation engine is down / unreachable / draining — a connection
error, HTTP 429, or a 5xx (commonly 502/503/504 through a reverse proxy while
the service restarts). Retry later, don't drop the item. ``retry_after`` holds
the server's Retry-After hint in seconds when it sent one, so the caller can
back off exactly as long as it's asked to."""
def __init__(self, message: str, *, retry_after: float | None = None):
super().__init__(message)
self.retry_after = retry_after
class InterpreterBadRequest(Exception):
@@ -24,13 +36,52 @@ def _url(base_url: str, path: str) -> str:
return f"{base_url.rstrip('/')}{path}"
def _parse_retry_after(resp) -> float | None:
"""Parse a Retry-After header (RFC 7231: delta-seconds or an HTTP-date) into
non-negative seconds, or None if absent/unparseable — so a gracefully-
draining Interpreter can tell Curator exactly how long to wait before it
tries again."""
raw = (resp.headers.get("Retry-After") or "").strip()
if not raw:
return None
try:
return max(0.0, float(int(raw))) # delta-seconds form
except ValueError:
pass
try: # HTTP-date form
when = parsedate_to_datetime(raw)
except (TypeError, ValueError):
return None
if when is None:
return None
if when.tzinfo is None:
when = when.replace(tzinfo=UTC)
return max(0.0, (when - datetime.now(UTC)).total_seconds())
# A shared session pools the keep-alive connection across the per-post sweep
# calls, and retries CONNECT failures only (connect=2, short backoff) — smoothing
# the instant a reverse proxy reloads. Status codes are deliberately NOT retried
# (status=0, raise_on_status=False): translate() maps 429/5xx → InterpreterUnavailable
# itself, and letting urllib3 retry a draining 503 would defeat the Retry-After
# backoff we honour upstream.
_retry = Retry(
total=None, connect=2, read=0, redirect=0, status=0,
backoff_factor=0.3, raise_on_status=False,
)
session = requests.Session()
_adapter = HTTPAdapter(max_retries=_retry)
session.mount("http://", _adapter)
session.mount("https://", _adapter)
def health(base_url: str, *, timeout: float = 5.0) -> bool:
"""True iff the Interpreter LLM engine is up. Any error (unset URL, network,
non-200, engine down) → False, so the sweep just no-ops rather than raising."""
if not base_url:
return False
try:
r = requests.get(_url(base_url, "/v1/health"), timeout=timeout)
r = session.get(_url(base_url, "/v1/health"), timeout=timeout)
except requests.RequestException:
return False
if r.status_code != 200:
@@ -45,22 +96,25 @@ def translate(
) -> dict:
"""Translate a batch. Returns::
{"translations": [str, ...], # SAME order & length as `texts`
"detected_lang": str | None, # aggregate: describes the FIRST item
{"translations": [str, ...], # SAME order & length as `texts`
"detected_lang": str | None, # aggregate: describes the FIRST item
"detected_confidence": float | None, # detector's confidence, if given
"engine": str | None,
"engine_version": str | None}
Interpreter batch metadata is aggregate (first item only) — fine for one
post's ``[title, description]`` batch since they share a language. Passthrough
items (already target-language / emoji-only) come back UNCHANGED in their
slot. Raises InterpreterUnavailable on 503 / connection error (retry later),
InterpreterBadRequest on 400. (Scribe note #1347.)
slot. Raises InterpreterUnavailable on a connection error / 429 / 5xx (retry
later, honouring Retry-After), InterpreterBadRequest on 400. (Scribe note
#1347.)
"""
if not texts:
return {"translations": [], "detected_lang": None,
"detected_confidence": None,
"engine": None, "engine_version": None}
try:
r = requests.post(
r = session.post(
_url(base_url, "/v1/translate"),
json={"q": list(texts), "source": source,
"target": target, "engine": "auto"},
@@ -68,10 +122,18 @@ def translate(
)
except requests.RequestException as e:
raise InterpreterUnavailable(str(e)) from e
if r.status_code == 503:
raise InterpreterUnavailable("interpreter engine unavailable")
if r.status_code == 400:
raise InterpreterBadRequest((r.json() or {}).get("error", "bad request"))
# A gracefully-draining service — often behind a reverse proxy — returns 429
# or a 5xx gateway error (502/503/504), not always a clean 503. Treat every
# one as "unavailable, retry later" and honour any Retry-After it sends, so a
# rolling Interpreter restart interrupts the sweep cleanly (instead of raising
# an opaque HTTPError) and resumes exactly when the service says it's ready.
if r.status_code == 429 or r.status_code >= 500:
raise InterpreterUnavailable(
f"interpreter unavailable (HTTP {r.status_code})",
retry_after=_parse_retry_after(r),
)
r.raise_for_status()
body = r.json() or {}
out = body.get("translatedText")
@@ -80,9 +142,11 @@ def translate(
if not isinstance(out, list):
out = [out]
interp = body.get("interpreter") or {}
detected = body.get("detectedLanguage") or {}
return {
"translations": out,
"detected_lang": (body.get("detectedLanguage") or {}).get("language"),
"detected_lang": detected.get("language"),
"detected_confidence": detected.get("confidence"),
"engine": interp.get("engine"),
"engine_version": interp.get("engine_version"),
}
+36
View File
@@ -79,6 +79,14 @@ VERIFY_PAGE = 200
FFPROBE_TIMEOUT_SECONDS = 10
TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h
TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days
# Orphaned staging files: downloads/imports stage into <name>.part / <name>.partial
# then os.replace() into place (importer / external_fetch / native_ingest_common /
# attachment_store), so a kill mid-write leaves a discardable temp, never a corrupt
# final. cleanup_orphaned_temp_files sweeps ones left behind; the min-age guard
# keeps it from deleting an in-flight download's staging file mid-write.
IMAGES_ROOT = Path("/images")
TEMP_STAGING_SUFFIXES = (".part", ".partial")
ORPHAN_TEMP_MIN_AGE_HOURS = 6
# Audit 2026-06-02: per-entity recovery sweep thresholds. Each must be
# > the entity's longest legitimate runtime (its task's time_limit + a
@@ -339,6 +347,34 @@ def cleanup_old_tasks() -> int:
return result.rowcount or 0
@celery.task(name="backend.app.tasks.maintenance.cleanup_orphaned_temp_files")
def cleanup_orphaned_temp_files() -> int:
"""Delete orphaned .part/.partial staging files under the images root, left by
a download/import killed mid-write. Only removes files older than
ORPHAN_TEMP_MIN_AGE_HOURS so an in-flight download's staging file is never
pulled out from under it. Returns the count removed."""
if not IMAGES_ROOT.is_dir():
return 0
cutoff = datetime.now(UTC).timestamp() - ORPHAN_TEMP_MIN_AGE_HOURS * 3600
removed = 0
for path in IMAGES_ROOT.rglob("*"):
if path.suffix not in TEMP_STAGING_SUFFIXES or not path.is_file():
continue
try:
if path.stat().st_mtime >= cutoff:
continue # too fresh — may be an active download
path.unlink()
removed += 1
except OSError as exc:
log.warning("cleanup_orphaned_temp_files: %s: %s", path, exc)
if removed:
log.info(
"cleanup_orphaned_temp_files: removed %d orphaned staging file(s)",
removed,
)
return removed
@celery.task(name="backend.app.tasks.maintenance.recover_stalled_task_runs")
def recover_stalled_task_runs() -> int:
"""Flip task_run rows stuck in 'running' past their queue-specific
+114 -35
View File
@@ -7,8 +7,10 @@ same pattern as the other backfill sweeps. No-op unless translation is enabled,
a base URL is set, and the service is healthy.
Two entry points:
- ``translate_posts`` — the daily untranslated sweep (translated_source_lang IS
NULL only); also the Settings "Translate now" button.
- ``translate_posts`` — the periodic untranslated sweep (every 8h;
translated_source_lang IS NULL only), one bounded chunk per fire. The Settings
"Translate now" button calls it with ``drain=True`` to chase the tail
run-until-done and clear the whole backlog in a single press.
- ``retranslate_posts`` — after a model change, resets the stored translation
columns for a scoped set of posts (all, or a given set of artists) so the
untranslated selection re-runs them, then chases the tail until drained
@@ -36,19 +38,39 @@ _MAX_POSTS_PER_RUN = 300
# Short gap between run-until-done chunks so a big re-translate finishes on its
# own without monopolising the maintenance_long worker in one uninterrupted run.
_RETRANSLATE_COUNTDOWN = 5
# When Interpreter drains mid-chunk (graceful restart), resume the bulk
# re-translate after the service's Retry-After hint if it sent one, else this
# default; capped so a bogus/huge hint can't park the work longer than the daily
# beat's own safety net would.
_INTERRUPT_BACKOFF = 60
_INTERRUPT_BACKOFF_MAX = 900
def _interrupt_backoff(retry_after) -> int:
"""Seconds to wait before resuming after an Interpreter interruption: the
server's Retry-After hint (capped), else the default."""
if retry_after and retry_after > 0:
return int(min(retry_after, _INTERRUPT_BACKOFF_MAX))
return _INTERRUPT_BACKOFF
@celery.task(
name="backend.app.tasks.translation.translate_posts",
soft_time_limit=1800, time_limit=2100,
)
def translate_posts() -> str:
def translate_posts(drain: bool = False) -> str:
"""Translate untranslated non-English post title/description (default target
en) via Interpreter. Per-post [title, description] batch → per-post detected
language. Passthrough / already-target posts are marked handled (source ==
target) with no stored translation. 503 or a connection error interrupts the
run (retry next cycle); 400 stops it (fix config); posts done so far stay
committed. No-op unless configured + healthy. Returns a summary string."""
committed. No-op unless configured + healthy. Returns a summary string.
``drain`` (the Settings "Translate now" button) chases the tail
run-until-done: on a clean chunk with work still remaining it re-enqueues
itself until the untranslated backlog is zero, like ``retranslate_posts``. The
periodic beat leaves it False → one bounded chunk per fire (the 8h cadence
drives the rest, enough for the trickle of newly-imported posts)."""
SessionLocal = _sync_session_factory()
with SessionLocal() as session:
ready = _translation_config(session)
@@ -56,7 +78,35 @@ def translate_posts() -> str:
return ready
base_url, target = ready
posts = _select_untranslated(session, None, _MAX_POSTS_PER_RUN)
status, translated = _translate_batch(session, posts, base_url, target)
status, translated, retry_after = _translate_batch(
session, posts, base_url, target,
)
# Drain mode (manual "Translate now"): chase the tail until the backlog is
# zero, so one press clears the whole pile instead of one 300-chunk.
# Termination is guaranteed — every handled post leaves the untranslated
# set, so the remaining count strictly decreases each chunk.
if status == "ok" and drain:
remaining = _count_untranslated(session, None)
if remaining:
translate_posts.apply_async(
(), {"drain": True}, countdown=_RETRANSLATE_COUNTDOWN,
)
log.info(
"translate_posts: draining — %d remaining → re-enqueued",
remaining,
)
# Interpreter drained mid-chunk (graceful restart): don't make a manual
# "Translate now" wait a whole beat cycle — re-enqueue after its
# Retry-After hint (or the default backoff), preserving `drain` so a bulk
# drain resumes cleanly. Self-terminating: once the service is down the
# health gate returns "interpreter unavailable" early.
elif status == "interrupted" and _count_untranslated(session, None):
delay = _interrupt_backoff(retry_after)
translate_posts.apply_async((), {"drain": drain}, countdown=delay)
log.info(
"translate_posts: interrupted (service draining) → retry in %ss",
delay,
)
return _summary(status, translated, len(posts))
@@ -88,11 +138,11 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
)
posts = _select_untranslated(session, artist_ids, _MAX_POSTS_PER_RUN)
status, translated = _translate_batch(session, posts, base_url, target)
status, translated, retry_after = _translate_batch(
session, posts, base_url, target,
)
# run-until-done: only chase the tail when the chunk finished cleanly. A
# service interruption / bad request stops here and leaves the rest NULL
# for the daily beat or a fresh trigger to resume (rule-89 recovery).
# run-until-done: chase the tail when the chunk finished cleanly.
# Termination is guaranteed: every handled post leaves the NULL set, so
# the remaining count strictly decreases each chunk.
if status == "ok":
@@ -106,6 +156,23 @@ def retranslate_posts(artist_ids=None, _reset_done=False) -> str:
log.info(
"retranslate_posts: %d remaining → re-enqueued", remaining,
)
# Interpreter drained mid-chunk (graceful restart / 429 / 5xx): don't
# stall the bulk re-translate until the daily beat — resume it after the
# service's Retry-After hint (or the default backoff), with
# _reset_done=True so the fresh work is never re-wiped. Self-terminating:
# if the service is still down next run, _translation_config's health gate
# returns "interpreter unavailable" early and nothing re-enqueues.
elif status == "interrupted" and _count_untranslated(session, artist_ids):
delay = _interrupt_backoff(retry_after)
retranslate_posts.apply_async(
(),
{"artist_ids": artist_ids, "_reset_done": True},
countdown=delay,
)
log.info(
"retranslate_posts: interrupted (service draining) → retry in %ss",
delay,
)
return _summary(status, translated, len(posts))
@@ -174,23 +241,25 @@ def _reset_translations(session, artist_ids) -> int:
def _translate_batch(session, posts, base_url: str, target: str):
"""Translate each post in the chunk with a per-post commit (an interrupted
run keeps progress). Returns (status, translated) where status is one of
"ok" / "interrupted" (503/conn) / "stopped" (400) / "timeout"."""
run keeps progress). Returns (status, translated, retry_after) where status is
one of "ok" / "interrupted" (unavailable/drain) / "stopped" (400) / "timeout";
retry_after is the server's Retry-After hint in seconds on an interrupt, else
None."""
translated = 0
for post in posts:
try:
translated += _translate_one(session, post, base_url, target)
session.commit()
except ic.InterpreterUnavailable:
except ic.InterpreterUnavailable as e:
session.rollback()
return "interrupted", translated
return "interrupted", translated, getattr(e, "retry_after", None)
except ic.InterpreterBadRequest as e:
session.rollback()
log.warning("translate bad request: %s", e)
return "stopped", translated
return "stopped", translated, None
except SoftTimeLimitExceeded:
return "timeout", translated
return "ok", translated
return "timeout", translated, None
return "ok", translated, None
def _summary(status: str, translated: int, scanned: int) -> str:
@@ -203,29 +272,39 @@ def _summary(status: str, translated: int, scanned: int) -> str:
return f"translated={translated} scanned={scanned}"
def _translate_field(text: str, base_url: str, target: str):
"""Translate ONE field independently. Returns (translated, source_lang,
engine_version), or (None, None, None) when there's nothing to store — empty
text, already the target language, or a passthrough (engine "none"). Per-field
(not aggregate first-item) detection so a non-English description still gets
translated when the title is already English (mixed-language posts)."""
if not text:
return None, None, None
res = ic.translate([text], base_url=base_url, target=target)
detected = res["detected_lang"] or target
if detected == target or res["engine"] == "none":
return None, None, None
return res["translations"][0], detected, res["engine_version"]
def _translate_one(session, post, base_url: str, target: str) -> int:
"""Translate one post's title/description in place. Returns 1 if it stored a
translation, 0 for passthrough/empty (which still marks the post handled)."""
"""Translate a post's title/description in place, each field independently.
Returns 1 if it stored at least one translation, 0 when the post is all
passthrough/empty (still marks it handled so the sweep won't revisit it)."""
title = (post.post_title or "").strip()
desc = (html_to_plain(post.description) if post.description else "") or ""
desc = desc.strip()
fields: list[tuple[str, str]] = []
if title:
fields.append(("title", title))
if desc:
fields.append(("desc", desc))
if not fields:
post.translated_source_lang = target # nothing to translate → handled
title_tr, title_lang, title_ev = _translate_field(title, base_url, target)
desc_tr, desc_lang, desc_ev = _translate_field(desc, base_url, target)
if title_tr is None and desc_tr is None:
# Nothing non-target (or nothing to translate) → handled, store nothing.
post.translated_source_lang = target
return 0
res = ic.translate([t for _, t in fields], base_url=base_url, target=target)
detected = res["detected_lang"] or target
if detected == target or res["engine"] == "none":
post.translated_source_lang = target # already target language → handled
return 0
by_field = {f: res["translations"][i] for i, (f, _) in enumerate(fields)}
post.post_title_translated = by_field.get("title")
post.description_translated = by_field.get("desc")
post.translated_source_lang = detected
post.translation_engine_version = res["engine_version"]
post.post_title_translated = title_tr
post.description_translated = desc_tr
# Source lang = whichever field was actually non-target (for a mixed post,
# the translated field's language, not the already-English one).
post.translated_source_lang = title_lang or desc_lang
post.translation_engine_version = title_ev or desc_ev
post.translated_at = datetime.now(UTC)
return 1
+65
View File
@@ -8,6 +8,35 @@
# run `docker compose up` from this directory and switches images to
# local builds + DEBUG logging.
# Rolling-deploy safety (Swarm / `docker stack deploy`): update one task at a
# time, START the new task before stopping the old (zero-downtime via the ingress
# mesh), and if the new task doesn't reach a healthy state within `monitor`, roll
# back to the previous image automatically. `monitor` is sized above web's
# healthcheck start_period so a broken image that never goes healthy is caught.
# Plain `docker compose up` ignores `deploy:` (it warns + skips), so the dev
# override is unaffected. Referenced by each long-lived service below.
x-deploy-policy: &deploy_policy
update_config:
order: start-first
failure_action: rollback
monitor: 90s
rollback_config:
order: start-first
restart_policy:
condition: any
delay: 10s
# Worker liveness: ping THIS container's celery node over the broker. Lenient
# (60s interval, 3 retries, 60s start_period) so a transient broker blip never
# false-flags a worker into a rollback. `$$HOSTNAME` → `$HOSTNAME` for the shell;
# celery's default node name is celery@<hostname> (the container id).
x-celery-healthcheck: &celery_healthcheck
test: ["CMD-SHELL", "celery -A backend.app.celery_app:celery inspect ping -d celery@$$HOSTNAME --timeout 10 >/dev/null 2>&1"]
interval: 60s
timeout: 15s
retries: 3
start_period: 60s
services:
redis:
image: redis:7-alpine
@@ -47,6 +76,24 @@ services:
web:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["web"]
# Graceful shutdown: give the container time to drain in-flight work on a
# deploy (docker SIGTERMs, then SIGKILLs after this window — default is only
# 10s, far too short for real jobs). Hypercorn/Celery both warm-shut-down on
# SIGTERM; per-lane values sized to typical task length. Anything that still
# outruns the window is re-queued (task_reject_on_worker_lost) and re-driven
# by the 5-min recovery sweeps, so a kill never corrupts. web = short HTTP
# requests + the occasional file download.
stop_grace_period: 30s
# Liveness for rolling deploys: /api/health is a no-DB 200 (just proves the
# app booted + serves HTTP after `alembic upgrade head`). start_period covers
# the migration + boot so a slow start isn't mis-flagged.
healthcheck:
test: ["CMD-SHELL", "python -c \"import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://localhost:8080/api/health', timeout=5).status==200 else 1)\""]
interval: 15s
timeout: 6s
retries: 3
start_period: 40s
deploy: *deploy_policy
ports:
- "${PORT:-8080}:8080"
environment: &app_env
@@ -77,6 +124,10 @@ services:
worker:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["worker"]
# Drain in-flight import/thumbnail/download tasks before SIGKILL on deploy.
stop_grace_period: 90s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment:
<<: *app_env
CELERY_QUEUES: default,import,thumbnail,download
@@ -93,6 +144,10 @@ services:
scheduler:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["scheduler"]
# Quick maintenance/scan lane + beat — short tasks, modest drain window.
stop_grace_period: 60s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment:
<<: *app_env
CELERY_QUEUES: maintenance,scan
@@ -110,6 +165,12 @@ services:
maintenance-long:
image: git.fabledsword.com/bvandeusen/fabledcurator:dev
command: ["worker"]
# Longest lane (DB backups, library audits, translation backfill) — give it
# the most room to finish a chunk gracefully. Chunked + idempotent, so a job
# that still outruns this resumes cleanly next run rather than corrupting.
stop_grace_period: 180s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment:
<<: *app_env
CELERY_QUEUES: maintenance_long
@@ -125,6 +186,10 @@ services:
ml-worker:
image: git.fabledsword.com/bvandeusen/fabledcurator-ml:dev
command: ["ml-worker"]
# A single GPU inference pass can run tens of seconds — let it finish.
stop_grace_period: 120s
healthcheck: *celery_healthcheck
deploy: *deploy_policy
environment:
<<: *app_env
volumes:
+31 -15
View File
@@ -1,7 +1,7 @@
<template>
<span class="fc-tag-chip" @mouseenter="onEnter" @mouseleave="onLeave">
<v-chip
size="small" :closable="!unconfirmedAuto"
size="default" :closable="!unconfirmedAuto"
:color="store.colorFor(tag.kind)" variant="tonal"
class="fc-tag-chip__nav"
role="link"
@@ -9,7 +9,7 @@
@click="$emit('navigate', tag)"
@click:close="$emit('remove', tag.id)"
>
<v-icon start size="x-small">{{ iconFor(tag.kind) }}</v-icon>
<v-icon start size="small">{{ iconFor(tag.kind) }}</v-icon>
<span class="fc-tag-chip__name">{{ tag.name }}</span><v-icon
v-if="tag.is_system" end size="x-small" class="fc-tag-chip__system"
title="System tag — tagged items are excluded from training other concepts"
@@ -27,13 +27,13 @@
:title="`Yes — keep “${tag.name}” (trains the model, won't be retracted)`"
:aria-label="`Confirm ${tag.name}`"
@click.stop="$emit('confirm', tag)"
><v-icon size="13">mdi-check</v-icon></button>
><v-icon size="15">mdi-check</v-icon></button>
<button
type="button" class="fc-tag-chip__no"
:title="`No — remove “${tag.name}”`"
:aria-label="`Reject ${tag.name}`"
@click.stop="$emit('remove', tag.id)"
><v-icon size="13">mdi-close</v-icon></button>
><v-icon size="15">mdi-close</v-icon></button>
</span>
</v-chip>
<!-- Modal-safe kebab is baked into KebabMenu (this chip lives in the
@@ -143,7 +143,19 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
chip's hover title. */
.fc-tag-chip { display: inline-flex; align-items: center; gap: 1px; max-width: 100%; min-width: 0; }
.fc-tag-chip__nav { max-width: 100%; min-width: 0; }
/* Tonal chips (esp. character = info) wash out against the dark rail — the fill
is intentionally faint. Give every tag chip a thin border in its OWN kind
colour (currentColor = the tonal chip's themed foreground) so the edge reads
clearly without touching the fill (operator-asked 2026-07-08). Theme-aware:
currentColor tracks the kind colour in either light or dark. */
.fc-tag-chip__nav { border: thin solid color-mix(in srgb, currentColor 55%, transparent); }
.fc-tag-chip__nav :deep(.v-chip__content) { min-width: 0; overflow: hidden; }
/* The bigger size=default chip widens Vuetify's negative start-margin on the
leading kind-icon, pulling it LEFT of the content box above — where that
overflow:hidden then clips the icon's left edge as a vertical slice
(operator-flagged 2026-07-08). Zero the pull so the icon sits inside the clip
box; the chip's own 12px padding keeps a comfortable left inset. */
.fc-tag-chip__nav :deep(.v-icon--start) { margin-inline-start: 0; }
.fc-tag-chip__name {
overflow: hidden; text-overflow: ellipsis; white-space: nowrap;
min-width: 0; flex: 0 1 auto;
@@ -159,23 +171,27 @@ function iconFor (k) { return KIND_ICONS[k] || 'mdi-tag' }
.fc-tag-chip__kebab { opacity: 0.7; }
.fc-tag-chip:hover .fc-tag-chip__kebab { opacity: 1; }
.fc-tag-chip__fandom { opacity: 0.7; font-size: 0.85em; }
/* Provisional auto-tag: a compact green ✓ / red ✗ pair in place of the ✕. The
yes/no is obvious enough on its own, so there's no "auto" label (operator-asked
/* Provisional auto-tag: a green ✓ / red ✗ pair in place of the ✕. The yes/no is
obvious enough on its own, so there's no "auto" label (operator-asked
2026-07-07). flex:0 0 auto keeps it visible while the name ellipsis-truncates. */
.fc-tag-chip__verdict {
flex: 0 0 auto; display: inline-flex; align-items: center; gap: 1px;
margin-left: 3px;
flex: 0 0 auto; display: inline-flex; align-items: center; gap: 3px;
margin-left: 5px;
}
/* Solid-filled (white glyph on a full success/error circle) so accept/reject
stay well-defined even on a muted tonal chip — e.g. character (info) — instead
of a faint icon that only lit up on hover. Mirrors the Suggestions rail's
verdict buttons so the affordance reads identically (operator-asked 2026-07-08). */
.fc-tag-chip__yes, .fc-tag-chip__no {
display: inline-flex; align-items: center; justify-content: center;
width: 18px; height: 18px; padding: 0; border: none; border-radius: 50%;
background: transparent; cursor: pointer; transition: background 0.1s;
width: 22px; height: 22px; padding: 0; border: none; border-radius: 50%;
color: #fff; cursor: pointer; opacity: 0.92;
transition: transform 0.1s, opacity 0.1s;
}
.fc-tag-chip__yes { color: rgb(var(--v-theme-success)); }
.fc-tag-chip__no { color: rgb(var(--v-theme-error)); }
.fc-tag-chip__yes:hover { background: rgb(var(--v-theme-success), 0.16); }
.fc-tag-chip__no:hover { background: rgb(var(--v-theme-error), 0.16); }
.fc-tag-chip__yes { background: rgb(var(--v-theme-success)); }
.fc-tag-chip__no { background: rgb(var(--v-theme-error)); }
.fc-tag-chip__yes:hover, .fc-tag-chip__no:hover { opacity: 1; transform: scale(1.1); }
.fc-tag-chip__yes:focus-visible, .fc-tag-chip__no:focus-visible {
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 1px;
outline: 2px solid rgb(var(--v-theme-accent)); outline-offset: 2px;
}
</style>
@@ -52,11 +52,25 @@
prepend-icon="mdi-refresh" :loading="retranslating"
:disabled="!enabled || !baseUrl" @click="confirmAll = true"
>Re-translate all</v-btn>
<span v-if="status" class="fc-muted text-caption">
<span
v-if="status && status.active"
class="fc-muted text-caption d-inline-flex align-center" style="gap: 6px;"
>
<v-progress-circular indeterminate size="12" width="2" color="accent" />
Translating… {{ status.untranslated_count }} remaining
</span>
<span v-else-if="status" class="fc-muted text-caption">
{{ status.untranslated_count }}
post{{ status.untranslated_count === 1 ? '' : 's' }} awaiting translation
</span>
</div>
<p
v-if="status && status.last_run && ['error', 'timeout'].includes(status.last_run.status)"
class="text-caption text-error mt-1 mb-0"
>
Last translation run ended with “{{ status.last_run.status }}”. It resumes on
the next sweep; check the Interpreter connection if it persists.
</p>
<p class="fc-muted text-caption mt-2 mb-0">
Use <strong>Re-translate all</strong> after switching the Interpreter model —
it clears every stored translation and re-runs it through the new model
@@ -64,6 +78,61 @@
artist, use the Re-translate button on that artist's page.
</p>
<v-divider class="my-3" />
<!-- Diagnostic probe (task #1376): paste text → what Interpreter DETECTS
(language + confidence) and returns, without saving. Used to see why a
short/abbreviation-heavy English title gets mis-detected, and to pick a
detection-confidence / min-length guard from real numbers. -->
<div class="mb-1">
<span class="fc-section-h">Test translation</span>
<p class="fc-muted text-caption mt-1 mb-2">
Paste a title or snippet to see what Interpreter detects and returns —
handy for checking why a short string is mis-detected. Nothing is saved.
</p>
</div>
<v-textarea
v-model="probeText" label="Text to test" rows="2" auto-grow
density="compact" hide-details class="mb-2" :disabled="!baseUrl"
/>
<div class="d-flex align-center flex-wrap mb-2" style="gap: 8px;">
<v-btn
size="small" variant="tonal" rounded="pill" prepend-icon="mdi-magnify"
:loading="probing" :disabled="!baseUrl || !probeText.trim()"
@click="onProbe"
>Test translation</v-btn>
</div>
<v-sheet
v-if="probeResult" rounded class="pa-3 mb-1 text-body-2"
color="rgba(var(--v-theme-accent), 0.06)"
>
<div class="d-flex flex-wrap" style="gap: 4px 16px;">
<span>
<span class="fc-muted">Detected:</span>
<strong>{{ probeResult.detected_lang || '' }}</strong>
</span>
<span v-if="probeResult.detected_confidence != null">
<span class="fc-muted">Confidence:</span>
<strong>{{ fmtConf(probeResult.detected_confidence) }}</strong>
</span>
<span>
<span class="fc-muted">Engine:</span>
{{ probeResult.engine || '' }}<template
v-if="probeResult.engine_version"
> ({{ probeResult.engine_version }})</template>
</span>
<span><span class="fc-muted">Target:</span> {{ probeResult.target }}</span>
</div>
<v-divider class="my-2" />
<div class="fc-muted mb-1">
Result<template v-if="probeResult.detected_lang === probeResult.target">
— already {{ probeResult.target }}, so the sweep leaves it as-is</template>:
</div>
<p class="mb-0" style="white-space: pre-wrap;">{{
probeResult.translated || '(no change)'
}}</p>
</v-sheet>
<v-alert
v-if="err" type="error" variant="tonal" density="compact"
class="mt-3" closable @click:close="err = null"
@@ -94,7 +163,7 @@
</template>
<script setup>
import { computed, onMounted, ref } from 'vue'
import { computed, onMounted, onUnmounted, ref } from 'vue'
import { toast } from '../../utils/toast.js'
import { useApi } from '../../composables/useApi.js'
import { useImportStore } from '../../stores/import.js'
@@ -115,6 +184,17 @@ const testResult = ref(null) // null = not tested this session; true/false = l
const status = ref(null)
const err = ref(null)
// #1376 "Test translation" probe: paste text → detected lang/confidence + result.
const probeText = ref('')
const probing = ref(false)
const probeResult = ref(null)
// Confidence scale is Interpreter-defined (0100 or 01) — show it as-is, just
// trimmed to 2 decimals, so the operator reads the true number.
function fmtConf (c) {
return c == null ? '' : Math.round(c * 100) / 100
}
const statusColor = computed(() => {
if (!enabled.value || !baseUrl.value) return 'grey'
return status.value?.healthy ? 'success' : 'error'
@@ -126,10 +206,16 @@ const statusText = computed(() => {
return status.value.healthy ? 'Interpreter reachable' : 'Interpreter unreachable'
})
let pollTimer = null
async function loadStatus() {
try { status.value = await api.get('/api/settings/translation/status') }
catch { status.value = null }
// Poll live while a sweep is running so the remaining count ticks down; stop
// when idle (a fresh trigger restarts it via its own setTimeout(loadStatus)).
clearTimeout(pollTimer)
if (status.value?.active) pollTimer = setTimeout(loadStatus, 3000)
}
onUnmounted(() => clearTimeout(pollTimer))
async function onTest() {
// Ping /v1/health for the CURRENTLY-typed URL (not the saved one), so a new
@@ -150,6 +236,21 @@ async function onTest() {
}
}
async function onProbe () {
probing.value = true
err.value = null
probeResult.value = null
try {
probeResult.value = await api.post('/api/settings/translation/probe', {
body: { text: probeText.value.trim() },
})
} catch (e) {
err.value = e.message
} finally {
probing.value = false
}
}
onMounted(async () => {
try {
await store.loadSettings()
+3 -1
View File
@@ -44,7 +44,9 @@ export const useExploreStore = defineStore('explore', () => {
// empty and let the view explain why (anchor.has_embedding === false).
if (detail.has_embedding) {
const body = await api.get('/api/gallery/similar', {
params: { similar_to: numId, limit: NEIGHBOR_LIMIT },
// exclude_wip: keep work-in-progress out of the Explore rabbit-hole
// (the gallery's own "similar" button still shows it) — operator 2026-07-08.
params: { similar_to: numId, limit: NEIGHBOR_LIMIT, exclude_wip: 1 },
})
if (!t.isCurrent()) return
neighbors.value = body.images || []
+70 -1
View File
@@ -61,6 +61,34 @@ async def test_translation_status_defaults(client):
assert body["base_url_set"] is False
assert body["healthy"] is False
assert "untranslated_count" in body
assert body["active"] is False # no sweep running
assert body["last_run"] is None
@pytest.mark.asyncio
async def test_translation_status_reports_active_and_last_run(client, db):
# A running translate/retranslate TaskRun → active True; the most recent
# finished one → last_run (task basename + status).
from datetime import UTC, datetime
from backend.app.models import TaskRun
db.add(TaskRun(
celery_task_id="r-run", queue="maintenance_long",
task_name="backend.app.tasks.translation.retranslate_posts",
started_at=datetime.now(UTC), status="running",
))
db.add(TaskRun(
celery_task_id="r-done", queue="maintenance_long",
task_name="backend.app.tasks.translation.translate_posts",
started_at=datetime.now(UTC), finished_at=datetime.now(UTC),
status="success",
))
await db.commit()
body = await (await client.get("/api/settings/translation/status")).get_json()
assert body["active"] is True
assert body["last_run"]["task"] == "translate_posts"
assert body["last_run"]["status"] == "success"
@pytest.mark.asyncio
@@ -85,9 +113,10 @@ async def test_translation_test_connection(client, monkeypatch):
@pytest.mark.asyncio
async def test_translation_run_enqueues_when_configured(client, monkeypatch):
sink = {}
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.delay",
lambda *a, **k: type("R", (), {"id": "t1"})(),
lambda *a, **k: sink.update(k) or type("R", (), {"id": "t1"})(),
)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
@@ -95,6 +124,46 @@ async def test_translation_run_enqueues_when_configured(client, monkeypatch):
resp = await client.post("/api/settings/translation/run")
assert resp.status_code == 202
assert (await resp.get_json())["celery_task_id"] == "t1"
assert sink["drain"] is True # "Translate now" chases the whole backlog
@pytest.mark.asyncio
async def test_translation_probe_requires_text(client):
resp = await client.post(
"/api/settings/translation/probe", json={"text": " "})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_requires_base_url(client):
# Text given but no Interpreter URL saved → nothing to probe against.
resp = await client.post(
"/api/settings/translation/probe", json={"text": "hello"})
assert resp.status_code == 400
@pytest.mark.asyncio
async def test_translation_probe_returns_detection(client, monkeypatch):
# The diagnostic surfaces detected language + confidence + result, unsaved.
monkeypatch.setattr(
"backend.app.api.settings.ic.translate",
lambda texts, **k: {
"translations": ["Work in Progress"],
"detected_lang": "de", "detected_confidence": 71.5,
"engine": "llm", "engine_version": "v1",
},
)
await client.patch("/api/settings/import", json={
"translation_enabled": True, "interpreter_base_url": "http://i.lan",
})
resp = await client.post(
"/api/settings/translation/probe", json={"text": "WIP"})
assert resp.status_code == 200
body = await resp.get_json()
assert body["detected_lang"] == "de"
assert body["detected_confidence"] == 71.5
assert body["translated"] == "Work in Progress"
assert body["target"] == "en"
def _fake_retranslate(monkeypatch, sink):
+24
View File
@@ -98,6 +98,30 @@ async def test_similar_excludes_presentation_tagged_images(db):
assert {i.id for i in res_from_banner} == {src.id, wipped.id, plain.id}
@pytest.mark.asyncio
async def test_similar_exclude_wip_drops_wip_neighbors(db):
"""Explore passes exclude_wip=True to also hide work-in-progress from the
rabbit-hole (banner is always hidden; wip only when asked)."""
src = await _img(db, 1, _vec(1, 0))
bannered = await _img(db, 2, _vec(1, 0.02)) # always hidden
wipped = await _img(db, 3, _vec(1, 0.3)) # hidden only with exclude_wip
plain = await _img(db, 4, _vec(1, 0.6))
banner_tag = (await db.execute(select(Tag).where(
Tag.is_system.is_(True), Tag.name == "banner"))).scalar_one()
wip_tag = (await db.execute(select(Tag).where(
Tag.is_system.is_(True), Tag.name == "wip"))).scalar_one()
await db.execute(image_tag.insert().values(
image_record_id=bannered.id, tag_id=banner_tag.id, source="manual"))
await db.execute(image_tag.insert().values(
image_record_id=wipped.id, tag_id=wip_tag.id, source="manual"))
svc = GalleryService(db)
res = await svc.similar(src.id, limit=10, exclude_wip=True)
assert [i.id for i in res] == [plain.id] # wip + banner both gone
# Default (gallery "similar" button) still keeps wip (#1274).
res_default = await svc.similar(src.id, limit=10)
assert wipped.id in {i.id for i in res_default}
@pytest.mark.asyncio
async def test_similar_composes_with_tag_filter(db):
src = await _img(db, 1, _vec(1, 0))
+44 -10
View File
@@ -6,9 +6,10 @@ from backend.app.services import interpreter_client as ic
class _Resp:
def __init__(self, status_code=200, payload=None):
def __init__(self, status_code=200, payload=None, headers=None):
self.status_code = status_code
self._payload = payload or {}
self.headers = headers or {}
def json(self):
return self._payload
@@ -29,10 +30,11 @@ def test_translate_maps_batch_in_order(monkeypatch):
"interpreter": {"engine": "llm", "engine_version": "ollama:x:12b"},
})
monkeypatch.setattr(ic.requests, "post", fake_post)
monkeypatch.setattr(ic.session, "post", fake_post)
out = ic.translate(["ねこが可愛い", "金髪ギャル!"], base_url="http://i.lan")
assert out["translations"] == ["The cat is cute", "Blonde gal!"]
assert out["detected_lang"] == "ja"
assert out["detected_confidence"] == 0.98 # surfaced for the detection guard
assert out["engine_version"] == "ollama:x:12b"
assert captured["json"]["q"] == ["ねこが可愛い", "金髪ギャル!"]
assert captured["json"]["target"] == "en"
@@ -40,7 +42,7 @@ def test_translate_maps_batch_in_order(monkeypatch):
def test_translate_passthrough_unchanged(monkeypatch):
# Already-English items come back unchanged in their slot (engine "none").
monkeypatch.setattr(ic.requests, "post", lambda *a, **k: _Resp(200, {
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(200, {
"translatedText": ["already english"],
"detectedLanguage": {"language": "en"},
"interpreter": {"engine": "none", "engine_version": None},
@@ -52,7 +54,7 @@ def test_translate_passthrough_unchanged(monkeypatch):
def test_translate_503_raises_unavailable(monkeypatch):
monkeypatch.setattr(ic.requests, "post", lambda *a, **k: _Resp(503, {}))
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(503, {}))
with pytest.raises(ic.InterpreterUnavailable):
ic.translate(["x"], base_url="http://i.lan")
@@ -61,14 +63,46 @@ def test_translate_connection_error_raises_unavailable(monkeypatch):
def boom(*a, **k):
raise ic.requests.ConnectionError("refused")
monkeypatch.setattr(ic.requests, "post", boom)
monkeypatch.setattr(ic.session, "post", boom)
with pytest.raises(ic.InterpreterUnavailable):
ic.translate(["x"], base_url="http://i.lan")
@pytest.mark.parametrize("code", [429, 500, 502, 503, 504])
def test_translate_429_and_5xx_raise_unavailable(monkeypatch, code):
# A gracefully-draining service behind a reverse proxy returns 429/502/503/504
# — all mean "retry later", not an opaque error.
monkeypatch.setattr(ic.session, "post", lambda *a, **k: _Resp(code, {}))
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after is None
def test_translate_honours_retry_after_seconds(monkeypatch):
monkeypatch.setattr(
ic.session, "post",
lambda *a, **k: _Resp(503, {}, headers={"Retry-After": "42"}),
)
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after == 42
def test_translate_retry_after_past_http_date_clamps_to_zero(monkeypatch):
# HTTP-date form; a past date → non-negative clamp to 0 (deterministic).
monkeypatch.setattr(
ic.session, "post",
lambda *a, **k: _Resp(
503, {}, headers={"Retry-After": "Wed, 21 Oct 2015 07:28:00 GMT"}),
)
with pytest.raises(ic.InterpreterUnavailable) as ei:
ic.translate(["x"], base_url="http://i.lan")
assert ei.value.retry_after == 0.0
def test_translate_400_raises_bad_request(monkeypatch):
monkeypatch.setattr(
ic.requests, "post", lambda *a, **k: _Resp(400, {"error": "bad target"})
ic.session, "post", lambda *a, **k: _Resp(400, {"error": "bad target"})
)
with pytest.raises(ic.InterpreterBadRequest):
ic.translate(["x"], base_url="http://i.lan")
@@ -78,20 +112,20 @@ def test_translate_empty_is_noop(monkeypatch):
def boom(*a, **k):
raise AssertionError("should not call the service for an empty batch")
monkeypatch.setattr(ic.requests, "post", boom)
monkeypatch.setattr(ic.session, "post", boom)
assert ic.translate([], base_url="http://i.lan")["translations"] == []
def test_health_true_when_llm_up(monkeypatch):
monkeypatch.setattr(
ic.requests, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": True}})
ic.session, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": True}})
)
assert ic.health("http://i.lan") is True
def test_health_false_when_engine_down(monkeypatch):
monkeypatch.setattr(
ic.requests, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": False}})
ic.session, "get", lambda *a, **k: _Resp(200, {"engines": {"llm": False}})
)
assert ic.health("http://i.lan") is False
@@ -100,7 +134,7 @@ def test_health_false_on_error(monkeypatch):
def boom(*a, **k):
raise ic.requests.ConnectionError("refused")
monkeypatch.setattr(ic.requests, "get", boom)
monkeypatch.setattr(ic.session, "get", boom)
assert ic.health("http://i.lan") is False
+22
View File
@@ -26,6 +26,28 @@ def _make_batch(session) -> int:
return batch.id
def test_cleanup_orphaned_temp_files_removes_stale_only(tmp_path, monkeypatch):
import os
from backend.app.tasks import maintenance as m
monkeypatch.setattr(m, "IMAGES_ROOT", tmp_path)
stale = tmp_path / "artist" / "img.jpg.part" # killed download → orphan
stale.parent.mkdir(parents=True)
stale.write_bytes(b"x")
old = datetime.now(UTC).timestamp() - 8 * 3600 # older than the 6h guard
os.utime(stale, (old, old))
fresh = tmp_path / "in_progress.jpg.partial" # active download → keep
fresh.write_bytes(b"x")
keep = tmp_path / "real.jpg" # a real image → keep
keep.write_bytes(b"x")
assert m.cleanup_orphaned_temp_files() == 1
assert not stale.exists()
assert fresh.exists()
assert keep.exists()
def test_recover_interrupted_only_old(db_sync, monkeypatch):
batch_id = _make_batch(db_sync)
now = datetime.now(UTC)
+175 -1
View File
@@ -4,7 +4,12 @@ import pytest
from sqlalchemy import select
from backend.app.models import Artist, ImportSettings, Post
from backend.app.tasks.translation import retranslate_posts, translate_posts
from backend.app.services import interpreter_client as ic
from backend.app.tasks.translation import (
_RETRANSLATE_COUNTDOWN,
retranslate_posts,
translate_posts,
)
pytestmark = pytest.mark.integration
@@ -114,6 +119,35 @@ def test_translate_posts_passthrough_english_marks_handled(db_sync, monkeypatch)
assert p.post_title_translated is None # ...but nothing to show
def test_translate_posts_mixed_language_translates_nonenglish_field(db_sync, monkeypatch):
# English title + Japanese description: the description is still translated
# even though the title is already English (per-field detection, not the old
# aggregate first-item bail). Source lang comes from the translated field.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def fake_translate(texts, **k):
t = texts[0]
if t.isascii(): # already English → passthrough
return {"translations": [t], "detected_lang": "en",
"engine": "none", "engine_version": None}
return {"translations": [f"EN:{t}"], "detected_lang": "ja",
"engine": "llm", "engine_version": "v9"}
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", fake_translate)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="hello", desc="ねこ")
_enable(db_sync)
db_sync.commit()
assert "translated=1" in translate_posts()
db_sync.refresh(p)
assert p.post_title_translated is None # English title untouched
assert p.description_translated == "EN:ねこ" # Japanese desc translated
assert p.translated_source_lang == "ja" # from the translated field
assert p.translation_engine_version == "v9"
def test_translate_posts_disabled_is_noop(db_sync, monkeypatch):
_patch(monkeypatch, db_sync)
a = _artist(db_sync)
@@ -136,6 +170,88 @@ def test_translate_posts_service_down_leaves_untranslated(db_sync, monkeypatch):
assert p.translated_source_lang is None # will retry next run
def test_translate_posts_interrupt_reenqueues_after_backoff(db_sync, monkeypatch):
# A drain mid-sweep (health passed, translate 503s w/ Retry-After) re-enqueues
# the daily sweep after the backoff instead of waiting for tomorrow's beat.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining", retry_after=30)
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_enable(db_sync)
db_sync.commit()
assert "interrupted" in translate_posts()
assert len(calls) == 1
assert calls[0][1]["countdown"] == 30
def _mock_ok(monkeypatch):
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
monkeypatch.setattr(
"backend.app.tasks.translation.ic.translate",
lambda texts, **k: {
"translations": [f"EN:{t}" for t in texts],
"detected_lang": "ja", "engine": "llm", "engine_version": "v1",
},
)
def test_translate_posts_drain_chases_tail(db_sync, monkeypatch):
# "Translate now" (drain=True): with the chunk capped at 1 and 2 untranslated
# posts, the first chunk re-enqueues itself — drain preserved — to finish the
# backlog rather than waiting for the next beat.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts(drain=True)
assert len(calls) == 1 # one re-enqueue for the tail
args, kw = calls[0]
assert args[1] == {"drain": True} # drain flag carried forward
assert kw["countdown"] == _RETRANSLATE_COUNTDOWN
def test_translate_posts_beat_single_chunk_no_tail_chase(db_sync, monkeypatch):
# The periodic beat (drain defaults False) does exactly ONE chunk and never
# re-enqueues — the 8h cadence drives the rest.
_patch(monkeypatch, db_sync)
_mock_ok(monkeypatch)
monkeypatch.setattr("backend.app.tasks.translation._MAX_POSTS_PER_RUN", 1)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.translate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
_post(db_sync, a.id, title="ねこ", ext="p1")
_post(db_sync, a.id, title="いぬ", ext="p2")
_enable(db_sync)
db_sync.commit()
translate_posts() # drain=False
assert calls == [] # no self-re-enqueue
def _mock_new_model(monkeypatch, *, ver="v2"):
"""Interpreter is up and translates with a NEW engine version."""
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
@@ -239,3 +355,61 @@ def test_retranslate_unhealthy_does_not_reset(db_sync, monkeypatch):
db_sync.refresh(p)
assert p.translated_source_lang == "ja"
assert p.post_title_translated == "OLD"
def test_retranslate_interrupt_resumes_after_retry_after(db_sync, monkeypatch):
# Interpreter drains mid-chunk (health passed, then translate 503s with a
# Retry-After): the bulk re-translate re-enqueues itself after the hinted
# backoff, with _reset_done=True so it never re-wipes fresh work — instead of
# stalling until the daily beat.
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining", retry_after=42)
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
assert "interrupted" in retranslate_posts(artist_ids=[a.id])
assert len(calls) == 1 # resume re-enqueued
args, kw = calls[0]
assert args[1]["_reset_done"] is True
assert args[1]["artist_ids"] == [a.id]
assert kw["countdown"] == 42 # honoured the Retry-After
db_sync.refresh(p)
assert p.translated_source_lang is None # reset happened, not re-run
def test_retranslate_interrupt_default_backoff_without_hint(db_sync, monkeypatch):
# No Retry-After header → fall back to the default backoff (60s).
_patch(monkeypatch, db_sync)
monkeypatch.setattr("backend.app.tasks.translation.ic.health", lambda *a, **k: True)
def _drain(texts, **k):
raise ic.InterpreterUnavailable("draining")
monkeypatch.setattr("backend.app.tasks.translation.ic.translate", _drain)
calls = []
monkeypatch.setattr(
"backend.app.tasks.translation.retranslate_posts.apply_async",
lambda *a, **k: calls.append((a, k)),
)
a = _artist(db_sync)
p = _post(db_sync, a.id, title="ねこ", ext="p1")
_mark_translated(p)
_enable(db_sync)
db_sync.commit()
retranslate_posts(artist_ids=[a.id])
assert len(calls) == 1
assert calls[0][1]["countdown"] == 60