feat(tagging): title-based WIP auto-tagging (#1458)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Failing after 3m50s

Auto-apply the `wip` system tag to posts whose TITLE explicitly declares
work-in-progress ("WIP" / "work in progress") — a deterministic, high-precision
complement to the image-based ML `wip` head. WIP images are excluded from the
Explore/gallery browse, so honouring the artist's own label keeps unfinished
pieces out of the main browse.

- services/wip_title.py: precision-first token-anchored matcher (swipe/wiped
  never trip it) + sync apply helpers (source='wip_title', ON CONFLICT DO
  NOTHING, chunked under the psycopg param ceiling).
- importer: live hook on FRESH import only (never on deep-scan/supersede), so a
  manually-removed WIP tag is never re-applied by a routine re-scan.
- maintenance.backfill_wip_title_tags: operator-triggered back-catalogue sweep
  (coarse SQL prefilter + regex confirm, keyset-paginated). Deliberately NOT a
  beat — a periodic re-run would silently undo manual removals.
- ImportSettings.wip_title_tagging_enabled (default ON, migration 0085) gating
  the live hook; GET/PATCH + POST /settings/wip-title/scan.
- Settings UI: toggle + "Scan existing posts" button.
- Tests: pure matcher unit tests + integration (apply idempotency, backfill
  precision).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-12 12:12:19 -04:00
parent 0cf3a02797
commit 571938781a
10 changed files with 483 additions and 3 deletions
+71
View File
@@ -76,6 +76,8 @@ DOWNLOAD_STALL_THRESHOLD_MINUTES = 30
OLD_TASK_DAYS = 7
PHASH_PAGE = 500
VERIFY_PAGE = 200
# Title-based WIP backfill (task #1458): posts scanned per keyset page.
WIP_BACKFILL_PAGE = 500
FFPROBE_TIMEOUT_SECONDS = 10
TASK_RUN_KEEP_OK_SECONDS = 24 * 3600 # 24 h
TASK_RUN_KEEP_FAILURE_SECONDS = 7 * 24 * 3600 # 7 days
@@ -1045,6 +1047,75 @@ def cleanup_old_download_events() -> int:
return result.rowcount or 0
@celery.task(
name="backend.app.tasks.maintenance.backfill_wip_title_tags",
# Coarse-prefiltered scan over posts; the candidate set is small on a typical
# library, but bound it like the other full-library sweeps.
soft_time_limit=1800, time_limit=2100,
)
def backfill_wip_title_tags() -> int:
"""Scan EXISTING posts for explicit WIP titles and apply the `wip` system tag
to their images — the operator-triggered back-catalogue catch-up for
title-based WIP tagging (task #1458). New imports are tagged live by the
importer; this covers everything already in the library.
Keyset-paginated over posts (restart-safe). A coarse SQL prefilter narrows to
titles that COULD match; the precise regex (matches_wip_title) confirms.
Idempotent-additive (ON CONFLICT DO NOTHING) — never disturbs an existing tag.
Deliberately NOT scheduled as a beat: a periodic re-run would re-apply to
matching posts and silently undo a manual WIP removal, so it stays an explicit
operator action (Settings → "Scan existing posts for WIP titles"). Returns the
number of image-tag rows newly applied.
"""
from ..models import Post
from ..models.image_provenance import ImageProvenance
from ..services.wip_title import (
WIP_TITLE_SQL_PREFILTER,
apply_wip_image_tags,
matches_wip_title,
resolve_wip_tag_id,
)
SessionLocal = _sync_session_factory()
applied = 0
last_id = 0
with SessionLocal() as session:
tag_id = resolve_wip_tag_id(session)
if tag_id is None:
log.warning(
"backfill_wip_title_tags: no `wip` system tag present; nothing to do"
)
return 0
like_a, like_b = WIP_TITLE_SQL_PREFILTER
while True:
rows = session.execute(
select(Post.id, Post.post_title)
.where(Post.id > last_id)
.where(Post.post_title.is_not(None))
.where(or_(
Post.post_title.ilike(like_a),
Post.post_title.ilike(like_b),
))
.order_by(Post.id.asc())
.limit(WIP_BACKFILL_PAGE)
).all()
if not rows:
break
last_id = rows[-1][0]
match_ids = [pid for pid, title in rows if matches_wip_title(title)]
if match_ids:
image_ids = session.execute(
select(ImageProvenance.image_record_id)
.where(ImageProvenance.post_id.in_(match_ids))
).scalars().all()
applied += apply_wip_image_tags(session, image_ids, tag_id)
session.commit()
if applied:
log.info("backfill_wip_title_tags: applied wip to %d image(s)", applied)
return applied
@celery.task(name="backend.app.tasks.maintenance.vacuum_analyze")
def vacuum_analyze() -> dict:
"""Periodic VACUUM (ANALYZE) over the high-churn tables (VACUUM_TABLES) to