feat(wip): soft title tier — sketch/doodle vocab + ring-loud audit (#1474)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 40s
CI / integration (push) Successful in 3m53s

Extends WIP title-tagging to lower-precision cues (sketch/doodle/scribble) safely.

- wip_title.py: soft matcher (word-anchored; sketchbook/kadoodle don't trip it);
  WIP_TITLE_SOFT_SOURCE + soft SQL prefilter; apply_wip_image_tags takes a source arg.
- training_data._AUTO_SOURCES += 'wip_title_soft' → the soft tier is PROVISIONAL and
  never trains the wip head (a finished "sketch" can't pollute it). Only the hard
  tier (wip_title) + manual train.
- ImportSettings.wip_soft_title_tagging_enabled (OFF by default, opt-in). Migration 0087.
- importer: hard tier wins, soft is the fallback (source wip_title_soft).
- backfill: refactored into a shared _backfill_wip_tier; hard always, soft when enabled.
- heads.soft_wip_conflict_audit + daily beat: score soft-tagged images against content
  heads, flag ring-loud ones (PresentationReview mode=process) for the review strip —
  the operator's "measure if they got falsely tagged" safety.
- api settings toggle; ImportFiltersForm soft toggle.
- tests: soft matcher pos/neg; soft source not a training positive; audit flags
  ring-loud + spares quiet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-13 10:14:38 -04:00
parent d9a14e890d
commit af0d39ed52
14 changed files with 355 additions and 58 deletions
+31 -7
View File
@@ -27,7 +27,13 @@ from ..models.tag import WIP_SYSTEM_TAG, Tag, image_tag
# image_tag.source stamped on title-heuristic WIP tags — distinct from the other
# apply sources so provenance stays legible and a future undo can target only these.
# HARD tier ("WIP"/"work in progress") is high-precision → trains the wip head.
WIP_TITLE_SOURCE = "wip_title"
# SOFT tier (sketch/doodle/scribble, #1474) is LOWER-precision — a finished "sketch"
# is often not WIP. This source is PROVISIONAL (in training_data._AUTO_SOURCES) so it
# NEVER trains the wip head; a soft-tagged image that also looks like real content is
# surfaced by the ring-loud audit for review.
WIP_TITLE_SOFT_SOURCE = "wip_title_soft"
# A standalone "WIP" / "W.I.P" token, or the phrase "work in progress"
# (space/underscore/hyphen separated). The letter-boundary lookarounds are what
@@ -39,11 +45,20 @@ _WIP_RE = re.compile(
re.IGNORECASE,
)
# Coarse SQL prefilter for the backfill sweep — narrows the post scan to rows that
# Soft tier: sketch / doodle / scribble (+ plurals), letter-boundary anchored so
# "sketchbook" / "kadoodle" don't trip it. Deliberately conservative — recall is
# secondary because the soft source doesn't train the head and the ring-loud audit
# catches false positives.
_SOFT_WIP_RE = re.compile(
r"(?<![A-Za-z])(?:sketch|sketches|doodle|doodles|scribble|scribbles)(?![A-Za-z])",
re.IGNORECASE,
)
# Coarse SQL prefilters for the backfill sweep — narrow the post scan to rows that
# COULD match before the precise regex confirms. Case-insensitive ILIKE patterns.
# MUST stay a SUPERSET of _WIP_RE (every regex match contains "wip" or
# "work…progress") or the sweep would silently miss posts.
# Each MUST stay a SUPERSET of its regex or the sweep would silently miss posts.
WIP_TITLE_SQL_PREFILTER = ("%wip%", "%work%progress%")
SOFT_WIP_TITLE_SQL_PREFILTER = ("%sketch%", "%doodle%", "%scribble%")
# Chunk bulk inserts so a large sweep can't blow past psycopg's 65535-parameter
# ceiling (3 params/row → ~21k rows max; 5k stays comfortably under).
@@ -51,12 +66,19 @@ _INSERT_CHUNK = 5000
def matches_wip_title(title: str | None) -> bool:
"""True when a post title explicitly marks it work-in-progress."""
"""True when a post title explicitly marks it work-in-progress (HARD tier)."""
if not title:
return False
return _WIP_RE.search(title) is not None
def matches_soft_wip_title(title: str | None) -> bool:
"""True when a title carries a SOFT WIP cue (sketch/doodle/scribble, #1474)."""
if not title:
return False
return _SOFT_WIP_RE.search(title) is not None
def resolve_wip_tag_id(session: Session) -> int | None:
"""The seeded ``wip`` system tag's id (migration 0075), or None if absent."""
return session.execute(
@@ -64,8 +86,10 @@ def resolve_wip_tag_id(session: Session) -> int | None:
).scalar_one_or_none()
def apply_wip_image_tags(session: Session, image_ids, tag_id: int) -> int:
"""Attach ``tag_id`` (source='wip_title') to each image id, idempotently —
def apply_wip_image_tags(
session: Session, image_ids, tag_id: int, *, source: str = WIP_TITLE_SOURCE
) -> int:
"""Attach ``tag_id`` (stamped with ``source``) to each image id, idempotently —
never disturbs an existing tag or its source. Returns the number of image_tag
rows newly inserted. Does NOT commit.
@@ -92,7 +116,7 @@ def apply_wip_image_tags(session: Session, image_ids, tag_id: int) -> int:
session.execute(
pg_insert(image_tag)
.values([
{"image_record_id": iid, "tag_id": tag_id, "source": WIP_TITLE_SOURCE}
{"image_record_id": iid, "tag_id": tag_id, "source": source}
for iid in to_insert
])
.on_conflict_do_nothing(index_elements=["image_record_id", "tag_id"])