feat: the matcher gets an identity route, so a name can link what time cannot (4392)

#4392 measured three compounding reasons the announcement matcher could not
fire for this creator. This is the first two.

**Vocabulary.** `discord` is not how these creators write once the audience
already knows where the server is. Across 20,558 Patreon bodies `discord`
appears in 486 and `the server` in 37 — but the distribution is the point, not
the totals: for artist 8, 21 of 42 posts say `discord`, 7 say `the server`,
and it is the RECENT ones that say the latter. A vocabulary list written from
old posts silently stops matching the posts that still need it.

**Two routes.** Circumstance and identity answer different questions.
Proximity and a declaration say two things happened near each other; a working
name the creator uses on these two posts and nowhere else says they are the
same piece. Averaging those makes the threshold uninterpretable and costs
both ways — adding identity as a fourth weight dilutes the others enough that
measured teaser/drop pairs an hour apart stop proposing, while capping
identity at a weight means the strongest evidence available can never carry a
pair alone. So identity may OVERRIDE, never dilute; below IDENTITY_FLOOR it is
recorded for the operator and moves nothing, which is the conservative
direction, since a wrong link asserts two different pieces are one.

The circumstantial bundle is now proximity/declared/marker at .45/.35/.20.
Rebalanced rather than extended: the existing guard still holds arithmetically
(no single weight reaches the threshold) and the existing pairs still score
where they did — a teaser and its drop an hour apart with an invite link go
from 0.815 to 0.781, both well over.

Measured against the live instance, artist 8: 17 candidate pairs, 14 carried
by a shared name, no false positives, and no drop claimed by two posts. The
operator's own example — `Anya -- 🍈🍈` and the drop 75 minutes before it —
links on the token `0-k`, which is what its two files are called on either
platform. Two pairs 21h apart that proximity scores 0.10 are reached, and two
more (`680lc`, `cnni18x`, four posts each) are the stated cost of the floor.

Both rare-token signals are artist-scoped and loaded once per artist per
sweep, not per candidate pair.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR

backend/app/services/post_association_service.py

@@ -15,24 +15,41 @@ where they already were, a wrong one actively misinforms and then propagates
 into whatever reads the association. So the matcher's job is to make a SHORT
 list worth reading, not a long list worth trusting.
 
-## Signals, and the one deliberately NOT built
+## Two routes, because the evidence is of two different kinds
+
+CIRCUMSTANTIAL evidence says two things happened near each other. It is
+additive, weighted, and no single one of its signals may reach the threshold:
 
 1. **Time proximity.** The Patreon post exists in order to announce the drop,
    so the two are minutes-to-hours apart. Nearly free, and strong.
 2. **The post says so.** These announcements routinely name Discord or carry
    an invite link, which is close to a declaration.
+3. **A shared marker.** The creator's own tie-back — `🍈🍈` in the Patreon
+   title and `@everyone 🍈 🍈` in the Discord message — gated on how rare that
+   marker is in THIS artist's posts, because a habitual emoji is punctuation.
 
-3. **Crop-to-source matching is HELD, on the plan's own instruction** — it is
+IDENTITY evidence says two things are the same thing, and it gets its own
-   real work with real false-positive risk, and it is only worth building once
+route (see `IDENTITY_FLOOR`):
-   1 and 2 are shown to be insufficient against the operator's actual artists.
-   Nothing here should be read as evidence it is unnecessary; it is deferred,
-   and the thing that would justify it is an empty review queue on a pair the
-   operator can see with their own eyes.
 
-   Note also that a naive whole-image SigLIP similarity is NOT that signal. A
+4. **A shared working name.** The creator exports the teaser and the release
-   cropped teaser and its full version are exactly the pair a whole-image
+   from one file, and the internal name survives into both platforms
-   comparison handles worst, so adding one as a "bonus" would mostly add noise
+   untouched. Measured on the operator's artist: `ConnFront` ↔ `ConnFront`.
-   while looking like progress.
+   This is the only signal that reaches a pair 23.8 hours apart, which
+   proximity scores at 0.005.
+
+## The one deliberately NOT built
+
+**Crop-to-source matching is HELD, on the plan's own instruction** — it is
+real work with real false-positive risk, and it is only worth building once
+the cheap signals are shown to be insufficient against the operator's actual
+artists. Half of this creator's recent teasers are screenshots carrying no
+working name at all, and those pairs are out of reach here; that, measured, is
+what would justify it.
+
+Note also that a naive whole-image SigLIP similarity is NOT that signal. A
+cropped teaser and its full version are exactly the pair a whole-image
+comparison handles worst, so adding one as a "bonus" would mostly add noise
+while looking like progress.
 
 ## Creator identity comes free, so E4 is not actually a prerequisite
 
@@ -56,14 +73,24 @@ from __future__ import annotations
 
 import logging
 import re
+from collections import Counter
+from dataclasses import dataclass
 from datetime import UTC, datetime, timedelta
 
 from sqlalchemy import func, or_, select
 from sqlalchemy.ext.asyncio import AsyncSession
 
-from ..models import ImportSettings, Post, PostAssociation
+from ..models import ImageRecord, ImportSettings, Post, PostAssociation
 from ..utils.text import html_to_plain
 from .discord_grouping import DROP_GROUPER
+from .post_naming import (
+    IDENTITY_FLOOR,
+    marker_frequencies,
+    marker_overlap,
+    shared_identity,
+    token_frequencies,
+    working_name_tokens,
+)
 
 log = logging.getLogger(__name__)
 
@@ -76,12 +103,20 @@ log = logging.getLogger(__name__)
 # enough" arithmetic rather than aspirational: on a busy day an artist posts
 # several times, and a matcher that could pair on proximity alone would turn
 # every busy day into false pairs. A guard test pins this.
-WEIGHTS = {"proximity": 0.55, "declared": 0.45}
+WEIGHTS = {"proximity": 0.45, "declared": 0.35, "marker": 0.20}
 
 # A Discord INVITE in the body is close to a declaration; the bare word is
 # weaker but still meaningful, because these posts are short and on-topic.
+#
+# "the server" and its possessives are here because the word `discord` is NOT
+# how these creators actually write. Measured across 20,558 Patreon bodies:
+# `discord` appears in 486 and `the server` in 37 — but the distribution is the
+# point, not the totals. For the artist this step was built for, 21 of 42 posts
+# say `discord` and 7 say `the server`, and it is the RECENT ones that say the
+# latter: the phrasing drifted once the audience already knew where the server
+# was. A vocabulary list written from old posts silently stops matching.
 _INVITE = re.compile(r"discord\.(?:gg|com/invite)/", re.I)
-_MENTION = re.compile(r"\bdiscord\b", re.I)
+_MENTION = re.compile(r"\b(?:discord|(?:the|our|my)\s+server)\b", re.I)
 
 DECLARED_INVITE = 1.0
 DECLARED_MENTION = 0.6
@@ -89,6 +124,23 @@ DECLARED_MENTION = 0.6
 MAX_CANDIDATES = 25
 
 
+@dataclass(frozen=True)
+class _Corpus:
+    """One artist's rare-token evidence, gathered once rather than per pair.
+
+    Both rare-token signals are scoped to a single artist — a working name and
+    a marker belong to the person who chose them — so the counts are useless
+    across artists and expensive to rebuild per candidate. A sweep touches an
+    artist's posts many times over; this is loaded on the first touch and kept
+    for the life of the service.
+    """
+
+    tokens_by_post: dict[int, set[str]]
+    token_posts: Counter[str]
+    text_by_post: dict[int, str]
+    marker_posts: Counter[str]
+
+
 def proximity_signal(gap: timedelta, window: timedelta) -> float:
     """1.0 when the two posts are simultaneous, decaying linearly to 0 at the
     window's edge. Linear rather than a step, so a pair an hour outside a
@@ -134,6 +186,46 @@ def _post_time(post: Post) -> datetime:
 class PostAssociationService:
     def __init__(self, session: AsyncSession):
         self.session = session
+        self._corpora: dict[int, _Corpus] = {}
+
+    async def _corpus(self, artist_id: int) -> _Corpus:
+        if artist_id in self._corpora:
+            return self._corpora[artist_id]
+
+        paths_by_post: dict[int, list[str]] = {}
+        rows = await self.session.execute(
+            select(ImageRecord.primary_post_id, ImageRecord.path).where(
+                ImageRecord.artist_id == artist_id,
+                ImageRecord.primary_post_id.is_not(None),
+            )
+        )
+        for post_id, path in rows:
+            paths_by_post.setdefault(post_id, []).append(path)
+
+        text_by_post: dict[int, str] = {}
+        rows = await self.session.execute(
+            select(Post.id, Post.post_title, Post.description).where(
+                Post.artist_id == artist_id
+            )
+        )
+        for post_id, title, description in rows:
+            text_by_post[post_id] = "\n".join(
+                part for part in (title, html_to_plain(description) or "") if part
+            )
+
+        corpus = _Corpus(
+            tokens_by_post={
+                pid: {t for path in paths for t in working_name_tokens(path)}
+                for pid, paths in paths_by_post.items()
+            },
+            # Both counts take POSTS, which is why they are built from these
+            # groupings rather than from flat lists — see post_naming.
+            token_posts=token_frequencies(paths_by_post.values()),
+            text_by_post=text_by_post,
+            marker_posts=marker_frequencies(text_by_post.values()),
+        )
+        self._corpora[artist_id] = corpus
+        return corpus
 
     async def _decided(self, announcement_id: int) -> set[int]:
         """Payload posts already proposed for this announcement, in ANY status.
@@ -186,20 +278,57 @@ class PostAssociationService:
         window = timedelta(hours=window_hours)
         declared = declared_signal(announcement.description)
         already = await self._decided(announcement_id)
+        corpus = await self._corpus(announcement.artist_id)
+        here = corpus.tokens_by_post.get(announcement.id, set())
+        here_text = corpus.text_by_post.get(announcement.id, "")
 
         made = 0
         for group in await self._candidate_groups(announcement, window=window):
             if group.id in already:
                 continue
-            signals = {
+            identity, token = shared_identity(
+                here,
+                corpus.tokens_by_post.get(group.id, set()),
+                corpus.token_posts,
+            )
+            circumstantial = {
                 "proximity": proximity_signal(
                     _post_time(group) - _post_time(announcement), window,
                 ),
                 "declared": declared,
+                "marker": marker_overlap(
+                    here_text,
+                    corpus.text_by_post.get(group.id, ""),
+                    corpus.marker_posts,
+                ),
             }
-            score = weighted_score(signals)
+            score = weighted_score(circumstantial)
+            # THE TWO ROUTES, and why identity is not simply a fourth weight.
+            #
+            # Circumstance and identity answer different questions. Proximity
+            # and a declaration say two things happened near each other and
+            # that one of them mentioned Discord; a working name the creator
+            # uses on these two posts and nowhere else says they are the same
+            # piece. Averaging those makes the threshold uninterpretable, and
+            # it costs both: adding identity as a weight dilutes the others
+            # enough that measured teaser/drop pairs an hour apart stop
+            # proposing, while capping identity's contribution at its weight
+            # means the strongest evidence available can never carry a pair on
+            # its own.
+            #
+            # So identity may override, never dilute. Below the floor it is
+            # recorded for the operator to read and moves nothing — which is
+            # the conservative direction, since a wrong link asserts that two
+            # different pieces are one.
+            if identity >= IDENTITY_FLOOR:
+                score = max(score, identity)
             if score < threshold:
                 continue
+            signals = {**circumstantial, "identity": identity}
+            if token:
+                # Carried so the queue can say WHY. A review queue that cannot
+                # explain itself is one the operator learns to click through.
+                signals["identity_token"] = token
             self.session.add(PostAssociation(
                 announcement_post_id=announcement.id,
                 payload_post_id=group.id,

tests/test_post_association.py

@@ -22,6 +22,7 @@ from backend.app.models import (
 )
 from backend.app.services.discord_grouping import DROP_GROUPER
 from backend.app.services.post_association_service import (
+    DECLARED_MENTION,
     WEIGHTS,
     PostAssociationService,
     declared_signal,
@@ -130,30 +131,42 @@ async def _artist_with_channels(db, name: str):
     return artist, patreon, discord
 
 
-async def _teaser(db, artist, source, *, at, body, ext="teaser"):
+async def _images(db, artist, post, ext, names):
-    post = Post(
+    """Attach named files to a post. The NAME is the point — the working-name
-        source_id=source.id, artist_id=artist.id, external_post_id=ext,
+    signal reads it, so a test that cares about identity supplies one."""
-        post_date=at, post_title="New piece", description=body,
+    for i, name in enumerate(names):
-    )
-    db.add(post)
-    await db.flush()
         db.add(ImageRecord(
-        path=f"/images/{source.id}-{ext}.jpg", sha256=f"{ext:0>64}"[:64],
+            path=f"/images/{artist.id}/{ext}_{i}_{name}.jpg",
+            sha256=f"{ext}{i}{name}".ljust(64, "0")[:64],
             size_bytes=10, mime="image/jpeg", width=10, height=10,
             origin="downloaded", primary_post_id=post.id, artist_id=artist.id,
         ))
     await db.flush()
+
+
+async def _teaser(db, artist, source, *, at, body, ext="teaser", names=None,
+                  title="New piece"):
+    post = Post(
+        source_id=source.id, artist_id=artist.id, external_post_id=ext,
+        post_date=at, post_title=title, description=body,
+    )
+    db.add(post)
+    await db.flush()
+    await _images(db, artist, post, ext, names or [ext])
     return post
 
 
-async def _drop(db, artist, source, *, at, ext="fc-drop:1"):
+async def _drop(db, artist, source, *, at, ext="fc-drop:1", body=None,
+                names=()):
     post = Post(
         source_id=source.id, artist_id=artist.id, external_post_id=ext,
-        post_date=at, synthesized_by=DROP_GROUPER,
+        post_date=at, description=body, synthesized_by=DROP_GROUPER,
         synthesis_details={"message_count": 4, "images_since_surface": 0},
     )
     db.add(post)
     await db.flush()
+    if names:
+        await _images(db, artist, post, ext.replace(":", "-"), names)
     return post
 
 
@@ -372,3 +385,217 @@ async def test_the_rescan_is_a_no_op_when_the_switch_is_off(db):
     settings.discord_link_enabled = False
     await db.commit()
     assert await rescan(db) == {"enabled": False, "scanned": 0, "proposed": 0}
+
+
+# --- the identity route ----------------------------------------------------
+#
+# Milestone 388. Everything below was calibrated against the operator's real
+# library (artist 8: 520 images, 15 same-artist pairs that share a name, no
+# false positives) rather than invented, so the numbers in these docstrings are
+# measurements.
+
+
+@pytest.mark.asyncio
+async def test_a_shared_working_name_proposes_a_pair_time_cannot_reach(db):
+    """The reason identity is a route and not a fourth weight.
+
+    These two are 20 hours apart and the teaser says nothing about Discord, so
+    every circumstantial signal is near zero — proximity scores 0.17 and the
+    weighted total 0.075, a long way under the bar. What links them is that
+    the creator exported both from one file and the name came along.
+
+    Measured equivalents on the live instance: `ConnFront` ↔ `ConnFront` and
+    `Thicc Tomboy scene2` ↔ `Thicc_Tomboy_scene2`, the latter 23.8 hours apart
+    at a proximity of 0.005.
+    """
+    artist, patreon, discord = await _artist_with_channels(db, "namedartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(hours=20),
+        body="a little preview", names=["ConnFront"],
+    )
+    drop = await _drop(db, artist, discord, at=now, names=["01_ConnFront"])
+    await db.commit()
+
+    made = await PostAssociationService(db).match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assert made == 1
+    assoc = (await db.execute(select(PostAssociation))).scalar_one()
+    assert assoc.payload_post_id == drop.id
+    assert assoc.signals["identity"] == 1.0
+    assert weighted_score(assoc.signals) < DEFAULT_THRESHOLD, (
+        "the circumstantial bundle alone must NOT reach the bar here — if it "
+        "does, this test has stopped proving what it claims to"
+    )
+
+
+@pytest.mark.asyncio
+async def test_the_proposal_says_which_name_it_matched_on(db):
+    """A review queue that cannot explain itself is one the operator learns to
+    click through without reading."""
+    artist, patreon, discord = await _artist_with_channels(db, "explainartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(hours=20),
+        body="a little preview", names=["LoisLaneTB2"],
+    )
+    await _drop(db, artist, discord, at=now, names=["01_LoisLaneTB2"])
+    await db.commit()
+
+    await PostAssociationService(db).match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assoc = (await db.execute(select(PostAssociation))).scalar_one()
+    assert assoc.signals["identity_token"] == "loislanetb2"
+
+
+@pytest.mark.asyncio
+async def test_a_name_the_creator_reuses_everywhere_links_nothing(db):
+    """THE false-positive guard for this route.
+
+    A character name is not an identity. Measured on artist 8: `anya` is on
+    images in 8 posts, `riju` 9, `bea` 16 — ungated, every Anya post would
+    match every Anya drop, and the matcher would confidently assert that a
+    year of unrelated pieces are all the same piece.
+    """
+    artist, patreon, discord = await _artist_with_channels(db, "habitartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(hours=20),
+        body="a little preview", ext="t0", names=["Anya"],
+    )
+    await _drop(db, artist, discord, at=now, names=["01_Anya"])
+    # The same name across enough of this artist's OTHER posts to make it a
+    # habit. Nothing about the pair above changes; only its context does.
+    for i in range(7):
+        await _teaser(
+            db, artist, patreon, at=now - timedelta(days=30 + i),
+            body="older", ext=f"other{i}", names=["Anya"],
+        )
+    await db.commit()
+
+    made = await PostAssociationService(db).match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assert made == 0
+    assert (await db.execute(select(PostAssociation))).scalars().all() == []
+
+
+@pytest.mark.asyncio
+async def test_a_name_below_the_floor_is_recorded_but_carries_nothing(db):
+    """Identity may override, never dilute. Below the floor it is real
+    evidence that is not strong enough to assert sameness on its own, so it is
+    written down for the operator and moves no score.
+
+    The measured cost of the floor sitting at 0.75: artist 8's `680lc` and
+    `cnni18x` pairs each span four posts and are genuine, and neither will
+    propose on this signal alone.
+    """
+    artist, patreon, discord = await _artist_with_channels(db, "floorartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(hours=20),
+        body="a little preview", ext="t0", names=["cnni18x"],
+    )
+    await _drop(db, artist, discord, at=now, names=["01_cnni18x"])
+    for i in range(2):
+        await _teaser(
+            db, artist, patreon, at=now - timedelta(days=30 + i),
+            body="older", ext=f"other{i}", names=["cnni18x"],
+        )
+    await db.commit()
+
+    svc = PostAssociationService(db)
+    made = await svc.match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assert made == 0
+    corpus = await svc._corpus(artist.id)
+    assert corpus.token_posts["cnni18x"] == 4, "four posts carry the name"
+
+
+# --- the marker, and the vocabulary ----------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_a_marker_the_creator_uses_once_lifts_a_pair_over_the_line(db):
+    """The operator's own tie-back: `🍈🍈` in the Patreon title and
+    `@everyone 🍈 🍈` in the Discord message. No vocabulary list would predict
+    it, and no filename carries it.
+
+    Measured here: 0.446 on proximity alone, 0.646 once the marker counts —
+    the bar is 0.60. The mirror case is the guard below it.
+    """
+    artist, patreon, discord = await _artist_with_channels(db, "markerartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(minutes=12),
+        body="it's up", title="Anya -- \U0001F348\U0001F348",
+    )
+    await _drop(db, artist, discord, at=now, body="@everyone \U0001F348 \U0001F348")
+    await db.commit()
+
+    made = await PostAssociationService(db).match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assert made == 1
+    assoc = (await db.execute(select(PostAssociation))).scalar_one()
+    assert assoc.signals["marker"] == 1.0
+
+
+@pytest.mark.asyncio
+async def test_a_marker_the_creator_uses_constantly_lifts_nothing(db):
+    """The same arrangement, with the marker turned into punctuation.
+
+    This is a real measured proposal, not a hypothetical: 💦 is in 13 of artist
+    8's 300 posts and it was the DECIDING term for a pair that proximity alone
+    scored 0.441. A habitual marker riding along with proximity is just
+    proximity wearing a hat, which the threshold sits above 0.45 to prevent.
+    """
+    artist, patreon, discord = await _artist_with_channels(db, "punctartist")
+    now = datetime.now(UTC)
+    teaser = await _teaser(
+        db, artist, patreon, at=now - timedelta(minutes=12),
+        body="it's up", title="Drizzle \U0001F4A6", ext="t0",
+    )
+    await _drop(db, artist, discord, at=now, body="@everyone \U0001F4A6")
+    for i in range(5):
+        await _teaser(
+            db, artist, patreon, at=now - timedelta(days=30 + i),
+            body="\U0001F4A6", title=f"older {i}", ext=f"other{i}",
+        )
+    await db.commit()
+
+    made = await PostAssociationService(db).match_post(
+        teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
+    )
+    await db.commit()
+
+    assert made == 0
+
+
+def test_the_creators_own_phrasing_counts_as_a_declaration():
+    """`discord` is not how these creators actually write once the audience
+    already knows where the server is. Measured on artist 8: 21 of 42 posts
+    say `discord`, 7 say `the server`, and it is the RECENT ones that say the
+    latter — so a vocabulary list written from old posts silently stops
+    matching the posts that still need it."""
+    assert declared_signal("<p>Full res is on the server</p>") == DECLARED_MENTION
+    assert declared_signal("<p>up on our server now</p>") == DECLARED_MENTION
+    assert declared_signal("<p>posted to my server</p>") == DECLARED_MENTION
+
+
+def test_an_unrelated_server_is_still_not_a_declaration():
+    """The widened vocabulary must not widen into ordinary prose."""
+    assert declared_signal("<p>the servers were down all morning</p>") == 0.0