feat: the matcher gets an identity route, so a name can link what time cannot (4392)
CI and images / lint (push) Successful in 4s
CI and images / extension-version (push) Successful in 4s
CI and images / frontend-build (push) Successful in 26s
CI and images / backend-lint-and-test (push) Successful in 34s
CI and images / integration (push) Successful in 2m52s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 6s
CI and images / build-web (push) Successful in 2m6s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped

#4392 measured three compounding reasons the announcement matcher could not
fire for this creator. This is the first two.

**Vocabulary.** `discord` is not how these creators write once the audience
already knows where the server is. Across 20,558 Patreon bodies `discord`
appears in 486 and `the server` in 37 — but the distribution is the point, not
the totals: for artist 8, 21 of 42 posts say `discord`, 7 say `the server`,
and it is the RECENT ones that say the latter. A vocabulary list written from
old posts silently stops matching the posts that still need it.

**Two routes.** Circumstance and identity answer different questions.
Proximity and a declaration say two things happened near each other; a working
name the creator uses on these two posts and nowhere else says they are the
same piece. Averaging those makes the threshold uninterpretable and costs
both ways — adding identity as a fourth weight dilutes the others enough that
measured teaser/drop pairs an hour apart stop proposing, while capping
identity at a weight means the strongest evidence available can never carry a
pair alone. So identity may OVERRIDE, never dilute; below IDENTITY_FLOOR it is
recorded for the operator and moves nothing, which is the conservative
direction, since a wrong link asserts two different pieces are one.

The circumstantial bundle is now proximity/declared/marker at .45/.35/.20.
Rebalanced rather than extended: the existing guard still holds arithmetically
(no single weight reaches the threshold) and the existing pairs still score
where they did — a teaser and its drop an hour apart with an invite link go
from 0.815 to 0.781, both well over.

Measured against the live instance, artist 8: 17 candidate pairs, 14 carried
by a shared name, no false positives, and no drop claimed by two posts. The
operator's own example — `Anya -- 🍈🍈` and the drop 75 minutes before it —
links on the token `0-k`, which is what its two files are called on either
platform. Two pairs 21h apart that proximity scores 0.10 are reached, and two
more (`680lc`, `cnni18x`, four posts each) are the stated cost of the floor.

Both rare-token signals are artist-scoped and loaded once per artist per
sweep, not per candidate pair.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-24 08:07:25 -04:00
co-authored by Claude Opus 5
parent fb7563e263
commit fd214f3a08
2 changed files with 382 additions and 26 deletions
+145 -16
View File
@@ -15,24 +15,41 @@ where they already were, a wrong one actively misinforms and then propagates
into whatever reads the association. So the matcher's job is to make a SHORT
list worth reading, not a long list worth trusting.
## Signals, and the one deliberately NOT built
## Two routes, because the evidence is of two different kinds
CIRCUMSTANTIAL evidence says two things happened near each other. It is
additive, weighted, and no single one of its signals may reach the threshold:
1. **Time proximity.** The Patreon post exists in order to announce the drop,
so the two are minutes-to-hours apart. Nearly free, and strong.
2. **The post says so.** These announcements routinely name Discord or carry
an invite link, which is close to a declaration.
3. **A shared marker.** The creator's own tie-back — `🍈🍈` in the Patreon
title and `@everyone 🍈 🍈` in the Discord message — gated on how rare that
marker is in THIS artist's posts, because a habitual emoji is punctuation.
3. **Crop-to-source matching is HELD, on the plan's own instruction** — it is
real work with real false-positive risk, and it is only worth building once
1 and 2 are shown to be insufficient against the operator's actual artists.
Nothing here should be read as evidence it is unnecessary; it is deferred,
and the thing that would justify it is an empty review queue on a pair the
operator can see with their own eyes.
IDENTITY evidence says two things are the same thing, and it gets its own
route (see `IDENTITY_FLOOR`):
Note also that a naive whole-image SigLIP similarity is NOT that signal. A
cropped teaser and its full version are exactly the pair a whole-image
comparison handles worst, so adding one as a "bonus" would mostly add noise
while looking like progress.
4. **A shared working name.** The creator exports the teaser and the release
from one file, and the internal name survives into both platforms
untouched. Measured on the operator's artist: `ConnFront` ↔ `ConnFront`.
This is the only signal that reaches a pair 23.8 hours apart, which
proximity scores at 0.005.
## The one deliberately NOT built
**Crop-to-source matching is HELD, on the plan's own instruction** — it is
real work with real false-positive risk, and it is only worth building once
the cheap signals are shown to be insufficient against the operator's actual
artists. Half of this creator's recent teasers are screenshots carrying no
working name at all, and those pairs are out of reach here; that, measured, is
what would justify it.
Note also that a naive whole-image SigLIP similarity is NOT that signal. A
cropped teaser and its full version are exactly the pair a whole-image
comparison handles worst, so adding one as a "bonus" would mostly add noise
while looking like progress.
## Creator identity comes free, so E4 is not actually a prerequisite
@@ -56,14 +73,24 @@ from __future__ import annotations
import logging
import re
from collections import Counter
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from sqlalchemy import func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from ..models import ImportSettings, Post, PostAssociation
from ..models import ImageRecord, ImportSettings, Post, PostAssociation
from ..utils.text import html_to_plain
from .discord_grouping import DROP_GROUPER
from .post_naming import (
IDENTITY_FLOOR,
marker_frequencies,
marker_overlap,
shared_identity,
token_frequencies,
working_name_tokens,
)
log = logging.getLogger(__name__)
@@ -76,12 +103,20 @@ log = logging.getLogger(__name__)
# enough" arithmetic rather than aspirational: on a busy day an artist posts
# several times, and a matcher that could pair on proximity alone would turn
# every busy day into false pairs. A guard test pins this.
WEIGHTS = {"proximity": 0.55, "declared": 0.45}
WEIGHTS = {"proximity": 0.45, "declared": 0.35, "marker": 0.20}
# A Discord INVITE in the body is close to a declaration; the bare word is
# weaker but still meaningful, because these posts are short and on-topic.
#
# "the server" and its possessives are here because the word `discord` is NOT
# how these creators actually write. Measured across 20,558 Patreon bodies:
# `discord` appears in 486 and `the server` in 37 — but the distribution is the
# point, not the totals. For the artist this step was built for, 21 of 42 posts
# say `discord` and 7 say `the server`, and it is the RECENT ones that say the
# latter: the phrasing drifted once the audience already knew where the server
# was. A vocabulary list written from old posts silently stops matching.
_INVITE = re.compile(r"discord\.(?:gg|com/invite)/", re.I)
_MENTION = re.compile(r"\bdiscord\b", re.I)
_MENTION = re.compile(r"\b(?:discord|(?:the|our|my)\s+server)\b", re.I)
DECLARED_INVITE = 1.0
DECLARED_MENTION = 0.6
@@ -89,6 +124,23 @@ DECLARED_MENTION = 0.6
MAX_CANDIDATES = 25
@dataclass(frozen=True)
class _Corpus:
"""One artist's rare-token evidence, gathered once rather than per pair.
Both rare-token signals are scoped to a single artist — a working name and
a marker belong to the person who chose them — so the counts are useless
across artists and expensive to rebuild per candidate. A sweep touches an
artist's posts many times over; this is loaded on the first touch and kept
for the life of the service.
"""
tokens_by_post: dict[int, set[str]]
token_posts: Counter[str]
text_by_post: dict[int, str]
marker_posts: Counter[str]
def proximity_signal(gap: timedelta, window: timedelta) -> float:
"""1.0 when the two posts are simultaneous, decaying linearly to 0 at the
window's edge. Linear rather than a step, so a pair an hour outside a
@@ -134,6 +186,46 @@ def _post_time(post: Post) -> datetime:
class PostAssociationService:
def __init__(self, session: AsyncSession):
self.session = session
self._corpora: dict[int, _Corpus] = {}
async def _corpus(self, artist_id: int) -> _Corpus:
if artist_id in self._corpora:
return self._corpora[artist_id]
paths_by_post: dict[int, list[str]] = {}
rows = await self.session.execute(
select(ImageRecord.primary_post_id, ImageRecord.path).where(
ImageRecord.artist_id == artist_id,
ImageRecord.primary_post_id.is_not(None),
)
)
for post_id, path in rows:
paths_by_post.setdefault(post_id, []).append(path)
text_by_post: dict[int, str] = {}
rows = await self.session.execute(
select(Post.id, Post.post_title, Post.description).where(
Post.artist_id == artist_id
)
)
for post_id, title, description in rows:
text_by_post[post_id] = "\n".join(
part for part in (title, html_to_plain(description) or "") if part
)
corpus = _Corpus(
tokens_by_post={
pid: {t for path in paths for t in working_name_tokens(path)}
for pid, paths in paths_by_post.items()
},
# Both counts take POSTS, which is why they are built from these
# groupings rather than from flat lists — see post_naming.
token_posts=token_frequencies(paths_by_post.values()),
text_by_post=text_by_post,
marker_posts=marker_frequencies(text_by_post.values()),
)
self._corpora[artist_id] = corpus
return corpus
async def _decided(self, announcement_id: int) -> set[int]:
"""Payload posts already proposed for this announcement, in ANY status.
@@ -186,20 +278,57 @@ class PostAssociationService:
window = timedelta(hours=window_hours)
declared = declared_signal(announcement.description)
already = await self._decided(announcement_id)
corpus = await self._corpus(announcement.artist_id)
here = corpus.tokens_by_post.get(announcement.id, set())
here_text = corpus.text_by_post.get(announcement.id, "")
made = 0
for group in await self._candidate_groups(announcement, window=window):
if group.id in already:
continue
signals = {
identity, token = shared_identity(
here,
corpus.tokens_by_post.get(group.id, set()),
corpus.token_posts,
)
circumstantial = {
"proximity": proximity_signal(
_post_time(group) - _post_time(announcement), window,
),
"declared": declared,
"marker": marker_overlap(
here_text,
corpus.text_by_post.get(group.id, ""),
corpus.marker_posts,
),
}
score = weighted_score(signals)
score = weighted_score(circumstantial)
# THE TWO ROUTES, and why identity is not simply a fourth weight.
#
# Circumstance and identity answer different questions. Proximity
# and a declaration say two things happened near each other and
# that one of them mentioned Discord; a working name the creator
# uses on these two posts and nowhere else says they are the same
# piece. Averaging those makes the threshold uninterpretable, and
# it costs both: adding identity as a weight dilutes the others
# enough that measured teaser/drop pairs an hour apart stop
# proposing, while capping identity's contribution at its weight
# means the strongest evidence available can never carry a pair on
# its own.
#
# So identity may override, never dilute. Below the floor it is
# recorded for the operator to read and moves nothing — which is
# the conservative direction, since a wrong link asserts that two
# different pieces are one.
if identity >= IDENTITY_FLOOR:
score = max(score, identity)
if score < threshold:
continue
signals = {**circumstantial, "identity": identity}
if token:
# Carried so the queue can say WHY. A review queue that cannot
# explain itself is one the operator learns to click through.
signals["identity_token"] = token
self.session.add(PostAssociation(
announcement_post_id=announcement.id,
payload_post_id=group.id,