Files
FabledCurator/tests/test_post_association.py
T
bvandeusenandClaude Opus 5 f7b3e15014
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 29s
CI and images / integration (push) Successful in 2m12s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 1m43s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped
feat: the drop carrying the teaser's own image links it (4392)
Crop-to-source matching was held until the cheap signals could be shown
insufficient. They can: of artist 8's 27 teasers with a drop inside a day, 11
go unlinked, and five are screenshot teasers with no working name at all.

So it was tried, on exactly those pairs. Every teaser image correlated against
every window of every nearby drop image at five scales, ground truth being the
pairs the working name independently confirms, control being unrelated
same-artist posts a month away. **It does not separate** — true pairs score as
low as 0.401 while the control reaches 0.605, and no threshold divides them.

The reason is the one the naive version was rejected for, which turns out to
apply just as hard to the careful one: a single artist's work is
stylistically homogeneous, so a whole-image comparison between two of their
pieces is high whether or not it is the same piece. That is now written down
in the module docstring with its numbers, so the next person to reach for it
inherits the measurement instead of repeating it.

What survived asks a narrower question the measurement shows IS answerable:
not "is this a crop of that" but "is this the same image". Same pairs, same
control, using the pHash FC already stores on every image — pairs the name
confirms score 0, 0 and 20 bits of 256; the nearest unrelated pair in a
29-sample control scores 108. The threshold sits at 32, which is the number
gallery_service already calls a near-duplicate, inside a 76-bit gap.

It earns its place by being the only signal needing no cooperation from the
creator: it works on a teaser called `Screenshot 2026-08-13`, and on a creator
whose two platforms share no naming convention. It is quiet most of the time,
because a teaser is usually a crop rather than a copy — but where it fires it
is close to certain, and it recovers `Cute Selfie, Cute Dress` from the
unreachable list.

utils/phash warns the hash alone must not decide a MERGE, since variants of
one piece collide at this distance. That does not invert here — it is the
point. A merge destroys a file, so a variant colliding with its original is a
loss; this asks whether two POSTS are about the same piece, and a variant of
the drop's image is exactly that. Nothing is deleted either way.

Gated on posts like the other two: an image on many of the creator's posts is
a banner, not a piece. `_rarity` is public as `rarity` now that all three
signals share it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
2026-09-24 08:27:20 -04:00

953 lines
36 KiB
Python

"""Milestone 388 E5: which Patreon post announced which Discord drop.
The failure this step must not have is a WRONG link. Telling the operator that
two different pieces are one is worse than telling them nothing — no link
leaves them where they already were, a wrong one actively misinforms. So most
of what follows pins refusals, and the central one is structural rather than
behavioural: the threshold sits above every single signal weight, which is what
makes "time proximity alone is never sufficient" arithmetic instead of a hope.
"""
from collections import Counter
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import select
from backend.app.models import (
Artist,
ImageRecord,
ImportSettings,
Post,
PostAssociation,
Source,
)
from backend.app.services.discord_grouping import DROP_GROUPER
from backend.app.services.post_association_service import (
DECLARED_MENTION,
DUPLICATE_MAX_DISTANCE,
WEIGHTS,
PostAssociationService,
declared_signal,
proximity_signal,
rescan,
shared_image,
weighted_score,
)
from backend.app.services.post_feed_service import PostFeedService
pytestmark = pytest.mark.integration
DEFAULT_THRESHOLD = 0.60
WINDOW = 24.0
# --- the structural guard -------------------------------------------------
def test_no_single_signal_can_reach_the_threshold():
"""THE load-bearing property of this matcher.
On a busy day an artist posts several times, so a matcher that could pair
on proximity alone would turn every busy day into false pairs and the
review queue would be abandoned. Requiring two signals is what prevents
that — and it is a fact about the WEIGHTS, not about any code path, so it
survives every refactor of the scorer.
If this fails, either a weight grew or the default threshold dropped. Do
not "fix" it by lowering the assertion; the arithmetic IS the safeguard.
"""
assert max(WEIGHTS.values()) < DEFAULT_THRESHOLD
assert sum(WEIGHTS.values()) == pytest.approx(1.0)
def test_perfect_proximity_alone_does_not_propose():
"""The same property, expressed through the scorer."""
score = weighted_score({"proximity": 1.0, "declared": 0.0})
assert score < DEFAULT_THRESHOLD
def test_an_explicit_declaration_alone_does_not_propose():
score = weighted_score({"proximity": 0.0, "declared": 1.0})
assert score < DEFAULT_THRESHOLD
def test_both_signals_together_do_propose():
assert weighted_score({"proximity": 1.0, "declared": 1.0}) >= DEFAULT_THRESHOLD
# --- the signals ----------------------------------------------------------
def test_proximity_decays_to_zero_at_the_window_edge():
window = timedelta(hours=24)
assert proximity_signal(timedelta(0), window) == 1.0
assert proximity_signal(timedelta(hours=24), window) == 0.0
assert proximity_signal(timedelta(hours=48), window) == 0.0
assert 0.4 < proximity_signal(timedelta(hours=12), window) < 0.6
def test_proximity_is_symmetric_because_either_can_land_first():
"""The teaser usually goes up around the drop, not reliably before it."""
window = timedelta(hours=24)
assert proximity_signal(timedelta(hours=-2), window) == proximity_signal(
timedelta(hours=2), window
)
def test_an_invite_link_scores_higher_than_a_bare_mention():
assert declared_signal("Full set on my discord.gg/abc123 now!") == 1.0
assert 0 < declared_signal("Posted the rest on discord earlier") < 1.0
assert declared_signal("New piece, hope you like it") == 0.0
assert declared_signal(None) == 0.0
def test_the_declaration_signal_reads_through_html():
"""Post bodies are HTML; a link inside an anchor tag must still count."""
assert declared_signal(
'<p>Full set: <a href="https://discord.gg/xyz">here</a></p>'
) > 0
def test_a_word_containing_discord_is_not_a_mention():
"""\"discordant\" is not a declaration. Without the word boundary the
signal fires on ordinary prose and drags pairs over the threshold."""
assert declared_signal("a discordant palette, deliberately") == 0.0
# --- end to end -----------------------------------------------------------
async def _artist_with_channels(db, name: str):
artist = Artist(name=name, slug=name.lower().replace(" ", "-"))
db.add(artist)
await db.flush()
patreon = Source(
artist_id=artist.id, platform="patreon",
url=f"https://patreon.com/{name}", enabled=True,
)
discord = Source(
artist_id=artist.id, platform="discord",
url=f"https://discord.com/channels/1/{name}", enabled=True,
)
db.add_all([patreon, discord])
await db.flush()
return artist, patreon, discord
async def _images(db, artist, post, ext, names, phashes=None):
"""Attach named files to a post. The NAME is the point — the working-name
signal reads it, so a test that cares about identity supplies one.
`phashes` aligns with `names`; a test that cares about the shared-image
signal supplies those instead (or as well).
"""
for i, name in enumerate(names):
db.add(ImageRecord(
path=f"/images/{artist.id}/{ext}_{i}_{name}.jpg",
sha256=f"{ext}{i}{name}".ljust(64, "0")[:64],
phash=(phashes or [None] * len(names))[i],
size_bytes=10, mime="image/jpeg", width=10, height=10,
origin="downloaded", primary_post_id=post.id, artist_id=artist.id,
))
await db.flush()
async def _teaser(db, artist, source, *, at, body, ext="teaser", names=None,
title="New piece", phashes=None):
post = Post(
source_id=source.id, artist_id=artist.id, external_post_id=ext,
post_date=at, post_title=title, description=body,
)
db.add(post)
await db.flush()
await _images(db, artist, post, ext, names or [ext], phashes)
return post
async def _drop(db, artist, source, *, at, ext="fc-drop:1", body=None,
names=(), phashes=None):
post = Post(
source_id=source.id, artist_id=artist.id, external_post_id=ext,
post_date=at, description=body, synthesized_by=DROP_GROUPER,
synthesis_details={"message_count": 4, "images_since_surface": 0},
)
db.add(post)
await db.flush()
if names:
await _images(db, artist, post, ext.replace(":", "-"), names, phashes)
return post
@pytest.mark.asyncio
async def test_a_teaser_and_its_drop_an_hour_apart_are_proposed(db):
artist, patreon, discord = await _artist_with_channels(db, "pairartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=3),
body="Full set is up on discord.gg/abc now",
)
drop = await _drop(db, artist, discord, at=now - timedelta(hours=2))
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (1, 0)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.announcement_post_id == teaser.id
assert assoc.payload_post_id == drop.id
assert assoc.status == "pending", "nothing is linked without the operator"
# The per-signal breakdown survives, so the proposal stays explicable
# after the weights or threshold move.
assert assoc.signals["declared"] == 1.0
assert assoc.signals["proximity"] > 0.9
@pytest.mark.asyncio
async def test_two_unrelated_posts_the_same_day_are_not_proposed(db):
"""Time proximity alone must not be sufficient, or every busy day becomes
a false pair. The teaser here says nothing about Discord."""
artist, patreon, discord = await _artist_with_channels(db, "busyartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=3),
body="Just a sketch I liked",
)
await _drop(db, artist, discord, at=now - timedelta(hours=2))
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (0, 0)
assert (await db.execute(select(PostAssociation))).scalars().all() == []
@pytest.mark.asyncio
async def test_a_drop_outside_the_window_is_not_proposed(db):
artist, patreon, discord = await _artist_with_channels(db, "farapartartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(days=10),
body="Everything is on discord.gg/abc",
)
await _drop(db, artist, discord, at=now)
await db.commit()
assert await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (0, 0)
@pytest.mark.asyncio
async def test_another_artists_drop_is_never_proposed(db):
"""Same-artist is a HARD filter, not a scored signal: two different
creators posting minutes apart is a coincidence, not evidence."""
artist_a, patreon_a, _ = await _artist_with_channels(db, "artista")
_artist_b, _patreon_b, discord_b = await _artist_with_channels(db, "artistb")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist_a, patreon_a, at=now - timedelta(hours=1),
body="new drop on discord.gg/abc",
)
await _drop(db, _artist_b, discord_b, at=now)
await db.commit()
assert await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (0, 0)
@pytest.mark.asyncio
async def test_an_artist_with_no_discord_source_produces_nothing_and_no_error(db):
artist = Artist(name="soloartist", slug="soloartist")
db.add(artist)
await db.flush()
patreon = Source(
artist_id=artist.id, platform="patreon",
url="https://patreon.com/solo", enabled=True,
)
db.add(patreon)
await db.flush()
teaser = await _teaser(
db, artist, patreon, at=datetime.now(UTC),
body="on discord.gg/abc", ext="solo",
)
await db.commit()
assert await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (0, 0)
@pytest.mark.asyncio
async def test_a_synthetic_post_cannot_announce_anything(db):
"""FC wrote it, so it announces nothing — and a grouping proposing itself
as the teaser for another grouping would be pure noise."""
artist, _patreon, discord = await _artist_with_channels(db, "noselfannounce")
now = datetime.now(UTC)
drop_a = await _drop(db, artist, discord, at=now - timedelta(hours=1), ext="fc-drop:a")
await _drop(db, artist, discord, at=now, ext="fc-drop:b")
await db.commit()
assert await PostAssociationService(db).match_post(
drop_a.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (0, 0)
@pytest.mark.asyncio
async def test_a_dismissed_pair_is_never_proposed_again(db):
"""The row is what remembers the rejection. Re-proposing a rejected pair on
every scan is the single behaviour that makes a review queue get ignored."""
artist, patreon, discord = await _artist_with_channels(db, "dismissartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=2),
body="discord.gg/abc has the rest",
)
await _drop(db, artist, discord, at=now - timedelta(hours=1))
await db.commit()
svc = PostAssociationService(db)
assert await svc.match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (1, 0)
await db.commit()
assoc = (await db.execute(select(PostAssociation))).scalar_one()
await svc.dismiss(assoc.id)
await db.commit()
assert await svc.match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
) == (0, 0)
await db.commit()
assert len((await db.execute(select(PostAssociation))).scalars().all()) == 1
@pytest.mark.asyncio
async def test_only_an_accepted_link_reaches_the_post_payload(db):
"""A pending proposal is a question for the review queue, not a claim to
render beside the artwork."""
artist, patreon, discord = await _artist_with_channels(db, "payloadartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=2),
body="rest is on discord.gg/abc",
)
drop = await _drop(db, artist, discord, at=now - timedelta(hours=1))
await db.commit()
svc = PostAssociationService(db)
await svc.match_post(teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW)
await db.commit()
feed = PostFeedService(db)
assert (await feed.get_post(teaser.id))["associations"] == []
assoc = (await db.execute(select(PostAssociation))).scalar_one()
await svc.accept(assoc.id)
await db.commit()
# Both ends see it, and each sees the OTHER post with its own role.
teaser_item = await feed.get_post(teaser.id)
assert teaser_item["associations"] == [
{"role": "announces", "post_id": drop.id, "id": assoc.id}
]
drop_item = await feed.get_post(drop.id)
assert drop_item["associations"] == [
{"role": "announced_by", "post_id": teaser.id, "id": assoc.id}
]
@pytest.mark.asyncio
async def test_deleting_the_grouping_takes_its_proposals_with_it(db):
"""E3's reversal path is one DELETE; it must not leave a dangling proposal
pointing at a post that no longer exists."""
artist, patreon, discord = await _artist_with_channels(db, "cascadeartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=2),
body="discord.gg/abc",
)
drop = await _drop(db, artist, discord, at=now - timedelta(hours=1))
await db.commit()
await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert len((await db.execute(select(PostAssociation))).scalars().all()) == 1
await db.delete(drop)
await db.commit()
assert (await db.execute(select(PostAssociation))).scalars().all() == []
@pytest.mark.asyncio
async def test_the_rescan_is_a_no_op_when_the_switch_is_off(db):
settings = await ImportSettings.load(db)
settings.discord_link_enabled = False
await db.commit()
assert await rescan(db) == {
"enabled": False, "scanned": 0, "proposed": 0, "linked": 0,
}
# --- the identity route ----------------------------------------------------
#
# Milestone 388. Everything below was calibrated against the operator's real
# library (artist 8: 520 images, 15 same-artist pairs that share a name, no
# false positives) rather than invented, so the numbers in these docstrings are
# measurements.
@pytest.mark.asyncio
async def test_a_shared_working_name_proposes_a_pair_time_cannot_reach(db):
"""The reason identity is a route and not a fourth weight.
These two are 20 hours apart and the teaser says nothing about Discord, so
every circumstantial signal is near zero — proximity scores 0.17 and the
weighted total 0.075, a long way under the bar. What links them is that
the creator exported both from one file and the name came along.
Measured equivalents on the live instance: `ConnFront` ↔ `ConnFront` and
`Thicc Tomboy scene2` ↔ `Thicc_Tomboy_scene2`, the latter 23.8 hours apart
at a proximity of 0.005.
"""
artist, patreon, discord = await _artist_with_channels(db, "namedartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", names=["ConnFront"],
)
drop = await _drop(db, artist, discord, at=now, names=["01_ConnFront"])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (1, 0)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.payload_post_id == drop.id
assert assoc.signals["identity"] == 1.0
assert weighted_score(assoc.signals) < DEFAULT_THRESHOLD, (
"the circumstantial bundle alone must NOT reach the bar here — if it "
"does, this test has stopped proving what it claims to"
)
@pytest.mark.asyncio
async def test_the_proposal_says_which_name_it_matched_on(db):
"""A review queue that cannot explain itself is one the operator learns to
click through without reading."""
artist, patreon, discord = await _artist_with_channels(db, "explainartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", names=["LoisLaneTB2"],
)
await _drop(db, artist, discord, at=now, names=["01_LoisLaneTB2"])
await db.commit()
await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.signals["identity_token"] == "loislanetb2"
@pytest.mark.asyncio
async def test_a_name_the_creator_reuses_everywhere_links_nothing(db):
"""THE false-positive guard for this route.
A character name is not an identity. Measured on artist 8: `anya` is on
images in 8 posts, `riju` 9, `bea` 16 — ungated, every Anya post would
match every Anya drop, and the matcher would confidently assert that a
year of unrelated pieces are all the same piece.
"""
artist, patreon, discord = await _artist_with_channels(db, "habitartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", ext="t0", names=["Anya"],
)
await _drop(db, artist, discord, at=now, names=["01_Anya"])
# The same name across enough of this artist's OTHER posts to make it a
# habit. Nothing about the pair above changes; only its context does.
for i in range(7):
await _teaser(
db, artist, patreon, at=now - timedelta(days=30 + i),
body="older", ext=f"other{i}", names=["Anya"],
)
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (0, 0)
assert (await db.execute(select(PostAssociation))).scalars().all() == []
@pytest.mark.asyncio
async def test_a_name_below_the_floor_is_recorded_but_carries_nothing(db):
"""Identity may override, never dilute. Below the floor it is real
evidence that is not strong enough to assert sameness on its own, so it is
written down for the operator and moves no score.
The measured cost of the floor sitting at 0.75: artist 8's `680lc` and
`cnni18x` pairs each span four posts and are genuine, and neither will
propose on this signal alone.
"""
artist, patreon, discord = await _artist_with_channels(db, "floorartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", ext="t0", names=["cnni18x"],
)
await _drop(db, artist, discord, at=now, names=["01_cnni18x"])
for i in range(2):
await _teaser(
db, artist, patreon, at=now - timedelta(days=30 + i),
body="older", ext=f"other{i}", names=["cnni18x"],
)
await db.commit()
svc = PostAssociationService(db)
made = await svc.match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (0, 0)
corpus = await svc._corpus(artist.id)
assert corpus.token_posts["cnni18x"] == 4, "four posts carry the name"
# --- the marker, and the vocabulary ----------------------------------------
@pytest.mark.asyncio
async def test_a_marker_the_creator_uses_once_lifts_a_pair_over_the_line(db):
"""The operator's own tie-back: `🍈🍈` in the Patreon title and
`@everyone 🍈 🍈` in the Discord message. No vocabulary list would predict
it, and no filename carries it.
Measured here: 0.446 on proximity alone, 0.646 once the marker counts —
the bar is 0.60. The mirror case is the guard below it.
"""
artist, patreon, discord = await _artist_with_channels(db, "markerartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(minutes=12),
body="it's up", title="Anya -- \U0001F348\U0001F348",
)
await _drop(db, artist, discord, at=now, body="@everyone \U0001F348 \U0001F348")
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (1, 0)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.signals["marker"] == 1.0
@pytest.mark.asyncio
async def test_a_marker_the_creator_uses_constantly_lifts_nothing(db):
"""The same arrangement, with the marker turned into punctuation.
This is a real measured proposal, not a hypothetical: 💦 is in 13 of artist
8's 300 posts and it was the DECIDING term for a pair that proximity alone
scored 0.441. A habitual marker riding along with proximity is just
proximity wearing a hat, which the threshold sits above 0.45 to prevent.
"""
artist, patreon, discord = await _artist_with_channels(db, "punctartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(minutes=12),
body="it's up", title="Drizzle \U0001F4A6", ext="t0",
)
await _drop(db, artist, discord, at=now, body="@everyone \U0001F4A6")
for i in range(5):
await _teaser(
db, artist, patreon, at=now - timedelta(days=30 + i),
body="\U0001F4A6", title=f"older {i}", ext=f"other{i}",
)
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
)
await db.commit()
assert made == (0, 0)
def test_the_creators_own_phrasing_counts_as_a_declaration():
"""`discord` is not how these creators actually write once the audience
already knows where the server is. Measured on artist 8: 21 of 42 posts
say `discord`, 7 say `the server`, and it is the RECENT ones that say the
latter — so a vocabulary list written from old posts silently stops
matching the posts that still need it."""
assert declared_signal("<p>Full res is on the server</p>") == DECLARED_MENTION
assert declared_signal("<p>up on our server now</p>") == DECLARED_MENTION
assert declared_signal("<p>posted to my server</p>") == DECLARED_MENTION
def test_an_unrelated_server_is_still_not_a_declaration():
"""The widened vocabulary must not widen into ordinary prose."""
assert declared_signal("<p>the servers were down all morning</p>") == 0.0
@pytest.mark.asyncio
async def test_a_drop_grouped_today_pulls_in_the_post_that_announced_it(db):
"""#4392's third cause, and the only one of the three about whether a pair
is SCORED AT ALL rather than how.
A drop's `post_date` is backdated to its first message, but FC cannot
author the drop until that message has an embedding and the hourly grouper
has run. So a drop created this minute lands wherever its messages were —
here, ten days back, far outside the sweep's horizon. Keyed only on how
recent the announcement is, the sweep looks straight past the pair and
never comes back to it.
Measured on the live instance: a pair scoring 0.800 with an empty review
queue.
"""
artist, patreon, discord = await _artist_with_channels(db, "lateartist")
now = datetime.now(UTC)
old = now - timedelta(days=10)
teaser = await _teaser(
db, artist, patreon, at=old, body="Full set is on discord.gg/abc",
)
# post_date is backdated; downloaded_at defaults to now, which is when FC
# actually wrote this row.
drop = await _drop(db, artist, discord, at=old + timedelta(hours=1))
settings = await ImportSettings.load(db)
settings.discord_link_enabled = True
await db.commit()
result = await rescan(db)
await db.commit()
assert result["proposed"] == 1, (
"the announcement is 10 days old and the drop was authored today — "
"a sweep keyed only on the announcement's own date never sees it"
)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.announcement_post_id == teaser.id
assert assoc.payload_post_id == drop.id
@pytest.mark.asyncio
async def test_the_sweep_still_ignores_a_drop_nothing_is_near(db):
"""The widened sweep must not become the full-library rescan it replaced.
A drop authored today with no announcement in range proposes nothing."""
artist, patreon, discord = await _artist_with_channels(db, "lonelyartist")
now = datetime.now(UTC)
await _teaser(
db, artist, patreon, at=now - timedelta(days=60),
body="Full set is on discord.gg/abc",
)
await _drop(db, artist, discord, at=now - timedelta(days=10))
settings = await ImportSettings.load(db)
settings.discord_link_enabled = True
await db.commit()
result = await rescan(db)
await db.commit()
assert result["proposed"] == 0
assert (await db.execute(select(PostAssociation))).scalars().all() == []
# --- linking without asking ------------------------------------------------
#
# Operator, 2026-09-24: "I don't want this to be manual that defeats the
# convenience that I'm going for." Confirm-only was right while every signal
# was circumstantial; a name the creator uses on these two posts and nowhere
# else is different in kind. These pin where that line sits.
@pytest.mark.asyncio
async def test_a_conclusive_name_links_without_asking(db):
"""A name appearing in exactly these two posts is not strong evidence —
within the artist's library it is conclusive, and there is nothing left for
the operator to adjudicate. 20 hours apart, with nothing else to go on."""
artist, patreon, discord = await _artist_with_channels(db, "autoartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", names=["Frieren1Clean"],
)
await _drop(db, artist, discord, at=now, names=["01_Frieren1Clean"])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made == (1, 1)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.status == "linked"
@pytest.mark.asyncio
async def test_the_same_pair_still_waits_when_the_switch_is_off(db):
"""The setting is the whole difference; nothing else about the pair
changes. Confirm-only has to remain reachable, because "link it yourself"
is a change of posture and not everyone wants it."""
artist, patreon, discord = await _artist_with_channels(db, "manualartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", names=["Frieren1Clean"],
)
await _drop(db, artist, discord, at=now, names=["01_Frieren1Clean"])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=False,
)
await db.commit()
assert made == (1, 0)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.status == "pending"
@pytest.mark.asyncio
async def test_evidence_short_of_conclusive_is_queued_not_linked(db):
"""The gap between IDENTITY_FLOOR and AUTO_LINK_FLOOR, which is the review
queue. This pair proposes on a name spanning three posts — real evidence,
not certain enough for FC to act on by itself. Measured equivalent on
artist 8: `0-k`, the operator's own example, at three posts."""
artist, patreon, discord = await _artist_with_channels(db, "queuedartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20),
body="a little preview", ext="t0", names=["0-k"],
)
await _drop(db, artist, discord, at=now, names=["01_0-k"])
await _teaser(
db, artist, patreon, at=now - timedelta(days=30),
body="older", ext="other", names=["0-k_wip1"],
)
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made == (1, 0)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.status == "pending"
assert assoc.signals["identity"] == 0.75
@pytest.mark.asyncio
async def test_two_drops_sharing_one_name_link_neither(db):
"""Not a tie to be broken by score.
One post can tease two pieces that were dropped separately, and then each
drop carries a DIFFERENT name from the same teaser — each spanning exactly
two posts, so each is individually conclusive. Two conclusive answers to
"which drop is this" is not a close call; it means the question was wrong.
Both are queued and FC says nothing.
(One name cannot do this on its own: under post-span counting, a token in
the teaser and two drops spans three posts and is not conclusive at all.
The ambiguity has to come from two names, which is why this fixture has
two.)
"""
artist, patreon, discord = await _artist_with_channels(db, "ambigartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=2),
body="a little preview", names=["Sabirth", "Terra3"],
)
await _drop(db, artist, discord, at=now, ext="fc-drop:1",
names=["01_Sabirth"])
await _drop(db, artist, discord, at=now - timedelta(hours=4),
ext="fc-drop:2", names=["01_Terra3"])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
proposed, linked = made
assert (proposed, linked) == (2, 0)
rows = (await db.execute(select(PostAssociation))).scalars().all()
assert {r.status for r in rows} == {"pending"}
@pytest.mark.asyncio
async def test_a_drop_another_post_already_claims_is_never_taken(db):
"""An accepted link is the operator's decision. FC reassigning its other
end on the next sweep would silently overrule them."""
artist, patreon, discord = await _artist_with_channels(db, "claimedartist")
now = datetime.now(UTC)
first = await _teaser(
db, artist, patreon, at=now - timedelta(hours=2),
body="first", ext="t0", names=["Terra3"],
)
# The drop carries both names, so the second teaser's claim on it is
# CONCLUSIVE on `Svtt` — two posts, nowhere else. Only the existing link
# stops it, which is the point being pinned.
drop = await _drop(db, artist, discord, at=now,
names=["01_Terra3", "02_Svtt"])
second = await _teaser(
db, artist, patreon, at=now - timedelta(hours=3),
body="second", ext="t1", names=["Svtt"],
)
db.add(PostAssociation(
announcement_post_id=first.id, payload_post_id=drop.id,
score=1.0, signals={"identity": 1.0}, status="linked",
))
await db.commit()
made = await PostAssociationService(db).match_post(
second.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made[1] == 0, "the drop is already spoken for"
rows = (await db.execute(
select(PostAssociation).where(
PostAssociation.announcement_post_id == second.id
)
)).scalars().all()
assert [r.status for r in rows] == ["pending"]
# --- when the drop just contains the teaser's image -------------------------
#
# Measured on artist 8, every teaser against every drop within a day: pairs the
# working name independently confirms score 0, 0 and 20 bits of 256, and the
# nearest unrelated same-artist pair in a 29-sample control scores 108. The
# threshold sits at 32 — the same number gallery_service already calls a
# near-duplicate — inside a 76-bit gap.
_PIECE = "a5" * 32 # the image
_REENCODED = "a5" * 31 + "a4" # the same image, one bit different
_UNRELATED = "5a" * 32 # 256 bits away — every bit differs
@pytest.mark.asyncio
async def test_the_drop_carrying_the_teasers_own_image_links_it(db):
"""The one signal that needs no cooperation from the creator. No shared
name, nothing said about Discord, 20 hours apart — and the drop is
carrying the same picture."""
artist, patreon, discord = await _artist_with_channels(db, "dupartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
ext="t0", names=["Alpha"], phashes=[_PIECE],
)
await _drop(db, artist, discord, at=now, names=["Beta"],
phashes=[_REENCODED])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made == (1, 1)
assoc = (await db.execute(select(PostAssociation))).scalar_one()
assert assoc.signals["identity_image"] == 1.0
assert "identity_token" not in assoc.signals, (
"the names share nothing — claiming one would be a false reason"
)
@pytest.mark.asyncio
async def test_a_different_picture_links_nothing(db):
"""The negative the threshold exists for. 256 bits apart is two different
images, whatever else the posts have in common."""
artist, patreon, discord = await _artist_with_channels(db, "diffartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
ext="t0", names=["Alpha"], phashes=[_PIECE],
)
await _drop(db, artist, discord, at=now, names=["Beta"],
phashes=[_UNRELATED])
await db.commit()
made = await PostAssociationService(db).match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made == (0, 0)
@pytest.mark.asyncio
async def test_an_image_the_creator_reuses_everywhere_links_nothing(db):
"""The same guard the other two signals have, on the third. A banner, a
watermark plate or a recurring title card is a habit, not a piece — and it
would otherwise link every post carrying it to every drop carrying it."""
artist, patreon, discord = await _artist_with_channels(db, "bannerartist")
now = datetime.now(UTC)
teaser = await _teaser(
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
ext="t0", names=["Alpha"], phashes=[_PIECE],
)
await _drop(db, artist, discord, at=now, names=["Beta"],
phashes=[_REENCODED])
for i in range(7):
await _teaser(
db, artist, patreon, at=now - timedelta(days=30 + i), body="older",
ext=f"other{i}", names=[f"Gamma{i}"], phashes=[_PIECE],
)
await db.commit()
svc = PostAssociationService(db)
made = await svc.match_post(
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
auto_link=True,
)
await db.commit()
assert made == (0, 0)
corpus = await svc._corpus(artist.id)
assert corpus.hash_posts[int(_PIECE, 16)] == 8
def test_a_missing_hash_is_not_a_match():
"""Fails CLOSED, like every gate in utils/phash. An image whose pHash was
never computed must read as "no evidence", never as "identical to the
other thing that also has none"."""
assert shared_image([], [], Counter()) == 0.0
def test_the_duplicate_threshold_sits_inside_the_measured_gap():
"""Stated as a property so the number cannot drift out of the gap that
justifies it: 20 bits was the widest true pair, 108 the nearest unrelated
one."""
assert 20 < DUPLICATE_MAX_DISTANCE < 108