feat: the drop carrying the teaser's own image links it (4392)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 29s
CI and images / integration (push) Successful in 2m12s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 1m43s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 19s
CI and images / backend-lint-and-test (push) Successful in 29s
CI and images / integration (push) Successful in 2m12s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 1m43s
CI and images / smoke-web (push) Successful in 55s
CI and images / promote (push) Skipped
Crop-to-source matching was held until the cheap signals could be shown insufficient. They can: of artist 8's 27 teasers with a drop inside a day, 11 go unlinked, and five are screenshot teasers with no working name at all. So it was tried, on exactly those pairs. Every teaser image correlated against every window of every nearby drop image at five scales, ground truth being the pairs the working name independently confirms, control being unrelated same-artist posts a month away. **It does not separate** — true pairs score as low as 0.401 while the control reaches 0.605, and no threshold divides them. The reason is the one the naive version was rejected for, which turns out to apply just as hard to the careful one: a single artist's work is stylistically homogeneous, so a whole-image comparison between two of their pieces is high whether or not it is the same piece. That is now written down in the module docstring with its numbers, so the next person to reach for it inherits the measurement instead of repeating it. What survived asks a narrower question the measurement shows IS answerable: not "is this a crop of that" but "is this the same image". Same pairs, same control, using the pHash FC already stores on every image — pairs the name confirms score 0, 0 and 20 bits of 256; the nearest unrelated pair in a 29-sample control scores 108. The threshold sits at 32, which is the number gallery_service already calls a near-duplicate, inside a 76-bit gap. It earns its place by being the only signal needing no cooperation from the creator: it works on a teaser called `Screenshot 2026-08-13`, and on a creator whose two platforms share no naming convention. It is quiet most of the time, because a teaser is usually a crop rather than a copy — but where it fires it is close to certain, and it recovers `Cute Selfie, Cute Dress` from the unreachable list. utils/phash warns the hash alone must not decide a MERGE, since variants of one piece collide at this distance. That does not invert here — it is the point. A merge destroys a file, so a variant colliding with its original is a loss; this asks whether two POSTS are about the same piece, and a variant of the drop's image is exactly that. Nothing is deleted either way. Gated on posts like the other two: an image on many of the creator's posts is a banner, not a piece. `_rarity` is public as `rarity` now that all three signals share it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -7,6 +7,7 @@ of what follows pins refusals, and the central one is structural rather than
|
||||
behavioural: the threshold sits above every single signal weight, which is what
|
||||
makes "time proximity alone is never sufficient" arithmetic instead of a hope.
|
||||
"""
|
||||
from collections import Counter
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import pytest
|
||||
@@ -23,11 +24,13 @@ from backend.app.models import (
|
||||
from backend.app.services.discord_grouping import DROP_GROUPER
|
||||
from backend.app.services.post_association_service import (
|
||||
DECLARED_MENTION,
|
||||
DUPLICATE_MAX_DISTANCE,
|
||||
WEIGHTS,
|
||||
PostAssociationService,
|
||||
declared_signal,
|
||||
proximity_signal,
|
||||
rescan,
|
||||
shared_image,
|
||||
weighted_score,
|
||||
)
|
||||
from backend.app.services.post_feed_service import PostFeedService
|
||||
@@ -131,13 +134,18 @@ async def _artist_with_channels(db, name: str):
|
||||
return artist, patreon, discord
|
||||
|
||||
|
||||
async def _images(db, artist, post, ext, names):
|
||||
async def _images(db, artist, post, ext, names, phashes=None):
|
||||
"""Attach named files to a post. The NAME is the point — the working-name
|
||||
signal reads it, so a test that cares about identity supplies one."""
|
||||
signal reads it, so a test that cares about identity supplies one.
|
||||
|
||||
`phashes` aligns with `names`; a test that cares about the shared-image
|
||||
signal supplies those instead (or as well).
|
||||
"""
|
||||
for i, name in enumerate(names):
|
||||
db.add(ImageRecord(
|
||||
path=f"/images/{artist.id}/{ext}_{i}_{name}.jpg",
|
||||
sha256=f"{ext}{i}{name}".ljust(64, "0")[:64],
|
||||
phash=(phashes or [None] * len(names))[i],
|
||||
size_bytes=10, mime="image/jpeg", width=10, height=10,
|
||||
origin="downloaded", primary_post_id=post.id, artist_id=artist.id,
|
||||
))
|
||||
@@ -145,19 +153,19 @@ async def _images(db, artist, post, ext, names):
|
||||
|
||||
|
||||
async def _teaser(db, artist, source, *, at, body, ext="teaser", names=None,
|
||||
title="New piece"):
|
||||
title="New piece", phashes=None):
|
||||
post = Post(
|
||||
source_id=source.id, artist_id=artist.id, external_post_id=ext,
|
||||
post_date=at, post_title=title, description=body,
|
||||
)
|
||||
db.add(post)
|
||||
await db.flush()
|
||||
await _images(db, artist, post, ext, names or [ext])
|
||||
await _images(db, artist, post, ext, names or [ext], phashes)
|
||||
return post
|
||||
|
||||
|
||||
async def _drop(db, artist, source, *, at, ext="fc-drop:1", body=None,
|
||||
names=()):
|
||||
names=(), phashes=None):
|
||||
post = Post(
|
||||
source_id=source.id, artist_id=artist.id, external_post_id=ext,
|
||||
post_date=at, description=body, synthesized_by=DROP_GROUPER,
|
||||
@@ -166,7 +174,7 @@ async def _drop(db, artist, source, *, at, ext="fc-drop:1", body=None,
|
||||
db.add(post)
|
||||
await db.flush()
|
||||
if names:
|
||||
await _images(db, artist, post, ext.replace(":", "-"), names)
|
||||
await _images(db, artist, post, ext.replace(":", "-"), names, phashes)
|
||||
return post
|
||||
|
||||
|
||||
@@ -831,3 +839,114 @@ async def test_a_drop_another_post_already_claims_is_never_taken(db):
|
||||
)
|
||||
)).scalars().all()
|
||||
assert [r.status for r in rows] == ["pending"]
|
||||
|
||||
|
||||
# --- when the drop just contains the teaser's image -------------------------
|
||||
#
|
||||
# Measured on artist 8, every teaser against every drop within a day: pairs the
|
||||
# working name independently confirms score 0, 0 and 20 bits of 256, and the
|
||||
# nearest unrelated same-artist pair in a 29-sample control scores 108. The
|
||||
# threshold sits at 32 — the same number gallery_service already calls a
|
||||
# near-duplicate — inside a 76-bit gap.
|
||||
|
||||
_PIECE = "a5" * 32 # the image
|
||||
_REENCODED = "a5" * 31 + "a4" # the same image, one bit different
|
||||
_UNRELATED = "5a" * 32 # 256 bits away — every bit differs
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_drop_carrying_the_teasers_own_image_links_it(db):
|
||||
"""The one signal that needs no cooperation from the creator. No shared
|
||||
name, nothing said about Discord, 20 hours apart — and the drop is
|
||||
carrying the same picture."""
|
||||
artist, patreon, discord = await _artist_with_channels(db, "dupartist")
|
||||
now = datetime.now(UTC)
|
||||
teaser = await _teaser(
|
||||
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
|
||||
ext="t0", names=["Alpha"], phashes=[_PIECE],
|
||||
)
|
||||
await _drop(db, artist, discord, at=now, names=["Beta"],
|
||||
phashes=[_REENCODED])
|
||||
await db.commit()
|
||||
|
||||
made = await PostAssociationService(db).match_post(
|
||||
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
|
||||
auto_link=True,
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
assert made == (1, 1)
|
||||
assoc = (await db.execute(select(PostAssociation))).scalar_one()
|
||||
assert assoc.signals["identity_image"] == 1.0
|
||||
assert "identity_token" not in assoc.signals, (
|
||||
"the names share nothing — claiming one would be a false reason"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_different_picture_links_nothing(db):
|
||||
"""The negative the threshold exists for. 256 bits apart is two different
|
||||
images, whatever else the posts have in common."""
|
||||
artist, patreon, discord = await _artist_with_channels(db, "diffartist")
|
||||
now = datetime.now(UTC)
|
||||
teaser = await _teaser(
|
||||
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
|
||||
ext="t0", names=["Alpha"], phashes=[_PIECE],
|
||||
)
|
||||
await _drop(db, artist, discord, at=now, names=["Beta"],
|
||||
phashes=[_UNRELATED])
|
||||
await db.commit()
|
||||
|
||||
made = await PostAssociationService(db).match_post(
|
||||
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
|
||||
auto_link=True,
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
assert made == (0, 0)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_image_the_creator_reuses_everywhere_links_nothing(db):
|
||||
"""The same guard the other two signals have, on the third. A banner, a
|
||||
watermark plate or a recurring title card is a habit, not a piece — and it
|
||||
would otherwise link every post carrying it to every drop carrying it."""
|
||||
artist, patreon, discord = await _artist_with_channels(db, "bannerartist")
|
||||
now = datetime.now(UTC)
|
||||
teaser = await _teaser(
|
||||
db, artist, patreon, at=now - timedelta(hours=20), body="a preview",
|
||||
ext="t0", names=["Alpha"], phashes=[_PIECE],
|
||||
)
|
||||
await _drop(db, artist, discord, at=now, names=["Beta"],
|
||||
phashes=[_REENCODED])
|
||||
for i in range(7):
|
||||
await _teaser(
|
||||
db, artist, patreon, at=now - timedelta(days=30 + i), body="older",
|
||||
ext=f"other{i}", names=[f"Gamma{i}"], phashes=[_PIECE],
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
svc = PostAssociationService(db)
|
||||
made = await svc.match_post(
|
||||
teaser.id, threshold=DEFAULT_THRESHOLD, window_hours=WINDOW,
|
||||
auto_link=True,
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
assert made == (0, 0)
|
||||
corpus = await svc._corpus(artist.id)
|
||||
assert corpus.hash_posts[int(_PIECE, 16)] == 8
|
||||
|
||||
|
||||
def test_a_missing_hash_is_not_a_match():
|
||||
"""Fails CLOSED, like every gate in utils/phash. An image whose pHash was
|
||||
never computed must read as "no evidence", never as "identical to the
|
||||
other thing that also has none"."""
|
||||
assert shared_image([], [], Counter()) == 0.0
|
||||
|
||||
|
||||
def test_the_duplicate_threshold_sits_inside_the_measured_gap():
|
||||
"""Stated as a property so the number cannot drift out of the gap that
|
||||
justifies it: 20 bits was the widest true pair, 108 the nearest unrelated
|
||||
one."""
|
||||
assert 20 < DUPLICATE_MAX_DISTANCE < 108
|
||||
|
||||
Reference in New Issue
Block a user