fix(explore): diversify "more like this" so it stops getting stuck (#1188)
Pure nearest-cosine piled near-identical images into the neighbour grid — a reposted banner filled all 24 slots, and once you wandered into a B&W / comic-panel cluster every neighbour was more of the same with no way back to colour without the Random button (operator-reported, with screenshot). similar() now over-fetches a wide candidate pool (5x the requested limit, cap 200), then diversifies down to `limit`: - pHash near-duplicate collapse: drop candidates within 6 Hamming bits of the anchor or an already-kept candidate, so a repost (and the anchor's own clones) appears at most once. - MMR re-rank: greedily pick for closeness-to-anchor minus similarity-to-already -picked (lambda 0.55), so the result SPANS clusters instead of returning 40 variations of one image. Falls back to nearest-order on any failure / small pool, so existing nearest-first behaviour is unchanged when there's nothing to diversify. Frontend forwardTarget drops the now-redundant skip-nearest-third hack (the list is already diversified server-side) — plain random-over-unvisited gives the variance now. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ttrj5P7upUTueSfoJcxEqa
This commit is contained in:
@@ -87,6 +87,29 @@ async def test_similar_composes_with_tag_filter(db):
|
||||
assert [i.id for i in res] == [tagged.id] # scope AND-narrows the ranked set
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_similar_collapses_near_duplicate_phashes(db):
|
||||
# The reported failure: a reposted image fills the whole neighbour grid.
|
||||
# A wall of same-pHash reposts must collapse to at most one, and the
|
||||
# genuinely distinct images must still come through.
|
||||
src = await _img(db, 1, _vec(1, 0))
|
||||
dupes = []
|
||||
for n in range(2, 7): # 5 near-identical reposts
|
||||
r = await _img(db, n, _vec(1, 0.01 * n))
|
||||
r.phash = "ffffffffffffffff" # identical perceptual hash
|
||||
dupes.append(r)
|
||||
distinct_a = await _img(db, 7, _vec(1, 1))
|
||||
distinct_a.phash = "0000000000000000"
|
||||
distinct_b = await _img(db, 8, _vec(0, 1))
|
||||
distinct_b.phash = "0f0f0f0f0f0f0f0f"
|
||||
await db.flush()
|
||||
|
||||
res = await GalleryService(db).similar(src.id, limit=10)
|
||||
ids = [i.id for i in res]
|
||||
assert sum(1 for d in dupes if d.id in ids) <= 1 # repost wall collapsed
|
||||
assert distinct_a.id in ids and distinct_b.id in ids
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_similar_respects_limit(db):
|
||||
src = await _img(db, 1, _vec(1, 0))
|
||||
|
||||
Reference in New Issue
Block a user