feat(explore): reach dial to escape dense clusters + anti-revisit (#1476)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 30s
CI / integration (push) Successful in 3m50s

The Explore walk got stuck in dense signatures — neighbours all too similar, so
forward-arrow couldn't escape and Random was the only exit. Root cause: MMR only
diversifies WITHIN the nearest ~400 pool; in a dense cluster that whole pool is
near-identical, so there's no escape route in it.

- gallery_service.similar(reach=0.0, exclude_ids=None): reach>0 widens the pool
  (cap 400→1000) and _reach_sample strides across an outward-growing distance span
  so the set handed to MMR spans near→mid-far (guaranteed escape routes), not just
  the tight cluster. exclude_ids drops already-walked images. Gallery 'more like
  this' (reach=0) is unchanged.
- api/gallery similar: parse reach + exclude_ids.
- explore store: default reach 0.4 (auto-diversifies without touching the dial),
  pass the breadcrumb as exclude_ids, setReach action.
- ExploreView: a Near↔Far reach slider in the trail.
- tests: _reach_sample math (deeper ranks with higher reach, near kept); similar
  exclude_ids drops walked + reach path runs clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-13 12:06:22 -04:00
parent af0d39ed52
commit fac5ae6ce5
6 changed files with 146 additions and 5 deletions
+13 -1
View File
@@ -148,6 +148,17 @@ async def similar():
# Explore passes exclude_wip=1 to also drop work-in-progress from the
# rabbit-hole; the gallery's own "similar" button omits it (keeps wip, #1274).
exclude_wip = request.args.get("exclude_wip") in ("1", "true", "True")
# Explore reach (#1476): 0 = nearest (gallery default), →1 reaches into farther
# distance bands so the walk can escape a dense cluster. exclude_ids = the
# breadcrumb, so already-walked images aren't re-served as neighbours.
try:
reach = max(0.0, min(1.0, float(request.args.get("reach", "0"))))
except ValueError:
reach = 0.0
exclude_ids = [
int(x) for x in request.args.get("exclude_ids", "").split(",")
if x.strip().isdigit()
] or None
# post_id is the exclusive post-detail view — not a similarity scope.
# include_hidden is a gallery-browse flag; similar() has its OWN presentation
# exclusion (a similarity-quality concern, #1274), so drop it here (#141).
@@ -158,7 +169,8 @@ async def similar():
svc = GalleryService(session)
try:
images = await svc.similar(
image_id=similar_to, limit=limit, exclude_wip=exclude_wip, **scope)
image_id=similar_to, limit=limit, exclude_wip=exclude_wip,
reach=reach, exclude_ids=exclude_ids, **scope)
except ValueError as exc:
return jsonify({"error": str(exc)}), 400
if images is None:
+35 -1
View File
@@ -396,6 +396,25 @@ def _diversify_similar(src, rows, limit, *, dup_threshold=8, lam=0.40):
return [kept[i] for i in order]
def _reach_sample(rows, limit, reach):
"""From a distance-sorted candidate pool (nearest first), pick a spread of ranks
that MIXES near (tag the current cluster) and mid-far (escape it) BEFORE dedup +
MMR — the Explore "reach" dial (#1476).
reach in (0, 1]: the sampled span grows outward from the anchor (0.25→1.0 of the
pool), evenly strided from rank 0 so the nearest are still represented. In a
dense signature the nearest ranks are near-identical, so reaching farther is the
only way to hand MMR genuinely different content — MMR alone can't escape a pool
that's already all-near. reach<=0 or a small pool passes through unchanged."""
n = len(rows)
want = max(limit * 8, 100)
if reach <= 0 or n <= want:
return rows
span = int(min(1.0, 0.25 + 0.75 * reach) * n)
idx = sorted({min(int(i * span / want), n - 1) for i in range(want)})
return [rows[i] for i in idx]
async def _artists_for(session, image_ids: list[int]) -> dict[int, dict]:
"""Map image_id -> {"name","slug"} via the canonical
image_record.artist_id (FC-2d-vii-c). Bounded by page size."""
@@ -717,6 +736,7 @@ class GalleryService:
untagged: bool = False, no_artist: bool = False,
date_from: datetime | None = None, date_to: datetime | None = None,
exclude_wip: bool = False,
reach: float = 0.0, exclude_ids: list[int] | None = None,
) -> list[GalleryImage] | None:
"""Visual "more like this": images near `image_id`'s SigLIP embedding
(pgvector, HNSW-indexed — alembic 0036), then DIVERSIFIED so the result
@@ -745,7 +765,13 @@ class GalleryService:
# wide pool there's nothing but the near-dupes to choose from. Widened
# (5×→8×, cap 200→400) so the stronger MMR has genuinely distinct
# neighbourhoods to reach into for more variance (operator, 2026-07-01).
pool_n = min(400, max(limit * 8, 100))
# Explore's reach>0 (#1476) widens it a LOT more: in a dense signature the
# nearest few hundred are all near-identical, so far-enough candidates only
# exist deeper in the ranked pool. _reach_sample then strides across them.
if reach > 0:
pool_n = min(1000, max(limit * 25, 100))
else:
pool_n = min(400, max(limit * 8, 100))
distance = ImageRecord.siglip_embedding.cosine_distance(src.siglip_embedding)
eff = _effective_date_col()
stmt = select(ImageRecord, Post.post_date, eff.label("eff"))
@@ -773,6 +799,10 @@ class GalleryService:
ImageRecord.id != image_id,
ImageRecord.id.not_in(presentation),
)
# Anti-revisit (#1476): the Explore walk passes its breadcrumb so already-
# walked images aren't re-served as neighbours — → can't loop you back in.
if exclude_ids:
stmt = stmt.where(ImageRecord.id.not_in(exclude_ids))
stmt = _apply_scope(
stmt, tag_ids=tag_ids, post_id=None,
artist_id=artist_id, media_type=media_type,
@@ -782,6 +812,10 @@ class GalleryService:
)
stmt = stmt.order_by(distance.asc()).limit(pool_n)
rows = (await self.session.execute(stmt)).all()
# Explore reach: stride across an outward-growing distance span so the pool
# handed to MMR spans near→mid-far, not just the tight cluster (#1476).
if reach > 0:
rows = _reach_sample(rows, limit, reach)
rows = _diversify_similar(src, rows, limit)
artists = await _artists_for(self.session, [r[0].id for r in rows])
return _gallery_images(rows, artists)