fix(showcase): over-sample + random-order to break near-dup clustering — #699
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 26s
CI / integration (push) Failing after 3m0s

TABLESAMPLE SYSTEM_ROWS reads CONTIGUOUS rows from each sampled page, so
sequentially-imported near-duplicates (multi-image posts, variant sets) came
back adjacent and clustered in the showcase ("three near-identical in a row").
Sample limit*5 rows (spanning more pages) then ORDER BY random() before taking
limit — breaks the physical adjacency for much better spread, still cheap
(random() over a few hundred rows, not the whole table).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 15:44:04 -04:00
parent 2b69540ecc
commit 89dfa42e18
+12 -3
View File
@@ -20,11 +20,20 @@ class ShowcaseService:
async def random_sample(self, limit: int = 60) -> list[dict]:
if limit < 1 or limit > 200:
raise ValueError("limit must be between 1 and 200")
# Over-sample then random-order (#699): SYSTEM_ROWS reads CONTIGUOUS rows
# from each sampled page, so sequentially-imported near-duplicates
# (multi-image posts, variant sets) come back adjacent and cluster in the
# showcase ("three near-identical in a row"). Sampling a multiple of
# `limit` spans more pages, and ORDER BY random() before taking `limit`
# breaks the physical adjacency — far better spread, still cheap
# (random() over a few hundred rows, not the whole table).
oversample = min(limit * 5, 1000)
stmt = select(ImageRecord).from_statement(
text(
"SELECT * FROM image_record "
"TABLESAMPLE SYSTEM_ROWS(:n)"
).bindparams(n=limit)
"SELECT * FROM ("
" SELECT * FROM image_record TABLESAMPLE SYSTEM_ROWS(:o)"
") sub ORDER BY random() LIMIT :n"
).bindparams(o=oversample, n=limit)
)
rows = (await self.session.execute(stmt)).scalars().all()
return [