feat: an open grouping — a later drop joins its post (milestone 388 step E3)
CI / extension-version (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m6s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m59s
Build images / promote (push) Skipped
CI / integration (push) Failing after 2m7s
CI / extension-version (push) Successful in 3s
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m6s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m59s
Build images / promote (push) Skipped
CI / integration (push) Failing after 2m7s
A synthetic post is no longer sealed at creation. A creator who adds two more variants the next day extends the existing post, its body grows with the new messages, and no rival post appears. That is what makes chat capture read as content trickling in rather than as a stream of separate arrivals. The sweep now runs two passes per source and the ORDER is load-bearing: offer new messages to still-open groups BEFORE founding new ones, because whichever runs first claims a message. E3's three named problems, each answered rather than discovered later: **Bridging.** A candidate near two groups joins NEITHER. Nearest-wins would silently make an arbitrary choice between two posts the operator may already have seen; merging them is worse still, because a merge rewrites history and anything pointing at the absorbed post dangles. Leaving it to found its own group is the recoverable failure. AMBIGUITY_MARGIN is a module constant and deliberately not a setting — it is not a quality dial anyone would tune toward a better feed, and exposing it would invite turning it to zero, which is exactly the silent arbitrary choice it prevents. **Re-surfacing without thrashing.** A grouping has two dates, and which one orders the feed is a real decision, so the feed orders by neither directly. Ordering by when the drop STARTED buries a group that grows a week later under a week of other posts — defeating the point of keeping it open. Ordering by every growth lets a group gaining one image a day live permanently at the top, so chat out-competes authored posts for the front page — the opposite of "post pacing stays front and centre". Instead `resurfaced_at` moves only when growth clears BOTH a minimum-images bar and a cooldown, so a drip-feed updates in place and a genuine second wave resurfaces exactly once. It is NULL on every ordinary post, so the sort key COALESCEs through it without moving anything that is not a grouping. **Reopening forever.** Groups close after a quiet period — artists reuse characters for years, and a group left open indefinitely will eventually absorb something it shouldn't. Openness is DERIVED, not stored: a group is open if it grew (or started) within the window. Lowering the setting closes old groups and raising it reopens them, with nothing to repair either way; a stored closed_at would have needed a sweep to set it and a repair path to ever change the policy. Rule 89 is satisfied structurally rather than by a parallel mechanism: celery_signals writes a TaskRun for every task, which already supplies duration, the 5-minute stalled-run recovery, and retention pruning. What this step owed on top of that was a wall-clock limit (present) and idempotence — re-running the joiner adds nothing, asserted directly rather than left to the unique (image, post) constraint to catch. Two bugs fixed in the writing, one of which my own test would have hit: * `assign_to_group` sorted bare (distance, Post) tuples, which falls through to comparing Posts when two distances tie — and a perfectly symmetric bridge, the exact case the function exists for, would have raised TypeError instead of declining to choose. Now keyed on the distance alone. * The cursor was still built from `post_date or downloaded_at` while the ORDER BY had gained `resurfaced_at`. Two expressions that disagree at a page boundary don't error, they silently skip or repeat rows; both sites now go through one `_post_sort_value`, and a test pages through one row at a time to prove the walk matches the whole list. Image linking is now one shared helper rather than written twice, because creation and joining would otherwise be free to drift on exactly the detail (which post owns the image) that makes a grouping reversible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -41,8 +41,40 @@ THUMBNAIL_LIMIT = 6
|
||||
|
||||
|
||||
def _sort_key():
|
||||
"""Postgres COALESCE expression used in ORDER BY and WHERE clauses."""
|
||||
return func.coalesce(Post.post_date, Post.downloaded_at)
|
||||
"""Postgres COALESCE expression used in ORDER BY and WHERE clauses.
|
||||
|
||||
`resurfaced_at` leads (milestone 388 E3). A synthetic post stays OPEN — a
|
||||
creator who adds variants the next day extends the existing post — so such
|
||||
a post has two dates, and which one orders the feed is a real decision:
|
||||
|
||||
* ordering by when the drop STARTED buries a group that grows a week later
|
||||
under a week of other posts, so the operator never sees the new content —
|
||||
which defeats keeping the group open at all;
|
||||
* ordering by every growth lets a group that gains one image a day sit
|
||||
permanently at the top, so chat out-competes authored posts for the front
|
||||
page — the opposite of "post pacing stays front and centre".
|
||||
|
||||
So the feed orders by neither directly. `resurfaced_at` moves only when the
|
||||
anti-thrash rule fires (discord_grouping.should_resurface: enough new
|
||||
images AND enough time since the last move), which means a drip-feed
|
||||
updates IN PLACE and a genuine second wave resurfaces exactly once.
|
||||
|
||||
It is NULL on every ordinary post, so this COALESCE cannot move anything
|
||||
that is not a grouping. Used identically in ORDER BY and in the cursor's
|
||||
WHERE, which is what keeps pagination stable across the change.
|
||||
"""
|
||||
return func.coalesce(Post.resurfaced_at, Post.post_date, Post.downloaded_at)
|
||||
|
||||
|
||||
def _post_sort_value(post: Post):
|
||||
"""The Python twin of `_sort_key()`, for building a cursor from a loaded row.
|
||||
|
||||
Kept next to it on purpose: these two are one expression in two languages,
|
||||
and the failure when they disagree is not an error but a quiet one — rows
|
||||
skipped or repeated at page boundaries, which reads as a backend bug
|
||||
anywhere but here.
|
||||
"""
|
||||
return post.resurfaced_at or post.post_date or post.downloaded_at
|
||||
|
||||
|
||||
class PostFeedService:
|
||||
@@ -134,7 +166,10 @@ class PostFeedService:
|
||||
# Far edge in the travel direction: oldest row going older,
|
||||
# newest row going newer (rows is descending for display).
|
||||
edge_post = rows[-1][0] if direction == "older" else rows[0][0]
|
||||
edge_key = edge_post.post_date or edge_post.downloaded_at
|
||||
# Must match _sort_key() exactly, including resurfaced_at's
|
||||
# precedence: a cursor built from a different expression than the
|
||||
# ORDER BY silently skips or repeats rows at every page boundary.
|
||||
edge_key = _post_sort_value(edge_post)
|
||||
next_cursor = encode_cursor(edge_key, edge_post.id)
|
||||
|
||||
post_ids = [p.id for p, _, _ in rows]
|
||||
@@ -168,7 +203,7 @@ class PostFeedService:
|
||||
if anchor is None:
|
||||
return None
|
||||
anchor_post, anchor_artist, anchor_source = anchor
|
||||
anchor_key = anchor_post.post_date or anchor_post.downloaded_at
|
||||
anchor_key = _post_sort_value(anchor_post)
|
||||
anchor_cursor = encode_cursor(anchor_key, anchor_post.id)
|
||||
|
||||
older = await self.scroll(
|
||||
@@ -414,6 +449,10 @@ class PostFeedService:
|
||||
# keys are always present so the frontend never branches on absence.
|
||||
"synthesized_by": post.synthesized_by,
|
||||
"synthesis": post.synthesis_details,
|
||||
# #388 E3. A grouping stays open, so the card can say "updated N
|
||||
# ago" — which is the whole signal that chat content is trickling
|
||||
# in. NULL means it has not grown since it was created.
|
||||
"last_grew_at": post.last_grew_at.isoformat() if post.last_grew_at else None,
|
||||
# Non-null on a chat message a synthetic post absorbed. The feed
|
||||
# filters these out, but `around`/`get_post` still reach them, and
|
||||
# the UI uses this to explain why a post it linked to is not in the
|
||||
|
||||
Reference in New Issue
Block a user