3 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	ba96ecfb2d	fix: the disabled sweep's shape assertion pinned the pre-E3 payload ... My own E2 test asserted `sweep`'s disabled return by exact equality, and E3 added `images_joined` to it — a rule 90 miss on a consumer I wrote an hour earlier. Every E3 test passed; this was the only failure (1 failed, 1202 passed). Fixed by extending the assertion, NOT by loosening it to a subset check. The exactness is the point: a disabled sweep reports a complete zeroed shape rather than a shorter one, so a caller can read any counter unconditionally, and this assertion is what notices when a new counter skips that path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:35:53 -04:00
bvandeusenandClaude Opus 5	1e45e2c56c	feat: an open grouping — a later drop joins its post (milestone 388 step E3) ... A synthetic post is no longer sealed at creation. A creator who adds two more variants the next day extends the existing post, its body grows with the new messages, and no rival post appears. That is what makes chat capture read as content trickling in rather than as a stream of separate arrivals. The sweep now runs two passes per source and the ORDER is load-bearing: offer new messages to still-open groups BEFORE founding new ones, because whichever runs first claims a message. E3's three named problems, each answered rather than discovered later: **Bridging.** A candidate near two groups joins NEITHER. Nearest-wins would silently make an arbitrary choice between two posts the operator may already have seen; merging them is worse still, because a merge rewrites history and anything pointing at the absorbed post dangles. Leaving it to found its own group is the recoverable failure. AMBIGUITY_MARGIN is a module constant and deliberately not a setting — it is not a quality dial anyone would tune toward a better feed, and exposing it would invite turning it to zero, which is exactly the silent arbitrary choice it prevents. **Re-surfacing without thrashing.** A grouping has two dates, and which one orders the feed is a real decision, so the feed orders by neither directly. Ordering by when the drop STARTED buries a group that grows a week later under a week of other posts — defeating the point of keeping it open. Ordering by every growth lets a group gaining one image a day live permanently at the top, so chat out-competes authored posts for the front page — the opposite of "post pacing stays front and centre". Instead `resurfaced_at` moves only when growth clears BOTH a minimum-images bar and a cooldown, so a drip-feed updates in place and a genuine second wave resurfaces exactly once. It is NULL on every ordinary post, so the sort key COALESCEs through it without moving anything that is not a grouping. **Reopening forever.** Groups close after a quiet period — artists reuse characters for years, and a group left open indefinitely will eventually absorb something it shouldn't. Openness is DERIVED, not stored: a group is open if it grew (or started) within the window. Lowering the setting closes old groups and raising it reopens them, with nothing to repair either way; a stored closed_at would have needed a sweep to set it and a repair path to ever change the policy. Rule 89 is satisfied structurally rather than by a parallel mechanism: celery_signals writes a TaskRun for every task, which already supplies duration, the 5-minute stalled-run recovery, and retention pruning. What this step owed on top of that was a wall-clock limit (present) and idempotence — re-running the joiner adds nothing, asserted directly rather than left to the unique (image, post) constraint to catch. Two bugs fixed in the writing, one of which my own test would have hit: * `assign_to_group` sorted bare (distance, Post) tuples, which falls through to comparing Posts when two distances tie — and a perfectly symmetric bridge, the exact case the function exists for, would have raised TypeError instead of declining to choose. Now keyed on the distance alone. * The cursor was still built from `post_date or downloaded_at` while the ORDER BY had gained `resurfaced_at`. Two expressions that disagree at a page boundary don't error, they silently skip or repeat rows; both sites now go through one `_post_sort_value`, and a test pages through one row at a time to prove the walk matches the whole list. Image linking is now one shared helper rather than written twice, because creation and joining would otherwise be free to drift on exactly the detail (which post owns the image) that makes a grouping reversible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:30:17 -04:00
bvandeusenandClaude Opus 5	73eeb7a377	feat: FC authors the post that Discord never wrote (milestone 388 step E2) ... Discord is a delivery channel, not a publisher. One message is not one post, and today every message lands as its own `post` row, so chat lines compete with authored work for the same surface. Rather than demote them into a second-class feed, FC now writes the post itself: one row per DROP, its images the drop's images, its body the messages' text in arrival order. Synthesising a `Post` (rather than inventing a parallel entity) is the whole point — the result is post-shaped by construction, so feed, provenance, translation, attachments and series keep working on it unchanged. The predicate is three axes ANDed, and the time one does the real work: same source AND cosine distance <= threshold AND no gap > window Similarity alone over-groups, and that is the failure that would make this useless: any two pieces of the same character by the same artist sit close in SigLIP space, so a cosine-only rule collapses a month of one character into a single "post". Two details inside the predicate are load-bearing — * distance is measured to the group's SEED, never to the previous member, because chaining lets a group DRIFT: twenty small steps walk from one piece to a completely different one, each hop individually within threshold; * the window is measured between CONSECUTIVE messages, not from the first, so an artist trickling variants out over an evening stays one drop. Why a post-import sweep and not part of ingest. The obvious alternative was to migrate Discord to the native post-first ingester (#1266) and group at capture time. That cannot work: the grouping signal is `siglip_embedding`, which is produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at capture time there is nothing to group on. Grouping is necessarily something that happens once the vectors catch up — hence a re-runnable sweep that skips what it cannot yet place, and an hourly (not daily) cadence. The honesty rule, enforced in the schema. `post.synthesized_by` names the grouper; `synthesis_details` records the members, the count, and the thresholds AS THEY WERE (they are operator-tunable, so without that "why did it group these" is unanswerable a month later). Member posts are absorbed, not destroyed — they remain the images' true origin and the audit trail — and `absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post releases its members back into the feed in one DELETE with no repair step. `post_title` stays NULL deliberately: a synthesised title is the one place this could put words in a creator's mouth. Two guards the first draft would have failed: * the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON forces its own ORDER BY, so the sort now happens outside the subquery; * a cap landing mid-drop would have published a truncated group claiming to be a whole drop, so the last group is left for the next run. And one vacuous test caught before it shipped: the support vector perturbed a single component of an all-ones vector, moving it ~1e-6, so every distance assertion passed regardless of what the predicate did. `_vec` now builds a unit vector at a stated angle, where distance is exactly 1 - cos(delta) — rule 167, a guard has to be able to fail. UI (rule 27) follows in the next commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9	2026-09-10 11:15:14 -04:00