feat: FC authors the post that Discord never wrote (milestone 388 step E2)
CI / lint (push) Failing after 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m5s

Discord is a delivery channel, not a publisher. One message is not one post,
and today every message lands as its own `post` row, so chat lines compete
with authored work for the same surface. Rather than demote them into a
second-class feed, FC now writes the post itself: one row per DROP, its
images the drop's images, its body the messages' text in arrival order.

Synthesising a `Post` (rather than inventing a parallel entity) is the whole
point — the result is post-shaped by construction, so feed, provenance,
translation, attachments and series keep working on it unchanged.

The predicate is three axes ANDed, and the time one does the real work:

    same source  AND  cosine distance <= threshold  AND  no gap > window

Similarity alone over-groups, and that is the failure that would make this
useless: any two pieces of the same character by the same artist sit close in
SigLIP space, so a cosine-only rule collapses a month of one character into a
single "post". Two details inside the predicate are load-bearing —

* distance is measured to the group's SEED, never to the previous member,
  because chaining lets a group DRIFT: twenty small steps walk from one piece
  to a completely different one, each hop individually within threshold;
* the window is measured between CONSECUTIVE messages, not from the first, so
  an artist trickling variants out over an evening stays one drop.

Why a post-import sweep and not part of ingest. The obvious alternative was to
migrate Discord to the native post-first ingester (#1266) and group at capture
time. That cannot work: the grouping signal is `siglip_embedding`, which is
produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at
capture time there is nothing to group on. Grouping is necessarily something
that happens once the vectors catch up — hence a re-runnable sweep that skips
what it cannot yet place, and an hourly (not daily) cadence.

The honesty rule, enforced in the schema. `post.synthesized_by` names the
grouper; `synthesis_details` records the members, the count, and the
thresholds AS THEY WERE (they are operator-tunable, so without that "why did
it group these" is unanswerable a month later). Member posts are absorbed, not
destroyed — they remain the images' true origin and the audit trail — and
`absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post
releases its members back into the feed in one DELETE with no repair step.
`post_title` stays NULL deliberately: a synthesised title is the one place
this could put words in a creator's mouth.

Two guards the first draft would have failed:

* the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON
  forces its own ORDER BY, so the sort now happens outside the subquery;
* a cap landing mid-drop would have published a truncated group claiming to be
  a whole drop, so the last group is left for the next run.

And one vacuous test caught before it shipped: the support vector perturbed a
single component of an all-ones vector, moving it ~1e-6, so every distance
assertion passed regardless of what the predicate did. `_vec` now builds a
unit vector at a stated angle, where distance is exactly 1 - cos(delta) —
rule 167, a guard has to be able to fail.

UI (rule 27) follows in the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
2026-09-10 11:15:14 -04:00
co-authored by Claude Opus 5
parent 4fe792b61c
commit 73eeb7a377
9 changed files with 983 additions and 0 deletions
+34
View File
@@ -252,6 +252,40 @@ class MLSettings(Base):
Integer, nullable=False, default=64,
server_default="64",
)
# -- Discord drop grouping (milestone 388) -----------------------------
# FC authors a post out of a creator's variant drop. The predicate is three
# axes ANDed together, and the time one does the real work: SIMILARITY
# ALONE OVER-GROUPS. Any two pieces of the same character by the same
# artist sit close in SigLIP space, so a cosine-only rule collapses a month
# of one character into a single "post". What makes a variant set a set is
# that it was dropped TOGETHER.
discord_grouping_enabled: Mapped[bool] = mapped_column(
# ON by default, matching the operator's standing opt-OUT preference for
# automatic behaviour (2026-06-29, recorded on the head/ccip auto-apply
# switches). Safe to default on because the act is reversible by one
# DELETE: removing a synthetic post un-absorbs its members.
Boolean, nullable=False, default=True,
server_default="true",
)
# Cosine DISTANCE, not similarity — this is the units gallery_service's
# `cosine_distance` already speaks, and converting at the query site is a
# step to get backwards. Lower = stricter. 0.10 is deliberately TIGHT: the
# two failure modes are not symmetric. Grouping too shy leaves a drop
# scattered, which is visible and fixable by raising this; grouping too
# greedy merges distinct pieces into a post that claims they belong
# together, which is the failure that would discredit the feature.
discord_group_max_distance: Mapped[float] = mapped_column(
Float, nullable=False, default=0.10,
server_default=text("0.10"),
)
# The gap that ENDS a drop, measured between CONSECUTIVE messages rather
# than from the first — an artist trickling variants out over an evening is
# one drop, and a window anchored on the first message would cut it in half
# at an arbitrary point.
discord_group_window_minutes: Mapped[float] = mapped_column(
Float, nullable=False, default=60.0,
server_default=text("60"),
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now()
)
+34
View File
@@ -102,3 +102,37 @@ class Post(Base):
downloaded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now()
)
# -- Synthetic posts (milestone 388). ----------------------------------
# Discord is a delivery CHANNEL, not a publisher: one message is not one
# post. So FC authors the post itself, grouping a creator's variant drop
# into a single row (services/discord_grouping.py).
#
# NULL for every post a creator actually wrote — which is all of them until
# a grouper runs. Non-NULL names the grouper that authored this row, and is
# the ONE flag the UI keys off to say so. The honesty rule is the whole
# point: a synthetic post must never present itself as authored, and a
# column that is absent-or-a-name makes "was this us?" answerable from the
# row rather than inferred from its shape.
#
# Plain String, no CHECK (rule 36 considered and declined) — same reasoning
# as source.error_type and service_seen.kind. There is exactly one grouper
# today; a second would be a value, not an invariant.
synthesized_by: Mapped[str | None] = mapped_column(String(32), nullable=True)
# What it was built from, so the operator can audit a grouping FC invented:
# member post ids, message count, and the thresholds in force when the
# decision was made. That last part matters — the thresholds are operator-
# tunable, so "why did it group these" is unanswerable a month later
# without recording the values that produced it.
synthesis_details: Mapped[dict | None] = mapped_column(JSON, nullable=True)
# Set on a MEMBER post, pointing at the synthetic post that absorbed it.
# The feed hides absorbed posts (they are the chat lines the synthetic post
# replaced); every other surface still reaches them by id, because they
# remain the image's true origin and the grouping has to be inspectable.
#
# Self-FK, ON DELETE SET NULL: deleting a synthetic post un-absorbs its
# members and they return to the feed on their own. That is the reversal
# path, and it is one DELETE — nothing to undo by hand.
absorbed_by_post_id: Mapped[int | None] = mapped_column(
ForeignKey("post.id", ondelete="SET NULL"), nullable=True, index=True
)