bvandeusenandClaude Opus 5
73eeb7a377
CI / lint (push) Failing after 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m5s
Discord is a delivery channel, not a publisher. One message is not one post,
and today every message lands as its own `post` row, so chat lines compete
with authored work for the same surface. Rather than demote them into a
second-class feed, FC now writes the post itself: one row per DROP, its
images the drop's images, its body the messages' text in arrival order.
Synthesising a `Post` (rather than inventing a parallel entity) is the whole
point — the result is post-shaped by construction, so feed, provenance,
translation, attachments and series keep working on it unchanged.
The predicate is three axes ANDed, and the time one does the real work:
same source AND cosine distance <= threshold AND no gap > window
Similarity alone over-groups, and that is the failure that would make this
useless: any two pieces of the same character by the same artist sit close in
SigLIP space, so a cosine-only rule collapses a month of one character into a
single "post". Two details inside the predicate are load-bearing —
* distance is measured to the group's SEED, never to the previous member,
because chaining lets a group DRIFT: twenty small steps walk from one piece
to a completely different one, each hop individually within threshold;
* the window is measured between CONSECUTIVE messages, not from the first, so
an artist trickling variants out over an evening stays one drop.
Why a post-import sweep and not part of ingest. The obvious alternative was to
migrate Discord to the native post-first ingester (#1266) and group at capture
time. That cannot work: the grouping signal is `siglip_embedding`, which is
produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at
capture time there is nothing to group on. Grouping is necessarily something
that happens once the vectors catch up — hence a re-runnable sweep that skips
what it cannot yet place, and an hourly (not daily) cadence.
The honesty rule, enforced in the schema. `post.synthesized_by` names the
grouper; `synthesis_details` records the members, the count, and the
thresholds AS THEY WERE (they are operator-tunable, so without that "why did
it group these" is unanswerable a month later). Member posts are absorbed, not
destroyed — they remain the images' true origin and the audit trail — and
`absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post
releases its members back into the feed in one DELETE with no repair step.
`post_title` stays NULL deliberately: a synthesised title is the one place
this could put words in a creator's mouth.
Two guards the first draft would have failed:
* the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON
forces its own ORDER BY, so the sort now happens outside the subquery;
* a cap landing mid-drop would have published a truncated group claiming to be
a whole drop, so the last group is left for the next run.
And one vacuous test caught before it shipped: the support vector perturbed a
single component of an all-ones vector, moving it ~1e-6, so every distance
assertion passed regardless of what the predicate did. `_vec` now builds a
unit vector at a stated angle, where distance is exactly 1 - cos(delta) —
rule 167, a guard has to be able to fail.
UI (rule 27) follows in the next commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
93 lines
3.6 KiB
Python
93 lines
3.6 KiB
Python
"""Synthetic posts — FC authors a post for content that arrived as chat.
|
|
|
|
Milestone 388, step E2. Discord is a delivery channel, not a publisher: one
|
|
message is not one post, and today every message becomes its own `post` row
|
|
competing with authored work for the same surface. This adds the three columns
|
|
that let FC group a creator's variant drop into a post it wrote itself, while
|
|
keeping that fact visible and the grouping reversible.
|
|
|
|
## Why a flag and a back-pointer rather than a separate table
|
|
|
|
A synthetic post has to BE a post — same row, same columns — or every existing
|
|
surface (feed, provenance, translation, attachments, series) would need a
|
|
second code path for it. `synthesized_by` marks the ones FC authored;
|
|
`absorbed_by_post_id` points a member message-post at the post that replaced
|
|
it in the feed. The members are not deleted: they remain the images' true
|
|
origin, and destroying them would make the grouping un-auditable at exactly
|
|
the moment somebody wants to check it.
|
|
|
|
Reversal is one DELETE. `absorbed_by_post_id` is ON DELETE SET NULL, so
|
|
removing a synthetic post releases its members and they return to the feed
|
|
unaided.
|
|
|
|
Revision ID: 0092
|
|
Revises: 0091
|
|
Create Date: 2026-09-10
|
|
|
|
"""
|
|
from typing import Sequence, Union
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision: str = "0092"
|
|
down_revision: Union[str, None] = "0091"
|
|
branch_labels: Union[str, Sequence[str], None] = None
|
|
depends_on: Union[str, Sequence[str], None] = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
# No CHECK on synthesized_by (rule 36 considered and declined): there is one
|
|
# grouper today and a second would be a new VALUE, not a new invariant —
|
|
# matching source.error_type and service_seen.kind.
|
|
op.add_column("post", sa.Column("synthesized_by", sa.String(length=32), nullable=True))
|
|
op.add_column("post", sa.Column("synthesis_details", sa.JSON(), nullable=True))
|
|
op.add_column(
|
|
"post", sa.Column("absorbed_by_post_id", sa.Integer(), nullable=True),
|
|
)
|
|
op.create_index(
|
|
op.f("ix_post_absorbed_by_post_id"), "post", ["absorbed_by_post_id"],
|
|
)
|
|
# SET NULL, not CASCADE: deleting the synthetic post must RELEASE its
|
|
# members, never take them with it. The members are the real capture.
|
|
op.create_foreign_key(
|
|
"fk_post_absorbed_by_post_id_post", "post", "post",
|
|
["absorbed_by_post_id"], ["id"], ondelete="SET NULL",
|
|
)
|
|
|
|
# Grouping tunables. Every one of these is operator-facing (project rule
|
|
# 25) because the quality bar here is a judgement call no test can settle:
|
|
# too greedy merges distinct pieces, too shy leaves a drop scattered.
|
|
op.add_column(
|
|
"ml_settings",
|
|
sa.Column(
|
|
"discord_grouping_enabled", sa.Boolean(),
|
|
server_default="true", nullable=False,
|
|
),
|
|
)
|
|
op.add_column(
|
|
"ml_settings",
|
|
sa.Column(
|
|
"discord_group_max_distance", sa.Float(),
|
|
server_default=sa.text("0.10"), nullable=False,
|
|
),
|
|
)
|
|
op.add_column(
|
|
"ml_settings",
|
|
sa.Column(
|
|
"discord_group_window_minutes", sa.Float(),
|
|
server_default=sa.text("60"), nullable=False,
|
|
),
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("ml_settings", "discord_group_window_minutes")
|
|
op.drop_column("ml_settings", "discord_group_max_distance")
|
|
op.drop_column("ml_settings", "discord_grouping_enabled")
|
|
op.drop_constraint("fk_post_absorbed_by_post_id_post", "post", type_="foreignkey")
|
|
op.drop_index(op.f("ix_post_absorbed_by_post_id"), table_name="post")
|
|
op.drop_column("post", "absorbed_by_post_id")
|
|
op.drop_column("post", "synthesis_details")
|
|
op.drop_column("post", "synthesized_by")
|