feat: FC authors the post that Discord never wrote (milestone 388 step E2)
CI / lint (push) Failing after 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m5s
CI / lint (push) Failing after 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
CI / frontend-build (push) Successful in 25s
CI / backend-lint-and-test (push) Successful in 31s
Build images / build-web (push) Successful in 1m5s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m53s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m5s
Discord is a delivery channel, not a publisher. One message is not one post,
and today every message lands as its own `post` row, so chat lines compete
with authored work for the same surface. Rather than demote them into a
second-class feed, FC now writes the post itself: one row per DROP, its
images the drop's images, its body the messages' text in arrival order.
Synthesising a `Post` (rather than inventing a parallel entity) is the whole
point — the result is post-shaped by construction, so feed, provenance,
translation, attachments and series keep working on it unchanged.
The predicate is three axes ANDed, and the time one does the real work:
same source AND cosine distance <= threshold AND no gap > window
Similarity alone over-groups, and that is the failure that would make this
useless: any two pieces of the same character by the same artist sit close in
SigLIP space, so a cosine-only rule collapses a month of one character into a
single "post". Two details inside the predicate are load-bearing —
* distance is measured to the group's SEED, never to the previous member,
because chaining lets a group DRIFT: twenty small steps walk from one piece
to a completely different one, each hop individually within threshold;
* the window is measured between CONSECUTIVE messages, not from the first, so
an artist trickling variants out over an evening stays one drop.
Why a post-import sweep and not part of ingest. The obvious alternative was to
migrate Discord to the native post-first ingester (#1266) and group at capture
time. That cannot work: the grouping signal is `siglip_embedding`, which is
produced asynchronously AFTER import (tasks/ml.py, the GPU backfill), so at
capture time there is nothing to group on. Grouping is necessarily something
that happens once the vectors catch up — hence a re-runnable sweep that skips
what it cannot yet place, and an hourly (not daily) cadence.
The honesty rule, enforced in the schema. `post.synthesized_by` names the
grouper; `synthesis_details` records the members, the count, and the
thresholds AS THEY WERE (they are operator-tunable, so without that "why did
it group these" is unanswerable a month later). Member posts are absorbed, not
destroyed — they remain the images' true origin and the audit trail — and
`absorbed_by_post_id` is ON DELETE SET NULL, so deleting a synthetic post
releases its members back into the feed in one DELETE with no repair step.
`post_title` stays NULL deliberately: a synthesised title is the one place
this could put words in a creator's mouth.
Two guards the first draft would have failed:
* the per-run cap took the lowest post IDs, not the oldest posts — DISTINCT ON
forces its own ORDER BY, so the sort now happens outside the subquery;
* a cap landing mid-drop would have published a truncated group claiming to be
a whole drop, so the last group is left for the next run.
And one vacuous test caught before it shipped: the support vector perturbed a
single component of an all-ones vector, moving it ~1e-6, so every distance
assertion passed regardless of what the predicate did. `_vec` now builds a
unit vector at a stated angle, where distance is exactly 1 - cos(delta) —
rule 167, a guard has to be able to fail.
UI (rule 27) follows in the next commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -48,6 +48,12 @@ _EDITABLE = (
|
||||
"process_conflict_threshold",
|
||||
"embedder_model_name",
|
||||
"embedder_model_version",
|
||||
# Discord drop grouping (#388 E2). Operator-facing because the quality bar
|
||||
# is a judgement no test can settle: too greedy merges distinct pieces, too
|
||||
# shy leaves a drop scattered.
|
||||
"discord_grouping_enabled",
|
||||
"discord_group_max_distance",
|
||||
"discord_group_window_minutes",
|
||||
*_DETECTOR_FIELDS,
|
||||
)
|
||||
|
||||
@@ -148,6 +154,14 @@ def _validate(p: dict) -> str | None:
|
||||
return f"process_auto_apply_threshold must be between {AUTO_APPLY_THRESHOLD_MIN} and {AUTO_APPLY_THRESHOLD_MAX}"
|
||||
if not (0.0 <= float(p["process_conflict_threshold"]) <= 1.0):
|
||||
return "process_conflict_threshold must be between 0 and 1"
|
||||
# Discord drop grouping (#388 E2). max_distance is a cosine DISTANCE, so
|
||||
# unlike the *_threshold family above it is not on the auto-apply scale:
|
||||
# 0 is identical and 1 is unrelated, and both ends are legal. The upper
|
||||
# bound is 1.0 rather than AUTO_APPLY_THRESHOLD_MAX for that reason.
|
||||
if not (0.0 <= float(p["discord_group_max_distance"]) <= 1.0):
|
||||
return "discord_group_max_distance must be between 0 and 1"
|
||||
if float(p["discord_group_window_minutes"]) <= 0:
|
||||
return "discord_group_window_minutes must be > 0"
|
||||
# Embedder model swap (#1190): both must be non-empty. Changing them means a
|
||||
# different embedding space — the operator must re-embed + retrain after.
|
||||
for key in ("embedder_model_name", "embedder_model_version"):
|
||||
|
||||
Reference in New Issue
Block a user