fix: a non-image file lands on the source it was downloaded for, and the undated shells it left are folded into their real posts (#4435)
CI and images / lint (push) Successful in 3s
CI and images / extension-version (push) Successful in 3s
CI and images / extension-test (push) Successful in 18s
CI and images / frontend-build (push) Successful in 22s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Successful in 2m22s
CI and images / sign-extension (push) Successful in 3s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 1m39s
CI and images / smoke-web (push) Successful in 54s
CI and images / promote (push) Successful in 1s

The attachment path looked the post's source up by (artist, platform),
taking the artist's lowest-id source. A Discord artist has one source per
channel, so every archive or pdf from a later channel became an undated
second post on the first channel's source. _post_for_sidecar now takes
the downloading source, as upsert_post_record and _apply_sidecar already
did. Migration 0114 re-points each shell's attachments to its dated twin
and deletes the shell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
2026-09-25 10:56:08 -04:00
co-authored by Claude Opus 5.5
parent e2eeb63115
commit ca802bd885
4 changed files with 253 additions and 9 deletions
@@ -0,0 +1,79 @@
"""Fold the undated shell posts a misfiled attachment created into the real post.
#4435. A non-image file (an archive, a pdf) downloaded for one of an artist's
Discord channels was filed under the artist's FIRST Discord source: the
attachment path looked the source up by (artist, platform), which takes the
lowest id. That created an undated, url-less post there holding only the
attachment, and the message's real post record then created the dated post
under the right source. The importer now uses the source it was downloading
for; this repairs the pairs it left.
A shell is folded only when all of this holds:
* it has no date and no url, and nothing synthesized it;
* another post of the same artist, platform and external id HAS a date;
* no image is linked to the shell (it held an attachment and nothing else).
Its attachments move to the dated post, dropping any the dated post already
has (same sha256, which the per-post unique forbids twice), and the shell is
deleted. A shell with no dated twin is left alone: which channel it belongs to
is not recorded anywhere but the file name. The downgrade does nothing.
Revision ID: 0114
Revises: 0113
Create Date: 2026-09-25
"""
import sqlalchemy as sa
from alembic import op
revision = "0114"
down_revision = "0113"
branch_labels = None
depends_on = None
_PAIRS = """
SELECT DISTINCT ON (shell.id) shell.id AS shell_id, real.id AS real_id
FROM post shell
JOIN source ss ON ss.id = shell.source_id
JOIN post real
ON real.artist_id = shell.artist_id
AND real.external_post_id = shell.external_post_id
AND real.id <> shell.id
AND real.post_date IS NOT NULL
JOIN source rs ON rs.id = real.source_id AND rs.platform = ss.platform
WHERE shell.post_date IS NULL
AND shell.post_url IS NULL
AND shell.synthesized_by IS NULL
AND NOT EXISTS (SELECT 1 FROM image_provenance ip WHERE ip.post_id = shell.id)
ORDER BY shell.id, real.id
"""
def fold_misfiled_attachment_posts(conn) -> int:
"""The data step, on a plain connection, so a test can run it directly.
Returns how many shells were folded."""
pairs = conn.execute(sa.text(_PAIRS)).all()
for shell_id, real_id in pairs:
conn.execute(sa.text("""
DELETE FROM post_attachment pa
WHERE pa.post_id = :shell
AND EXISTS (
SELECT 1 FROM post_attachment keep
WHERE keep.post_id = :real AND keep.sha256 = pa.sha256
)
"""), {"shell": shell_id, "real": real_id})
conn.execute(
sa.text("UPDATE post_attachment SET post_id = :real WHERE post_id = :shell"),
{"shell": shell_id, "real": real_id},
)
conn.execute(sa.text("DELETE FROM post WHERE id = :shell"), {"shell": shell_id})
return len(pairs)
def upgrade() -> None:
fold_misfiled_attachment_posts(op.get_bind())
def downgrade() -> None:
pass