fix(sidecar): synthesize post_title from content first-line when title is empty (subscribestar)

SubscribeStar gallery-dl always writes `title: ""` and embeds the leading
sentence inside `content` HTML. Confirmed against the operator's
/mnt/Data/Patreon/Cheunart/subscribestar/ dump 2026-05-27: every
post's JSON has `title: ""` and a content like
`<div>Lets say hello to you guys with my Belle <br><br><br></div>`.
FC's sidecar parser, treating empty strings as missing, had been leaving
post_title NULL on every subscribestar post since FC-3 shipped.

Fix at two layers:

1. `backend/app/utils/sidecar.py` — new `_first_line_text(body, limit)`
   helper strips HTML tags, collapses whitespace, returns the first
   non-empty line truncated to 120 chars with ellipsis. `parse_sidecar`
   now falls back to this when `title` resolves to None and a
   `content`/`description`/`caption` value is present. Patreon's
   non-empty titles short-circuit the fallback so existing behavior is
   unchanged. Four new tests in test_sidecar_util.py pin: derivation
   from content, truncation at 120 chars, explicit-title precedence,
   no-content no-fallback.

2. `alembic 0024_backfill_post_title_from_description` — backfills the
   same logic across existing Post rows where `post_title IS NULL OR
   post_title = ''` AND description is present. Idempotent (re-running
   is a no-op once titles are populated). Downgrade is a no-op since
   there's no safe way to tell derived rows from genuine ones.

After deploy + migration: subscribestar posts will surface a meaningful
title in PostCard, post feed search, etc.
This commit is contained in:
2026-05-27 14:44:09 -04:00
parent 4d2c464045
commit ae8c78ae09
3 changed files with 154 additions and 2 deletions
@@ -0,0 +1,80 @@
"""backfill post.post_title from description first-line — 2026-05-27
Revision ID: 0024
Revises: 0023
Create Date: 2026-05-27
SubscribeStar gallery-dl always writes `title: ""` and embeds the leading
sentence inside `content` HTML. FC's sidecar parser was leaving
post_title NULL for every SubscribeStar post since FC-3 shipped. The
parser fix (sidecar._first_line_text fallback) now synthesizes a title
at parse time; this migration applies the same logic retroactively to
existing rows.
Operator-flagged 2026-05-27 after inspecting
/mnt/Data/Patreon/Cheunart/subscribestar/ sidecars.
Idempotent: only touches rows where post_title IS NULL or empty AND
description IS NOT NULL. Re-running the migration is a no-op.
"""
from __future__ import annotations
import re
from typing import Sequence, Union
from alembic import op
from sqlalchemy import text
revision: str = "0024"
down_revision: Union[str, None] = "0023"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_TAG_RE = re.compile(r"<[^>]+>")
_WS_RE = re.compile(r"\s+")
def _first_line_text(body: str, limit: int = 120) -> str | None:
"""Mirror of sidecar._first_line_text. Kept inline so the migration
doesn't carry a runtime import dependency from app code that may
have moved by the time the migration is replayed years from now."""
if not body:
return None
text_ = _TAG_RE.sub(" ", body)
text_ = text_.replace("\xa0", " ")
for line in text_.splitlines():
line = _WS_RE.sub(" ", line).strip()
if line:
if len(line) > limit:
return line[: limit - 1].rstrip() + ""
return line
return None
def upgrade() -> None:
bind = op.get_bind()
rows = bind.execute(
text(
"SELECT id, description FROM post "
"WHERE (post_title IS NULL OR post_title = '') "
"AND description IS NOT NULL AND description <> ''"
)
).fetchall()
updated = 0
for row in rows:
derived = _first_line_text(row.description)
if not derived:
continue
bind.execute(
text("UPDATE post SET post_title = :t WHERE id = :id"),
{"t": derived, "id": row.id},
)
updated += 1
print(f"0024: backfilled post_title on {updated} row(s)")
def downgrade() -> None:
# No safe restore — we can't tell which post_titles were derived vs
# genuinely present. Leave the column alone on rollback.
pass