fix(sidecar): synthesize post_title from content first-line when title is empty (subscribestar)
SubscribeStar gallery-dl always writes `title: ""` and embeds the leading sentence inside `content` HTML. Confirmed against the operator's /mnt/Data/Patreon/Cheunart/subscribestar/ dump 2026-05-27: every post's JSON has `title: ""` and a content like `<div>Lets say hello to you guys with my Belle <br><br><br></div>`. FC's sidecar parser, treating empty strings as missing, had been leaving post_title NULL on every subscribestar post since FC-3 shipped. Fix at two layers: 1. `backend/app/utils/sidecar.py` — new `_first_line_text(body, limit)` helper strips HTML tags, collapses whitespace, returns the first non-empty line truncated to 120 chars with ellipsis. `parse_sidecar` now falls back to this when `title` resolves to None and a `content`/`description`/`caption` value is present. Patreon's non-empty titles short-circuit the fallback so existing behavior is unchanged. Four new tests in test_sidecar_util.py pin: derivation from content, truncation at 120 chars, explicit-title precedence, no-content no-fallback. 2. `alembic 0024_backfill_post_title_from_description` — backfills the same logic across existing Post rows where `post_title IS NULL OR post_title = ''` AND description is present. Idempotent (re-running is a no-op once titles are populated). Downgrade is a no-op since there's no safe way to tell derived rows from genuine ones. After deploy + migration: subscribestar posts will surface a meaningful title in PostCard, post feed search, etc.
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
"""backfill post.post_title from description first-line — 2026-05-27
|
||||
|
||||
Revision ID: 0024
|
||||
Revises: 0023
|
||||
Create Date: 2026-05-27
|
||||
|
||||
SubscribeStar gallery-dl always writes `title: ""` and embeds the leading
|
||||
sentence inside `content` HTML. FC's sidecar parser was leaving
|
||||
post_title NULL for every SubscribeStar post since FC-3 shipped. The
|
||||
parser fix (sidecar._first_line_text fallback) now synthesizes a title
|
||||
at parse time; this migration applies the same logic retroactively to
|
||||
existing rows.
|
||||
|
||||
Operator-flagged 2026-05-27 after inspecting
|
||||
/mnt/Data/Patreon/Cheunart/subscribestar/ sidecars.
|
||||
|
||||
Idempotent: only touches rows where post_title IS NULL or empty AND
|
||||
description IS NOT NULL. Re-running the migration is a no-op.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
revision: str = "0024"
|
||||
down_revision: Union[str, None] = "0023"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
_TAG_RE = re.compile(r"<[^>]+>")
|
||||
_WS_RE = re.compile(r"\s+")
|
||||
|
||||
|
||||
def _first_line_text(body: str, limit: int = 120) -> str | None:
|
||||
"""Mirror of sidecar._first_line_text. Kept inline so the migration
|
||||
doesn't carry a runtime import dependency from app code that may
|
||||
have moved by the time the migration is replayed years from now."""
|
||||
if not body:
|
||||
return None
|
||||
text_ = _TAG_RE.sub(" ", body)
|
||||
text_ = text_.replace("\xa0", " ")
|
||||
for line in text_.splitlines():
|
||||
line = _WS_RE.sub(" ", line).strip()
|
||||
if line:
|
||||
if len(line) > limit:
|
||||
return line[: limit - 1].rstrip() + "…"
|
||||
return line
|
||||
return None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
bind = op.get_bind()
|
||||
rows = bind.execute(
|
||||
text(
|
||||
"SELECT id, description FROM post "
|
||||
"WHERE (post_title IS NULL OR post_title = '') "
|
||||
"AND description IS NOT NULL AND description <> ''"
|
||||
)
|
||||
).fetchall()
|
||||
updated = 0
|
||||
for row in rows:
|
||||
derived = _first_line_text(row.description)
|
||||
if not derived:
|
||||
continue
|
||||
bind.execute(
|
||||
text("UPDATE post SET post_title = :t WHERE id = :id"),
|
||||
{"t": derived, "id": row.id},
|
||||
)
|
||||
updated += 1
|
||||
print(f"0024: backfilled post_title on {updated} row(s)")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# No safe restore — we can't tell which post_titles were derived vs
|
||||
# genuinely present. Leave the column alone on rollback.
|
||||
pass
|
||||
Reference in New Issue
Block a user