fix(sidecar): synthesize post_title from content first-line when title is empty (subscribestar)

SubscribeStar gallery-dl always writes `title: ""` and embeds the leading
sentence inside `content` HTML. Confirmed against the operator's
/mnt/Data/Patreon/Cheunart/subscribestar/ dump 2026-05-27: every
post's JSON has `title: ""` and a content like
`<div>Lets say hello to you guys with my Belle <br><br><br></div>`.
FC's sidecar parser, treating empty strings as missing, had been leaving
post_title NULL on every subscribestar post since FC-3 shipped.

Fix at two layers:

1. `backend/app/utils/sidecar.py` — new `_first_line_text(body, limit)`
   helper strips HTML tags, collapses whitespace, returns the first
   non-empty line truncated to 120 chars with ellipsis. `parse_sidecar`
   now falls back to this when `title` resolves to None and a
   `content`/`description`/`caption` value is present. Patreon's
   non-empty titles short-circuit the fallback so existing behavior is
   unchanged. Four new tests in test_sidecar_util.py pin: derivation
   from content, truncation at 120 chars, explicit-title precedence,
   no-content no-fallback.

2. `alembic 0024_backfill_post_title_from_description` — backfills the
   same logic across existing Post rows where `post_title IS NULL OR
   post_title = ''` AND description is present. Idempotent (re-running
   is a no-op once titles are populated). Downgrade is a no-op since
   there's no safe way to tell derived rows from genuine ones.

After deploy + migration: subscribestar posts will surface a meaningful
title in PostCard, post feed search, etc.
This commit is contained in:
2026-05-27 14:44:09 -04:00
parent 4d2c464045
commit ae8c78ae09
3 changed files with 154 additions and 2 deletions
+33
View File
@@ -110,6 +110,39 @@ def test_parse_date_epoch_and_unparseable_and_naive():
assert naive is not None and naive.utcoffset().total_seconds() == 0
def test_parse_title_derived_from_content_when_empty():
"""SubscribeStar gallery-dl writes `title: ""` and puts the leading
sentence in `content` HTML. When `title` is empty, synthesize the
post title from the content body's first non-empty text line."""
sd = parse_sidecar({
"title": "",
"content": "\n<div>Lets say hello to you guys with my Belle <br><br><br>\n</div>\n",
})
assert sd.post_title == "Lets say hello to you guys with my Belle"
assert sd.description == (
"<div>Lets say hello to you guys with my Belle <br><br><br>\n</div>"
)
def test_parse_title_derived_truncates_long_content():
long = "x" * 200
sd = parse_sidecar({"title": "", "content": long})
assert sd.post_title is not None
assert len(sd.post_title) <= 120
assert sd.post_title.endswith("")
def test_parse_title_explicit_wins_over_content_fallback():
"""If `title` is non-empty, the fallback never runs."""
sd = parse_sidecar({"title": "Real Title", "content": "<p>body line</p>"})
assert sd.post_title == "Real Title"
def test_parse_title_no_fallback_when_no_content():
sd = parse_sidecar({"title": ""})
assert sd.post_title is None
def test_parse_ignores_non_str_junk():
sd = parse_sidecar({"category": 5, "title": 7, "page_count": "x",
"id": True})