fix(sidecar): synthesize post_title from content first-line when title is empty (subscribestar)
SubscribeStar gallery-dl always writes `title: ""` and embeds the leading sentence inside `content` HTML. Confirmed against the operator's /mnt/Data/Patreon/Cheunart/subscribestar/ dump 2026-05-27: every post's JSON has `title: ""` and a content like `<div>Lets say hello to you guys with my Belle <br><br><br></div>`. FC's sidecar parser, treating empty strings as missing, had been leaving post_title NULL on every subscribestar post since FC-3 shipped. Fix at two layers: 1. `backend/app/utils/sidecar.py` — new `_first_line_text(body, limit)` helper strips HTML tags, collapses whitespace, returns the first non-empty line truncated to 120 chars with ellipsis. `parse_sidecar` now falls back to this when `title` resolves to None and a `content`/`description`/`caption` value is present. Patreon's non-empty titles short-circuit the fallback so existing behavior is unchanged. Four new tests in test_sidecar_util.py pin: derivation from content, truncation at 120 chars, explicit-title precedence, no-content no-fallback. 2. `alembic 0024_backfill_post_title_from_description` — backfills the same logic across existing Post rows where `post_title IS NULL OR post_title = ''` AND description is present. Idempotent (re-running is a no-op once titles are populated). Downgrade is a no-op since there's no safe way to tell derived rows from genuine ones. After deploy + migration: subscribestar posts will surface a meaningful title in PostCard, post feed search, etc.
This commit is contained in:
@@ -110,6 +110,39 @@ def test_parse_date_epoch_and_unparseable_and_naive():
|
||||
assert naive is not None and naive.utcoffset().total_seconds() == 0
|
||||
|
||||
|
||||
def test_parse_title_derived_from_content_when_empty():
|
||||
"""SubscribeStar gallery-dl writes `title: ""` and puts the leading
|
||||
sentence in `content` HTML. When `title` is empty, synthesize the
|
||||
post title from the content body's first non-empty text line."""
|
||||
sd = parse_sidecar({
|
||||
"title": "",
|
||||
"content": "\n<div>Lets say hello to you guys with my Belle <br><br><br>\n</div>\n",
|
||||
})
|
||||
assert sd.post_title == "Lets say hello to you guys with my Belle"
|
||||
assert sd.description == (
|
||||
"<div>Lets say hello to you guys with my Belle <br><br><br>\n</div>"
|
||||
)
|
||||
|
||||
|
||||
def test_parse_title_derived_truncates_long_content():
|
||||
long = "x" * 200
|
||||
sd = parse_sidecar({"title": "", "content": long})
|
||||
assert sd.post_title is not None
|
||||
assert len(sd.post_title) <= 120
|
||||
assert sd.post_title.endswith("…")
|
||||
|
||||
|
||||
def test_parse_title_explicit_wins_over_content_fallback():
|
||||
"""If `title` is non-empty, the fallback never runs."""
|
||||
sd = parse_sidecar({"title": "Real Title", "content": "<p>body line</p>"})
|
||||
assert sd.post_title == "Real Title"
|
||||
|
||||
|
||||
def test_parse_title_no_fallback_when_no_content():
|
||||
sd = parse_sidecar({"title": ""})
|
||||
assert sd.post_title is None
|
||||
|
||||
|
||||
def test_parse_ignores_non_str_junk():
|
||||
sd = parse_sidecar({"category": 5, "title": 7, "page_count": "x",
|
||||
"id": True})
|
||||
|
||||
Reference in New Issue
Block a user