fix(subscribestar): port gallery-dl date extraction (wrapped dates) + parse canary
Image posts wrap the post date in an <a> permalink (<div class="post-date"><a href="/posts/ID">DATE</a></div>); text-only posts don't. Our hand-written <div class="post-date">([^<]+)</div> regex matched ONLY the unwrapped case, so every image post got a null published_at and sorted to the top of the feed looking broken (cheunart 2026-06-17). Port gallery-dl's _data_from_post method: text up to the first </, then after the last > — handles both. Verified against the live raw feed (all 6 dates now parse). Robust logging (operator request): _parse_posts now logs per-page parse stats (posts / dated / with-body) and a WARNING canary when posts parse but NONE get a date or body while the raw markers are present — i.e. our extraction diverged from the live markup. Makes this failure class diagnosable from the worker log alone, no authed re-fetch needed. Test: a permalink-wrapped date parses to ISO. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -82,6 +82,21 @@ def test_parse_ss_datetime_variants():
|
||||
assert _parse_ss_datetime("nonsense") is None
|
||||
|
||||
|
||||
def test_date_parses_when_wrapped_in_permalink_anchor():
|
||||
# Image posts wrap the date in an <a> permalink; a plain regex missed them
|
||||
# → null dates (cheunart 2026-06-17). gallery-dl's method handles both.
|
||||
client = SubscribeStarClient(None)
|
||||
wrapped = (
|
||||
'<div class="post is-shown" data-id="900">'
|
||||
'<div class="post-date"><a href="/posts/900">May 01, 2026 12:00 pm</a></div>'
|
||||
'<div class="post-content" data-role="post_content-text">'
|
||||
'<div class="trix-content"><p>x</p></div></div>'
|
||||
'<div class="post-uploads for-youtube"></div></div>'
|
||||
)
|
||||
[p] = client._parse_posts(wrapped)
|
||||
assert p["attributes"]["published_at"] == "2026-05-01T12:00:00"
|
||||
|
||||
|
||||
# -- client: request headers (regression: live-fetch drift) -----------------
|
||||
|
||||
def test_request_profile_matches_gallery_dl(monkeypatch):
|
||||
|
||||
Reference in New Issue
Block a user