fix(subscribestar): port gallery-dl date extraction (wrapped dates) + parse canary
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 20s
CI / backend-lint-and-test (push) Successful in 34s
CI / integration (push) Successful in 3m15s

Image posts wrap the post date in an <a> permalink
(<div class="post-date"><a href="/posts/ID">DATE</a></div>); text-only posts
don't. Our hand-written <div class="post-date">([^<]+)</div> regex matched ONLY
the unwrapped case, so every image post got a null published_at and sorted to the
top of the feed looking broken (cheunart 2026-06-17). Port gallery-dl's
_data_from_post method: text up to the first </, then after the last > — handles
both. Verified against the live raw feed (all 6 dates now parse).

Robust logging (operator request): _parse_posts now logs per-page parse stats
(posts / dated / with-body) and a WARNING canary when posts parse but NONE get a
date or body while the raw markers are present — i.e. our extraction diverged
from the live markup. Makes this failure class diagnosable from the worker log
alone, no authed re-fetch needed.

Test: a permalink-wrapped date parses to ISO.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 20:51:18 -04:00
parent 479b7b54da
commit 7f6345dccf
2 changed files with 48 additions and 3 deletions
+15
View File
@@ -82,6 +82,21 @@ def test_parse_ss_datetime_variants():
assert _parse_ss_datetime("nonsense") is None
def test_date_parses_when_wrapped_in_permalink_anchor():
# Image posts wrap the date in an <a> permalink; a plain regex missed them
# → null dates (cheunart 2026-06-17). gallery-dl's method handles both.
client = SubscribeStarClient(None)
wrapped = (
'<div class="post is-shown" data-id="900">'
'<div class="post-date"><a href="/posts/900">May 01, 2026 12:00 pm</a></div>'
'<div class="post-content" data-role="post_content-text">'
'<div class="trix-content"><p>x</p></div></div>'
'<div class="post-uploads for-youtube"></div></div>'
)
[p] = client._parse_posts(wrapped)
assert p["attributes"]["published_at"] == "2026-05-01T12:00:00"
# -- client: request headers (regression: live-fetch drift) -----------------
def test_request_profile_matches_gallery_dl(monkeypatch):