feat(posts): extract + record external file-host links (Phase 3)
CI / lint (push) Failing after 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m20s

Capture off-platform links (mega/gdrive/mediafire/dropbox/pixeldrain) embedded
in post bodies so they're never silently dropped, and surface them in the post
view. The download worker (Phase 4) walks these rows.

- link_extract.py: pure extractor — <a href> + bare URLs, unwraps Patreon
  redirect shims, PRESERVES the full url incl. #fragment (mega's key), dedups.
  Reusable by every platform (runs off Post.description).
- external_link model + migration 0049: post_id/artist_id/host/url/label/status
  /attempts/last_error/attachment_id/timing; CHECK whitelists (full enum incl.
  worker statuses up front) + (post_id,url) unique.
- importer._sync_external_links: insert-missing on both import paths
  (_apply_sidecar + upsert_post_record) so a re-import never resets a link's
  status; runs for all platforms.
- post_feed_service.get_post: returns external_links (detail-only).
- PostCard: renders the links (host chip + label + status) once expanded.
- tests: extractor (5 hosts, fragment, shim unwrap, dedup), importer (record +
  no-dup on reimport), serializer.

Refs FC #830.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-14 13:15:36 -04:00
parent c342c73a25
commit d96918d777
10 changed files with 525 additions and 0 deletions
+22
View File
@@ -11,6 +11,7 @@ from sqlalchemy import select
from backend.app.models import (
Artist,
ExternalLink,
ImageProvenance,
ImageRecord,
Post,
@@ -437,6 +438,27 @@ async def test_get_post_returns_sanitized_html_body(db):
assert "<script>" not in item["description_html"]
@pytest.mark.asyncio
async def test_get_post_returns_external_links(db):
artist = await _seed_artist(db, "alice-extlinks")
src = await _seed_source(db, artist.id, "patreon", "https://p/alice-extlinks")
p = await _seed_post(
db, src.id, external_id="EXTLINKS", post_date=datetime.now(UTC),
)
db.add(ExternalLink(
post_id=p.id, artist_id=artist.id, host="mega",
url="https://mega.nz/file/x#k", label="Mega",
))
await db.commit()
item = await PostFeedService(db).get_post(p.id)
assert len(item["external_links"]) == 1
link = item["external_links"][0]
assert link["host"] == "mega"
assert link["url"] == "https://mega.nz/file/x#k"
assert link["status"] == "pending"
@pytest.mark.asyncio
async def test_get_post_unknown_returns_none(db):
item = await PostFeedService(db).get_post(99999)