Commit Graph
3 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 ddf896078c refactor(platforms): retire deviantart end-to-end (#3069)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 23s
extension / lint (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m43s
Executes the 2026-07-05 product decision (FC downloaders = art-dedicated
services only), which removed Twitter/X and Bluesky but left deviantart
fully wired for seven weeks — the half-retired state rule 22 exists to
prevent.

Removed: the PlatformInfo module and its registry entry, the gallery-dl
extractor block, extension_service's artist-page pattern, the extension's
PLATFORMS + PLATFORM_ARTIST_PATTERNS entries, its manifest host permission
and content-script match, the frontend icon/colour/label, and the operator-
facing "supported platforms" list that still advertised it.

Two judgment calls, both recorded in migration 0088:

  * existing `source` rows are DISABLED, not deleted. The row is the only
    record of the artist's DeviantArt URL. Disabling is also required for
    correctness rather than tidiness: with the platform unregistered the
    download path falls through to gallery-dl, which carries its OWN
    deviantart extractor, so an enabled row would have kept downloading
    from a dropped platform.
  * the `credential` row IS deleted — a live session cookie for a site FC
    will never call again.

Adds the invariant whose absence is why manifest.json drifted in the first
place: nothing tied its domain lists back to the platform table. The
extension suite now asserts both directions, plus that no host permission
belongs to an unclaimed domain (`*://*/*` exempted — FC is self-hosted at
an operator-chosen URL the extension cannot enumerate).

Extension version 1.0.10 -> 1.0.11: ci.yml's guard hard-fails a packaged
extension change without a bump. No release is cut — build.yml's
sign-extension job only runs on main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 07:24:08 -04:00
bvandeusenandClaude Opus 4.8 d8d8ecd78f feat(subscribestar): flip dispatch to the native ingester (#893, Step 5)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Failing after 3m18s
SubscribeStar now downloads + verifies through the native core ingester instead
of gallery-dl — the go-live switch for milestone #71.

- download_backends: subscribestar added to NATIVE_INGESTER_PLATFORMS; a
  _NATIVE_INGESTERS registry + _resolve_native_campaign_id make _run_native_ingester
  / preview_source / verify_source_credential platform-aware. SubscribeStar's
  campaign_id IS the creator URL (no resolver); Patreon still resolves the vanity.
  preview now catches the shared NativeIngestError (covers both platforms).
- platform_lock: subscribestar serialized (one paced walk at a time).
- gallery_dl: subscribestar entry removed from PLATFORM_DEFAULTS (rule 22 — no
  fallback once native works).
- frontend SourceActions: isPatreon → isNative (patreon|subscribestar) so the
  recover/recapture actions show for subscribestar; download_service's
  cursor/mode/post_first + the preview endpoint already key on
  uses_native_ingester, so backfill/recovery/recapture/preview light up for free.
- tests: download_backends (subscribestar native), platform_lock (serialized),
  and three gallery-dl-sample tests repointed to hentaifoundry (api_credentials
  verify, gallery_dl_service skip-value, api_sources arm-no-preflight).

post_is_gated stays best-effort (can't cause junk downloads); not gating this.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:03:48 -04:00
bvandeusenandClaude Opus 4.8 87c7318125 feat(downloads): serialize same-platform downloads (Patreon concurrency cap)
Two concurrent Patreon walks could trip the server rate limit even with each
source pacing its own requests. The platform-cooldown handled the aftermath of
a 429; this adds the preventive half — a per-platform Redis lock so only one
Patreon walk runs at a time. Different platforms still run concurrently up to
the worker concurrency; only a second walk on the SAME serialized platform
waits.

download_source acquires fc:download_lock:<platform> (non-blocking) before the
run. On contention it re-enqueues itself with a short countdown (the pending
event stays — no new event, no log spam), bounded to ~15 min then runs uncapped
as a safety valve. The lock TTL sits just past the hard kill so a SIGKILL'd
worker auto-releases; a backfill chunk only holds it ~10 min, well under the
30-min DownloadEvent recovery sweep. A broker hiccup degrades to uncapped
(prior behaviour) rather than stalling downloads. SERIALIZED_PLATFORMS={patreon};
gallery-dl platforms are left uncapped (self-pacing subprocesses).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 21:22:32 -04:00