Commit Graph

2 Commits

Author SHA1 Message Date
bvandeusen bd2807cdd1 feat(external): mega.nz via megatools in the runtime image (Phase 4c)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 17s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m17s
Use `megatools dl` (Debian-native apt package) for mega.nz public links rather
than MEGAcmd — no external MEGA apt repo/key to add, one apt line. Adds
`megatools` to the runtime Dockerfile; the fetcher's mega backend now shells
`megatools dl --path <dir> <url>` (key in the #fragment is preserved by the
extractor). gdown (gdrive) is already a pip dep in the runtime image.

NOTE: build.yml builds the image on main/tags only (not dev), so this Dockerfile
change is verified on the next dev→main merge, not by this dev push. The fetcher
code path is unit-tested via the mocked _run_mega_get seam.

With this, all 5 hosts download end-to-end once a celery download-worker runs.
Refs FC #830 (Phase 4c).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 15:41:18 -04:00
bvandeusen 13253b18d1 feat(external): file-host fetcher subsystem (Phase 4a)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 18s
CI / backend-lint-and-test (push) Successful in 37s
CI / integration (push) Successful in 3m21s
Shared, reusable fetchers for the 5 off-platform hosts behind one signature
(fetch_external(host, url, dest_dir, ...) -> FetchResult):
- dropbox    : force dl=1 + stream GET
- pixeldrain : GET /api/file/{id}
- mediafire  : scrape the download page for the direct link + stream GET
- gdrive     : gdown (confirm-token + virus-scan interstitial); added to reqs
- mega       : MEGAcmd `mega-get` subprocess (public link incl. #key)

HTTP/gdown/subprocess go through module seams so unit tests run without
network/gdown/MEGAcmd. fetch_external never raises — every backend failure
(transport, non-200, scrape miss, subprocess error, stop) is captured on
.error so the worker (next slice) records it and moves on. mega's binary lands
in the runtime image in a later slice; the code is complete + tested now.

Refs FC #830 (Phase 4a).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 13:29:20 -04:00