feat(external): file-host fetcher subsystem (Phase 4a)
Shared, reusable fetchers for the 5 off-platform hosts behind one signature
(fetch_external(host, url, dest_dir, ...) -> FetchResult):
- dropbox : force dl=1 + stream GET
- pixeldrain : GET /api/file/{id}
- mediafire : scrape the download page for the direct link + stream GET
- gdrive : gdown (confirm-token + virus-scan interstitial); added to reqs
- mega : MEGAcmd `mega-get` subprocess (public link incl. #key)
HTTP/gdown/subprocess go through module seams so unit tests run without
network/gdown/MEGAcmd. fetch_external never raises — every backend failure
(transport, non-200, scrape miss, subprocess error, stop) is captured on
.error so the worker (next slice) records it and moves on. mega's binary lands
in the runtime image in a later slice; the code is complete + tested now.
Refs FC #830 (Phase 4a).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -38,3 +38,8 @@ nh3>=0.2,<0.3
|
||||
# Archive extraction (FC-2d-iii filesystem-import aid)
|
||||
rarfile>=4.2,<5
|
||||
py7zr>=1,<2
|
||||
|
||||
# Google Drive fetcher for off-platform file-host links (external downloads,
|
||||
# #830). Handles Drive's confirm-token + virus-scan interstitial. mega.nz uses
|
||||
# the MEGAcmd `mega-get` binary instead (added to the runtime image, not pip).
|
||||
gdown>=5,<6
|
||||
|
||||
Reference in New Issue
Block a user