fix: a Mega folder link's files in subfolders are imported, and a retry no longer deletes complete files
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 20s
CI and images / backend-lint-and-test (push) Failing after 33s
CI and images / extension-test (push) Successful in 18s
CI and images / integration (push) Successful in 2m30s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped
CI and images / build-agent (push) Skipped

_fetch_mega listed only the top level of the link dir, so a folder link's
files in subfolders never reached the importer: the link either failed with
"wrote no new file" or read downloaded with its subfolders left on disk.
It now walks the tree.

The worker cleared every unrecorded file in a link dir before each fetch
and after a failed one. Combined with the daily recovery of dead links,
that deletes complete downloads whose upstream is gone, which may be the
only copy. Every fetcher stages to .part or .megatmp.* and renames on
completion, so the clear now removes staging files only, and a successful
fetch imports every complete unrecorded file in the link dir, including
ones an earlier attempt finished.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-06 10:24:14 -05:00
co-authored by Claude Opus 5.5
parent 5259caa846
commit 2bea4b0d31
4 changed files with 136 additions and 25 deletions
+21
View File
@@ -108,6 +108,27 @@ def test_mega_collects_new_files(tmp_path, monkeypatch):
assert res.files[0].name == "film.mp4"
def test_mega_folder_link_collects_files_in_subfolders(tmp_path, monkeypatch):
"""A folder link keeps the folder's tree; every file in it is collected,
files already there before the fetch and staging files are not."""
(tmp_path / "old.png").write_bytes(b"earlier")
def fake_mega(url, out_dir, *, timeout):
nested = Path(out_dir) / "images" / "2020"
nested.mkdir(parents=True)
(nested / "kara.png").write_bytes(b"png")
(Path(out_dir) / "videos").mkdir()
(Path(out_dir) / "videos" / "loop.webm").write_bytes(b"webm")
(nested / ".megatmp.AbCd").write_bytes(b"half")
monkeypatch.setattr(ef, "_run_mega_get", fake_mega)
res = fetch_external("mega", "https://mega.nz/folder/x#k", tmp_path)
assert res.ok
assert sorted(p.relative_to(tmp_path).as_posix() for p in res.files) == [
"images/2020/kara.png", "videos/loop.webm",
]
def test_unsupported_host(tmp_path):
res = fetch_external("nope", "https://x", tmp_path)
assert not res.ok