fix(pixiv): match gallery-dl's exact on-disk filename to avoid re-download at cutover
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 33s
CI / integration (push) Successful in 3m35s

The native downloader used the Windows-safe sanitize_segment, but gallery-dl on
Linux (path-restrict auto→'/', path-remove default control chars, path-strip
auto→'') replaces ONLY '/' and deletes control chars — the Windows-forbidden set
(<>:"|?*) and trailing dots/spaces stay RAW in on-disk titles. Any pixiv title
with those chars would therefore miss the tier-2 disk-skip and re-download the
whole work at cutover (seen-ledger starts empty). Replace sanitize_segment with
gdl_clean_filename, a byte-exact mirror of gallery-dl 1.32.5 build_filename
(verified against path.py). Directory + template already matched; this closes the
last parity gap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CDgx8bQS5YrGRK76v8HUnM
This commit is contained in:
2026-07-03 13:49:04 -04:00
parent a7f715ec43
commit 544e30bfb9
2 changed files with 45 additions and 7 deletions
+18 -3
View File
@@ -178,11 +178,15 @@ def test_download_post_honours_should_stop(tmp_path):
assert outcomes == []
def test_filename_is_sanitized(tmp_path):
def test_filename_matches_gallery_dl_linux_restrict(tmp_path):
# gallery-dl on Linux replaces ONLY "/" and deletes control chars — the
# Windows-forbidden set (<>:"|?*) stays RAW in the on-disk name. Matching
# that byte-for-byte is what lets the tier-2 disk-skip recognize
# pre-cutover files; a stricter sanitizer would re-download them.
post, _ = _post_and_media(222)
weird = MediaItem(
url="https://i.pximg.net/img-original/img/2026/06/18/12/30/00/222_p0.jpg",
filename='222_What? A "Title"/Slash_00.jpg',
filename='222_What? A "Title"/Slash\x0900.jpg', # ? " stay, / → _, \t dropped
kind="image",
filehash=None,
post_id="222",
@@ -191,7 +195,18 @@ def test_filename_is_sanitized(tmp_path):
dl = _downloader(tmp_path)
outcomes = dl.download_post(post, [weird], "artist-a")
assert outcomes[0].status == "downloaded"
assert outcomes[0].path.name == "222_What_ A _Title__Slash_00.jpg"
assert outcomes[0].path.name == '222_What? A "Title"_Slash00.jpg'
def test_gdl_clean_filename_parity():
from backend.app.services.pixiv_downloader import gdl_clean_filename
# Only "/" is replaced; the Windows-forbidden set is untouched.
assert gdl_clean_filename('a<>:"|?*b.png') == 'a<>:"|?*b.png'
assert gdl_clean_filename("a/b/c.png") == "a_b_c.png"
# Control chars (newline, tab, DEL) are deleted, not replaced.
assert gdl_clean_filename("a\nb\tc\x7fd.png") == "abcd.png"
# Trailing dots/spaces are NOT stripped on Linux.
assert gdl_clean_filename("title . .png") == "title . .png"
def test_own_session_carries_app_referer(tmp_path):