feat: Settings can re-download the Discord images the None naming broke (3999)
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m9s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m3s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m35s
CI / lint (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 6s
CI / frontend-build (push) Successful in 19s
CI / backend-lint-and-test (push) Successful in 33s
Build images / build-web (push) Successful in 1m9s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m3s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m35s
The operator chose a clean re-download over relinking in place. The ~1,600 broken files can't be relinked reliably: their message ids are gone, and their sidecars collided.
Settings → Maintenance → "Repair Discord downloads" previews, then applies:
- Deletes every image whose path is `…/discord/None/<8 digits>_None_…`. Both the folder and the name are required, because that pair is only what the bug produced. It reuses cleanup_service.delete_images for the record and file deletes.
- Sweeps the leftover collided sidecars from those folders and removes the emptied folders.
- Only then clears gallery-dl's archive rows `discord%`, excluding `discordasset_%`. Upstream keys message attachments as `discord{message_id}_{num}`. Since the broken files lost their message ids, per-source forgetting is impossible. Every pre-fix Discord download is broken, and files fetched after the fix still exist on disk, so gallery-dl's `skip` won't re-fetch them.
- Arms a fresh backfill on every Discord source.
The apply defaults to preview at both the route and the task, runs on maintenance_long, and is never on a beat. The card uses the confirm-dialog pattern of AttachmentReclaimCard.
Supporting refactors, with no behaviour change:
- gallery_dl.archive_path() is the single definition of the archive location.
- source_service.arm_backfill() is the mutation start_backfill already did, now shared with the sync repair.
Tests (tests/test_discord_repair.py):
- The archive clear leaves other platforms and Discord assets alone, and counting mutates nothing.
- Case-twin artist folders are both found.
- The folder sweep works.
- An integration run shows only the broken image goes. A correctly named Discord file and a `None` folder under Patreon survive, and only Discord sources are re-armed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
This commit is contained in:
@@ -136,6 +136,16 @@ def metadata_postprocessor(filename: str) -> dict:
|
||||
return {"name": "metadata", "mode": "json", "directory": ".", "filename": filename}
|
||||
|
||||
|
||||
def archive_path(images_root: Path) -> Path:
|
||||
"""gallery-dl's download archive: the record of what it has already fetched.
|
||||
|
||||
One definition, because the Discord repair (services/discord_repair.py) has
|
||||
to find the same file the downloader writes, without constructing a service
|
||||
whose __init__ creates directories.
|
||||
"""
|
||||
return Path(images_root) / ".gallery-dl" / "archive.sqlite3"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SourceConfig:
|
||||
"""Per-source overrides loaded from Source.config_overrides JSON.
|
||||
@@ -447,7 +457,7 @@ class GalleryDLService:
|
||||
config = {
|
||||
"extractor": {
|
||||
"base-directory": str(self.images_root),
|
||||
"archive": str(self._config_dir / "archive.sqlite3"),
|
||||
"archive": str(archive_path(self.images_root)),
|
||||
"skip": True,
|
||||
"sleep": self._rate_limit,
|
||||
"sleep-request": max(0.5, self._rate_limit / 4),
|
||||
|
||||
Reference in New Issue
Block a user