bvandeusenandClaude Opus 5
dc840feec7
CI / lint (push) Successful in 2s
Build images / sign-extension (push) Successful in 3s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 5s
CI / frontend-build (push) Successful in 23s
CI / backend-lint-and-test (push) Successful in 35s
Build images / build-web (push) Successful in 1m17s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m15s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
Operator report: a Discord source showed the wrong post time. Listing the real downloads found every Discord folder on the instance (7 artist folders, ~1,600 media) in a directory named `None`, as `<date>_None_<original name>`, next to sidecars named `<original name>.json`.
The real sidecar settles why:
- gallery-dl's discord extractor emits `channel` as a plain string, the message as `message_id`, and the attachment position as `num`. It has no `id` key.
- The patterns asked for `{channel[name]}` and `{id}`. gallery-dl renders a missing field as "None" and carries on.
- The sidecar was named `{filename}.json`, the attachment's ORIGINAL name. find_sidecar never pairs that with `<date>_None_<name>.png`, so no Discord file ever got a Post or a post date. The card fell back to downloaded_at.
- Every `image.png` in a channel also overwrote the same `image.json`.
Fix (gallery_dl.py):
- The directory is `{channel}`.
- The filename is `{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}`, unique per attachment.
- A discord-scoped metadata postprocessor names the sidecar exactly like the media minus its extension, so find_sidecar's first candidate matches. A per-source filename override re-derives the sidecar name. save_metadata=False drops it.
Guard (tests/test_gallery_dl_naming.py) renders the patterns through Python's formatter against a sanitized copy of the real sidecar (same keys and types, invented values). A missing key or a subscript into a string raises, which is the loud failure gallery-dl doesn't give. A positive control shows both shipped patterns fail it.
Existing broken downloads are NOT repaired by this. gallery-dl's archive already records them, so a re-run skips them, and their collided sidecars no longer describe them. That repair is a separate, destructive step for the operator to decide on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
29 lines
739 B
JSON
29 lines
739 B
JSON
{
|
|
"author": "example-artist",
|
|
"author_files": [],
|
|
"author_id": "100000000000000001",
|
|
"category": "discord",
|
|
"channel": "nsfw-drops",
|
|
"channel_id": "200000000000000002",
|
|
"channel_topic": "",
|
|
"channel_type": 0,
|
|
"date": "2024-07-16 16:43:23",
|
|
"extension": "png",
|
|
"filename": "image",
|
|
"files": [],
|
|
"is_thread": false,
|
|
"message": "",
|
|
"message_id": "300000000000000003",
|
|
"num": 1,
|
|
"owner_id": "100000000000000001",
|
|
"parent": "",
|
|
"parent_id": "",
|
|
"parent_type": 0,
|
|
"server": "Example Server",
|
|
"server_files": [],
|
|
"server_id": "400000000000000004",
|
|
"subcategory": "channel",
|
|
"type": "attachment",
|
|
"url": "https://cdn.discordapp.com/attachments/200000000000000002/500000000000000005/image.png"
|
|
}
|