feat: finish the Discord switchover — recapture on every native source, backfills that run, gallery-dl's Discord config retired (milestone 428)
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 21s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Failing after 2m19s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
CI and images / lint (push) Successful in 2s
CI and images / extension-version (push) Successful in 3s
CI and images / frontend-build (push) Successful in 21s
CI and images / backend-lint-and-test (push) Successful in 32s
CI and images / integration (push) Failing after 2m19s
CI and images / sign-extension (push) Skipped
CI and images / build-web (push) Skipped
CI and images / smoke-web (push) Skipped
CI and images / promote (push) Skipped
CI and images / build-agent (push) Skipped
- Recover and Recapture show on every native source. The menu gated them on
a copied platform list ('patreon', 'subscribestar') that went stale when
Discord moved over. Sources now carry `native_ingester` from the backend's
own predicate.
- A running backfill is due on every scheduler tick. Nothing queued a
backfill's next chunk: each one waited for the source's regular interval,
so an armed backfill sat idle until the next check (8h at the default) and
a five-chunk walk took most of two days. The in-flight guard and the
platform lock keep one chunk at a time. A failing source falls back to its
backoff, and a stalled or out-of-budget walk stops being due. It also runs
when the artist has auto-check off, since the operator started it by hand.
- gallery-dl no longer carries Discord: its naming constants, platform
defaults, sidecar-mirroring postprocessor and token injection are gone.
The naming test moves to the native downloader and still renders against
the real gallery-dl sidecar fixture. That is the guard that the files
gallery-dl wrote are found on disk rather than fetched again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjrnpQjRgHdvq95rASoiR
This commit is contained in:
@@ -6,9 +6,11 @@ existing file on disk (`skipped_disk`) instead of fetching it again:
|
||||
|
||||
<images>/<artist>/discord/<channel>/<YYYYMMDD>_<message_id>_<NN>_<name>.<ext>
|
||||
|
||||
That is FC's gallery-dl config (`gallery_dl.DISCORD_DIRECTORY` /
|
||||
`DISCORD_FILENAME`) under the per-source base directory
|
||||
`<images>/<artist>/<platform>`. The name is cleaned the way gallery-dl cleans it
|
||||
That is what FC's gallery-dl config produced (directory `{channel}`, filename
|
||||
`{date:%Y%m%d}_{message_id}_{num:>02}_{filename}.{extension}`, under the
|
||||
per-source base directory `<images>/<artist>/<platform>`), retired from that
|
||||
config once Discord moved here; tests/test_discord_naming.py pins the match
|
||||
against a real gallery-dl sidecar. The name is cleaned the way gallery-dl cleans it
|
||||
on Linux — `/` becomes `_` and control characters are removed, nothing else
|
||||
(`path-restrict: auto`, `path-remove` defaults). It is NOT `sanitize_segment`,
|
||||
whose Windows set would turn a `:` in a channel or file name into `_` and miss
|
||||
|
||||
Reference in New Issue
Block a user