After gallery-dl returns, parse stdout for written file paths and run
the magic-byte validator on each. Files that fail are moved to
{download_path}/_quarantine/{subscription}/{platform}/ with a sidecar
JSON capturing the source URL, validation reason, and original path —
enough for an operator to redownload-from-source or delete.
Adds ErrorType.VALIDATION_FAILED. A successful gallery-dl run that
produced quarantined files is now a soft failure (success=False,
error_message names the dominant failure reason and count) so the
source's error_count ticks up and the dashboard surfaces it. Gated
by download.validate_files setting (default True).
files_quarantined and quarantined_paths are persisted into download
metadata for the UI/API to consume.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Detects truncated/incomplete downloads by checking format-specific
head and tail bytes (JPEG SOI/EOI, PNG signature/IEND, GIF header/3B,
WEBP RIFF size). Catches the production failure mode where a file
landed missing its 2-byte JPEG EOI marker and PIL.ImageFile.load
raised "image file is truncated (6 bytes not processed)" downstream.
Reads only 16 bytes from each end — O(1) per file. Unknown extensions
(JSON sidecars, ugoira zips, etc.) pass through. Hooks into the
download flow in a follow-up commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Supported platform list was missing Pixiv and DeviantArt even though both
have gallery-dl extractors, PLATFORM_DEFAULTS entries, and extension
platform definitions (Pixiv: OAuth token, DeviantArt: cookies).
- Drop DEFAULT_CHECK_INTERVAL from the env var table: no such env var
exists. The default check interval is stored in the database under
download.schedule_interval and edited via Settings → Download settings.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a Patreon source loses tier access, gallery-dl emits "Not allowed to
view post N" warnings for every post AND yt-dlp tries to fetch HLS manifests
which also 403. The ytdl error text contained "HTTP Error 403: Forbidden",
which the classifier's ACCESS_DENIED_PATTERNS matched against the full
combined stdout+stderr — so the run was labeled ACCESS_DENIED instead of
TIER_LIMITED.
Two fixes in one edit, because they're interlocked:
- Compute per-item vs source-level error lines upfront. has_actual_error
now reflects only source-level errors (previously the per-item exclusion
was gated on skip evidence, which tier-limited runs don't produce since
no content was accessed).
- Strip per-item error lines from `combined` before downstream pattern
matching so noise from recovered per-item failures doesn't latch onto
ACCESS_DENIED / HTTP_ERROR / NOT_FOUND classifiers.
Regression test: tier-gated Patreon run with yt-dlp HLS 403s → TIER_LIMITED,
not ACCESS_DENIED.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the four stat cards with a compact three-card strip: 7-day activity
with inline SVG sparkline (new ActivitySparkline), Running Now / Next Check
with per-source countdown list, and System with credential health + disk
usage bar.
- Add GET /downloads/activity-timeline — per-day completed/failed/files
counts, pre-filled with zero buckets so the sparkline always has N points.
- Report filesystem-level usage via shutil.disk_usage in storage rollup,
plus a live fallback in GET /settings so the capacity bar works before
the first Celery rollup runs and for cached rows that predate the field.
- Extract Download Details into a reusable DownloadDetailsModal component
and wire Recent Activity rows to open it (previously Downloads-page only).
- Compute next scheduled check per source from global schedule_interval;
surface credential expiration/missing alerts scoped to platforms that
actually have enabled sources.
- Two-speed polling (5s when downloads are active, 30s idle) with Page
Visibility awareness so background tabs don't churn the API.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Classify Patreon "Not allowed to view post" warnings as TIER_LIMITED so
subscription-gated runs are distinguished from genuine failures, and persist
error_type/message on completed runs so the distinction survives to the UI.
- Fold PLATFORM_DEFAULTS into _get_default_config so gallery-dl.conf is a
complete, editable document; _build_config_for_source now preserves the
user's conf and only re-seeds missing platform sections.
- Add Reset-to-Defaults action in Settings (vs. Revert Changes which just
reloads disk); show info banner when no gallery-dl.conf exists yet.
- Fix Discord embeds option: must be "all" string, not bool (gallery-dl
iterates the value).
- Install yt-dlp + ffmpeg in Docker images so Patreon/Mux HLS video posts
download instead of logging "Cannot import yt-dlp" and skipping.
- Recognize yt-dlp import failures as per-item errors so they don't mask
TIER_LIMITED/NO_NEW_CONTENT classification.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Dashboard's "Retry All Failed" button used to fetch every non-superseded
failure, then fire one retry HTTP call per row. Two missing guards:
- No per-source dedup — a source with N historical failures enqueued N jobs
for the same URL.
- No "recent success" check — a source that succeeded two hours ago could
still have stale un-superseded failures from last week that got re-queued.
Adds POST /api/downloads/retry-failed-bulk:
• Selects non-superseded failures, ordered (source_id ASC, created_at DESC)
• Per-source dedup keeps only the most recent failure per source
• Skips sources whose last_success is within the configurable recent window
(default 24h, via recent_window_hours body param)
• Resets + enqueues in a single DB transaction, single Celery dispatch loop
• Returns a per-reason skip breakdown: duplicate_source, recent_success,
no_source (orphaned rows)
The Dashboard button now calls the bulk endpoint and surfaces the full
skip breakdown in the toast, so the user knows exactly what was (and
wasn't) queued. Logic is extracted into a pure _plan_bulk_retry helper
and unit-tested against SimpleNamespace fixtures — 7 new tests covering
the dedup, window, and orphan paths.
Builds on 513b191. Consumes the new metadata fields in the Downloads
Details modal:
• Run stats rows — exit code, downloaded/skipped/per-item-failures/
warning counts, duration. Zero-value rows hide to reduce noise.
Files Downloaded row prefers run_stats.downloaded_count with
file_count fallback for pre-migration rows.
• Errors & Warnings panel — new panel at the top, populated from
metadata.stderr_errors_warnings and open-by-default when present.
• Copy-to-clipboard buttons — small icon button on each log panel
title, uses navigator.clipboard with snackbar confirmation.
• Verbose log filter — substring filter on the stderr panel.
Graceful for existing rows: every new field is v-if-gated so sessions
saved before the backend change still render correctly with their
old-shape metadata.
Backend foundation for the upcoming Downloads modal refresh. Adds three
helpers on GalleryDLService:
_compute_run_stats exit_code + downloaded/skipped/per-item-failure/
warning counts, derived from stdout+stderr
_extract_errors_warnings filters stderr to just [error]/[warning] lines
so the modal can show a concise errors-only view
_truncate_log caps a log string (default 500KB) with head+tail+
elision marker, preventing verbose multi-hour runs
from bloating the download_sessions JSONB
Wires all three into download.metadata_ in the download task. No schema
change — everything nests into the existing metadata JSONB column.
Follow-up PR will consume these fields in the Downloads.vue modal.
gallery-dl logs `[download][error] Failed to download <file>.part` when a
single post attachment's media URL expires or is deleted, then recovers
and proceeds with the rest of the run. Our error categorizer was letting
a urllib3 debug line like `"HEAD /media-u/v3/<id> HTTP/1.1" 404 0` match
the NOT_FOUND_PATTERNS list, flagging the whole source as deleted even
though gallery-dl kept downloading subsequent items successfully.
Adds "failed to download" to the per-item allowlist so the mask flips
when skip activity is present, matching the shape of the campaign-ID
fix. Total-failure runs still classify as failures.
Also renames the Download Details modal's "Error Log" panel to
"Verbose Log (stderr)" — gallery-dl's `-v` flag sends all logging
(including [debug] lines) to stderr, so the original label was
misleading. Drops the blanket red text color for the same reason.
Both cards now use h-100 with a flex column layout so v-row stretches
them to match, and their v-card-text grows to fill so their footers
line up. Move Active's "+N more queued" out of the list body into a
proper v-card-actions footer mirroring Recent Activity's "View all
downloads" button.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Both lists could grow long enough to dominate the viewport (21 queued
downloads was common). Limit each to 5 entries with a "+N more"
affordance: Active shows a "+N more queued" button that deep-links to
/downloads?status=queued, and Recent Activity folds the overflow count
into the existing "View all downloads" button. Seed Downloads.vue's
status filter from the URL query param so the deep-link actually filters
on arrival.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When gallery-dl logs "[patreon][error] Failed to extract campaign ID"
but then recovers via its /cw/<vanity> fallback and skips every file
(all already archived), _categorize_error was flagging the [error] line
as fatal and returning UNKNOWN_ERROR. Add "failed to extract campaign
id" to the per-item error allowlist alongside "not allowed to view" and
"unable to get post" so skip detection proceeds and the run is
classified NO_NEW_CONTENT.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Swaps v-select for v-autocomplete on the Source filter so users can
type-to-filter instead of scrolling a long list. Same data source, same
selection semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces two stacked filter rows (4 fields + superseded switch) with a
single "Filters" button that opens a popover containing all controls.
Active filters surface as dismissable chips on the bar alongside a
"Clear all" button, so what's currently applied stays visible without
eating vertical space above the table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The dual meaning of the progress bar (red = failing share, green = all
healthy) was not self-explanatory. Added a small legend beside the
section header and a hover tooltip on each bar that states "N of M
<platform> sources failing".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Converts the Source Health last-check column and the Downloads created-at
column from locale date strings to human-relative times ("6 hours ago").
The absolute timestamp is preserved as a native browser tooltip so full
precision is still one hover away; detail modals keep the absolute form.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Consecutive failures on the same source no longer spam the feed as
separate rows. Failed downloads are now bucketed per subscription and
rendered as a single row with a failure count and the latest
timestamp, while successful/running entries still render individually.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The anyio_backend fixture is only consumed by pytest-anyio, which is not a
dependency of this project (we use pytest-asyncio with asyncio_mode=auto).
Also drop the unused pytest import from test_sanity.py.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four bite-sized tasks: pytest bootstrap, resolver service, downloads.py
pure helpers, retry-hook integration. Full TDD with 35 tests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Auto-heal "Failed to extract campaign ID" failures by resolving via
Patreon's campaigns API, caching the ID in Source.metadata_, and retrying
once within the same task run.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
v-model.number on the Settings input returns an empty string when cleared,
which persists through the PATCH and then coerces error_count >= '' to
error_count >= 0 — flagging every source as failing. Coerce to Number and
fall back to 5 unless the value is a finite number >= 1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The platform-health row icon binds to getPlatformColor(row.platform) but the
helper was only present in Sources.vue/Subscriptions.vue/Credentials.vue/
Settings.vue — not copied into Dashboard.vue when the template was introduced,
so the icons next to each platform row rendered in Vuetify's default color.
Also collapse the double blank line left where sourcesWithErrors was removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Defines "failing source" as error_count >= configurable threshold
(default 5), adds per-platform health bars and a strict failing-source
list to the Dashboard, surfaces the threshold on the Settings page.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Earlier container rolled 1.31.5, which fails Patreon fetches with
HTTP 426 Upgrade Required. 1.31.10 restores the OAuth flow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a new filter toggle to the Downloads view that allows users to hide
superseded failed downloads. The filter is enabled by default and passes
exclude_superseded=true to the API when active.
Changes:
- Add filterExcludeSuperseded ref initialized to true
- Add filterExcludeSuperseded to watch dependencies
- Include exclude_superseded param in loadDownloads() when true
- Add v-switch toggle UI control below date filters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated empty state messages to include descriptive icons, main text (body-1), and helper text (caption)
- Active Downloads: shows idle status with check-circle icon
- Recent Activity: shows history icon with context about failures/new downloads
- Sources Needing Attention: shows check-all icon confirming 7-day health check
- Added pulse animation to running downloads with left border animation
- Consistent styling across all empty states with py-8 padding and secondary color icons
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add .claude/ to .gitignore and untrack settings.local.json
- Create .dockerignore to exclude docs, extension, analysis, scripts,
node_modules, and other noise from the Docker build context
- Dockerfile: Node 20→22-alpine, Python 3.11→3.13-slim, DEBIAN_FRONTEND
set, pip upgrade before install
- Track frontend/package-lock.json for reproducible builds
- Update summary.md: date, db.py in project structure, superseded column
in schema, two new key patterns (12/13), full 2026-03-19 changelog
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bvandeusen