Commit Graph

19 Commits

Author SHA1 Message Date
bvandeusen 86bf43c80a feat(downloader): tier-limited classification, config defaults, yt-dlp support
- Classify Patreon "Not allowed to view post" warnings as TIER_LIMITED so
  subscription-gated runs are distinguished from genuine failures, and persist
  error_type/message on completed runs so the distinction survives to the UI.
- Fold PLATFORM_DEFAULTS into _get_default_config so gallery-dl.conf is a
  complete, editable document; _build_config_for_source now preserves the
  user's conf and only re-seeds missing platform sections.
- Add Reset-to-Defaults action in Settings (vs. Revert Changes which just
  reloads disk); show info banner when no gallery-dl.conf exists yet.
- Fix Discord embeds option: must be "all" string, not bool (gallery-dl
  iterates the value).
- Install yt-dlp + ffmpeg in Docker images so Patreon/Mux HLS video posts
  download instead of logging "Cannot import yt-dlp" and skipping.
- Recognize yt-dlp import failures as per-item errors so they don't mask
  TIER_LIMITED/NO_NEW_CONTENT classification.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-20 19:32:44 -04:00
bvandeusen 513b191e47 feat(downloads): capture structured run stats and filtered logs at save time
Backend foundation for the upcoming Downloads modal refresh. Adds three
helpers on GalleryDLService:

  _compute_run_stats   exit_code + downloaded/skipped/per-item-failure/
                      warning counts, derived from stdout+stderr
  _extract_errors_warnings   filters stderr to just [error]/[warning] lines
                             so the modal can show a concise errors-only view
  _truncate_log        caps a log string (default 500KB) with head+tail+
                      elision marker, preventing verbose multi-hour runs
                      from bloating the download_sessions JSONB

Wires all three into download.metadata_ in the download task. No schema
change — everything nests into the existing metadata JSONB column.

Follow-up PR will consume these fields in the Downloads.vue modal.
2026-04-19 14:18:25 -04:00
bvandeusen ae140dbb0a feat(ui): show relative timestamps on Dashboard and Downloads tables
Converts the Source Health last-check column and the Downloads created-at
column from locale date strings to human-relative times ("6 hours ago").
The absolute timestamp is preserved as a native browser tooltip so full
precision is still one hover away; detail modals keep the absolute form.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 15:07:19 -04:00
bvandeusen b57e93f1bf chore(downloads): log failed resolutions and tighten retry dataclass/tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 14:18:43 -04:00
bvandeusen 35589e996a feat(downloads): auto-heal patreon campaign-ID failures via resolver retry 2026-04-18 14:11:11 -04:00
bvandeusen 8e3da096ec feat(downloads): add patreon URL-rewrite and error-pattern helpers 2026-04-18 13:43:21 -04:00
bvandeusen 8cd0ce4ecb fix: restore AsyncSession import and fix dead regex patterns in NOT_FOUND_PATTERNS
- Added missing 'from sqlalchemy.ext.asyncio import AsyncSession' import in downloads.py
  This fixes the NameError at module load time that was breaking all download tasks

- Replaced regex patterns with plain-string equivalents in NOT_FOUND_PATTERNS:
  * 'account.*deleted' → 'account deleted' (. and * are literal in substring search)
  * 'profile.*not.*exist' → 'profile does not exist' (same issue)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:43:01 -04:00
bvandeusen 6432210a36 refactor: extract shared DB helpers to tasks/db.py
Extract identical get_async_session() and cleanup_engine() functions
from downloads.py and maintenance.py into a shared db.py module.
This eliminates code duplication and improves maintainability.

db.py imports only from app.config and SQLAlchemy to avoid circular
dependencies with the task modules that import from it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:36:42 -04:00
bvandeusen 0f2901ef58 feat: mark prior failures as superseded on successful download 2026-03-18 23:32:51 -04:00
bvandeusen 6036c34680 fix: pass download_id in process_download to prevent retry skip 2026-03-18 23:27:15 -04:00
bvandeusen 2190a9c16b correcting pixiv connection functionality 2026-02-04 12:45:31 -05:00
Bryan Van Deusen 097ab16c50 more UI polish and schedule safe guards and optimization 2026-01-30 00:01:45 -05:00
Bryan Van Deusen fcccfb24ee changes to task scheduling and readability change on downloads view 2026-01-29 19:33:04 -05:00
Bryan Van Deusen 575e20f58c ongoing polish in queue handling 2026-01-28 09:32:47 -05:00
Bryan Van Deusen c7a259a63b correcting worker db connection issue 2026-01-27 18:05:54 -05:00
Bryan Van Deusen 13b4da5ca0 better lifecycle visibility implementation. 2026-01-27 17:33:25 -05:00
Bryan Van Deusen d2e4d6c588 corrected time record type, mismatch caused issues and kept credential verification from occuring 2026-01-26 22:51:03 -05:00
Bryan Van Deusen 058901cd05 added discord functionality to extension and polished some of downloader process. 2026-01-24 23:15:03 -05:00
Bryan Van Deusen b9b8048a2d rapid interations of server side app and firefox extension 2026-01-24 22:52:51 -05:00