Commit Graph

8 Commits

Author SHA1 Message Date
bvandeusen eb811e11f6 refactor(ingest): per-post handling into run stdout via a downloader outcome (#842)
CI / lint (push) Successful in 3s
CI / frontend-build (push) Successful in 22s
CI / backend-lint-and-test (push) Successful in 38s
CI / integration (push) Successful in 3m18s
Two corrections from operator review:
1. Reuse the existing 'Raw stdout' panel instead of a bespoke structured UI
   section — the native ingester now writes a per-post line into the run stdout
   (parity with gallery-dl's per-file stdout), so the per-post handling shows in
   the panel the operator already uses.
2. DRY: stop re-reading post['attributes'] inline in ingest_core. write_post_record
   now returns a PostRecordOutcome (path, post_type, title, body_chars) — mirroring
   the download_post -> MediaOutcome contract — and the downloader owns the read;
   ingest_core only formats the outcome into the log line.

Reverts the post_diagnostics metadata field + DownloadDetailModal 'Post capture'
section added earlier. Per-post line: 'post <id> [<post_type>] body: N chars' (+
' — EMPTY' when 0), so an empty body is self-explanatory by post_type.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:27:38 -04:00
bvandeusen bcc7266021 feat(downloads): per-post body-capture diagnostics in the event UI (#842)
CI / lint (push) Successful in 2s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Successful in 28s
CI / integration (push) Successful in 3m32s
Operator can't (and shouldn't have to) hunt worker logs to see why a recapture
left a post body empty. Surface per-post handling ON THE EVENT, in the UI.

The feed already requests post_type (in _FIELDS_POST), so ingest_core builds a
per-post diagnostic {post_id, title, post_type, body_chars} with zero extra
fetching — a 0-char body next to its post_type explains an empty post at a
glance (e.g. polls/embeds whose body the API never returns).

- ingest_core: accumulate post_diagnostics; thread via DownloadResult
- download_service: write to DownloadEvent.metadata_['post_diagnostics']
- DownloadDetailModal: 'Post capture' section — totals + empty-body table
  (post_type + chars, flagged) + all-posts table; included in Copy-all
- tests: ingester diag (post_type + body_chars), download_service metadata

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:09:37 -04:00
bvandeusen 4bff1d8558 fix(audit-g4): status-enum miss batch
CI / lint (push) Successful in 3s
CI / backend-lint-and-test (push) Successful in 14s
CI / frontend-build (push) Successful in 19s
CI / intimp (push) Successful in 3m48s
CI / intapi (push) Successful in 7m37s
CI / intcore (push) Successful in 8m18s
Five extension-miss findings from the 2026-06-02 audit, where a status
value was added on one side but a downstream consumer didn't pick it up.

- download_service._phase3_persist: explicit branches for
  ImportResult.status in ('failed','refreshed'). For 'failed' (archive
  probe crash from _import_archive), unlink the source file so the
  filesystem scanner doesn't re-import and re-crash on the same
  archive forever. 'refreshed' is currently unreachable from the
  download path (no deep=True) but matches the importer's documented
  contract; treat as 'attached'.

- gallery-dl backfill auto-complete now gates on dl_result.success +
  no error_type, not just return_code==0 + files_downloaded==0.
  VALIDATION_FAILED exits the subprocess with returncode=0 and
  files_downloaded=0 when every file was quarantined, matching the
  prior predicate exactly and zeroing the operator's armed backfill
  budget on the FIRST quarantine run instead of decrementing.

- attach_in_place archive dispatch now threads artist + source_row
  through _import_archive (and _import_media for archive members)
  and _supersede. The path-walk fallback (_resolve_artist) is still
  used by filesystem-import; the download path now binds
  ImageProvenance to the explicit subscription Source instead of
  rediscovering by (artist_id, platform).

- Three FE handlers now recognize status:'deferred' from
  /api/sources/<id>/check: SubscriptionsTab.onCheck (was toasting
  "event #undefined"), SubscriptionsTab.checkAll (was counting
  deferred as queued), DownloadEventRow.onRetry (was saying
  "re-queued" when nothing was). Pattern matches DownloadsTab.onRetryAll
  which already had it.

- celery_signals._queue_for now maps backup/admin/library_audit
  prefixes to 'maintenance' (matching task_routes). TaskRun.queue
  was returning 'default' for those rows, so per-queue dashboard
  filters and per-queue threshold overrides (added in G3) silently
  missed them.
2026-06-02 16:04:59 -04:00
bvandeusen c87e8e0932 fix(ui): render absolute timestamps in the viewer's local timezone
Download event times showed raw UTC wall-clock (iso.slice). Added
formatDateTime()/formatLocalDate() (local tz, robust to naive vs tz-aware
ISO) and applied them to the download row + detail modal datetimes and the
credential/artist date displays. formatPostDate stays UTC (date-only,
locale-stable, unit-tested).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 19:36:42 -04:00
bvandeusen eebc8e2413 refactor(dry-F2): centralize shared UI primitives (relative-time, toast, download-status)
- utils/date.js: add formatRelative(iso, {future,nullText}); migrate 6 sites
  (SourceRow, SubscriptionsTab, SourceHealthDot, SchedulerStatusBar +
  thin adapters in BackupRunsTable/SystemActivityTab for their '—' null text).
  PostCard (30d->absolute) and CredentialCard (mo/y buckets) intentionally
  keep bespoke formatters.
- utils/toast.js: toast(opts) wraps the globalThis.window?.__fcToast?.(...)
  incantation; migrate 63 call sites across 24 files.
- utils/downloadStatus.js: single source for the download-event status enum
  -> label/color/icon; collapse the 3 duplicate maps (DownloadStatChips,
  DownloadsFilterPopover, DownloadEventRow).

Net -33 lines. Platform metadata was already centralized in platformColor.js.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 11:20:07 -04:00
bvandeusen 73520b7cc3 feat(downloads): copy buttons in the download detail modal
Operator-flagged 2026-05-28: the download event detail modal had no way
to copy the error / stdout / stderr text for researching a failure
(parity with the ErrorDetailModal Copy button shipped in v26.05.26.0).

Added per-block Copy buttons (Error, Errors & warnings, Raw stdout, Raw
stderr — the stdout/stderr ones sit in the expansion-panel title with
@click.stop so they don't toggle the panel) plus a "Copy all
diagnostics" button in the footer that assembles a single block: header
line (event id / platform / artist / status / timestamps) + error +
errors&warnings + full stdout + stderr, ready to paste into an issue.

All routed through utils/clipboard.js copyText() — the navigator.clipboard
→ execCommand fallback that works on the plain-HTTP homelab origin
(per feedback_no_secure_context_apis). Each copy shows a confirmation
toast.
2026-05-28 07:54:40 -04:00
bvandeusen b1b129ce9f feat(downloads-tab): A+B dashboard improvements — row restyle + date-grouped sections with failed-pinned
Operator-flagged 2026-05-27: the Downloads subtab "doesn't feel like a
dashboard" — status was a tiny mdi icon at the far left, platform chip
was neutral-tonal, errors were plain orange text floating on the right,
and all 28 rows from the same hour visually had the same priority.

**Row restyle (A):**
- 4px colored left-edge bar by status (success/error/info/warning/grey)
  — visually scannable at the edge without parsing the chip text
- Status chip with text label (Completed/Failed/Running/Queued/Skipped)
  + leading icon, tonal-colored. Replaces the bare mdi-icon.
- Platform chip swapped to the color-coded subscriptions/PlatformChip
  (Patreon=red mdi-patreon, SubscribeStar=amber, HentaiFoundry=purple,
  Discord=indigo, Pixiv=blue, DeviantArt=green).
- File count: tonal info chip when > 0, dim middle-dot when 0 (so
  scheduled "no-change" scans don't dominate the column visually).
- Error: red tonal pill chip with leading icon, truncated to 60 chars
  with full text in the title tooltip. Replaces plain text.
- Per-row actions (hidden at 50% opacity, fade to full on row hover):
  Retry (only when status=error AND source_id known — hits
  POST /api/sources/<id>/check via the existing sources.checkNow),
  Details (opens the detail modal), Open artist (navigates to the
  artist page). Clicks stop-propagation so they don't bubble to the
  row click.

**Date-grouped sections (B):**
- Events are bucketed into four sections: Today / Yesterday /
  Last 7 days / Earlier. Empty buckets are skipped. Buckets boundaries
  are computed against the operator's local-time start-of-day so
  "Today" matches their intuition.
- Each section has a collapsible header with a row-count chip + a
  red "failed in this section" chip when any failures are in scope.
- Within each section, status='error' rows are pinned to the top
  (operator's eye lands on failures first; successful scans flow
  below).
- Collapsed state persists across refresh within the SubscriptionsView
  lifetime (reactive object, default all-expanded).

DownloadEventRow grid widened to accommodate the status chip + actions
column. PolyMasonry-style ellipsis on the artist link prevents long
names from breaking the layout.

No new endpoints; the Retry path reuses the existing /api/sources/<id>/check
flow (the source-check endpoint was already in place, just not wired
into a per-row button).
2026-05-27 22:18:02 -04:00
bvandeusen d35605ab73 feat(fc3c): /downloads view + DownloadEventRow + DownloadDetailModal + FilterPills
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 20:46:48 -04:00