Most-overdue-first scheduling + rich timeout diagnostics #36

2026-05-30T14:30:33-04:00

bvandeusen commented

2026-05-30 14:30:33 -04:00

(Migrated from git.fabledsword.com)

Summary

77f7a23 Order due sources by last_checked_at ASC NULLS FIRST, id in select_due_sources. Combined with Celery's FIFO download queue, the most-overdue source in each tick hits a worker first — closes the "freshly-rerun source keeps cutting in line ahead of a still-waiting one" anti-starvation gap when queue throughput is below the tick population.
99b66aa Preserve gallery-dl's partial output on timeout. The TimeoutExpired branch was returning an empty DownloadResult (no stdout/stderr/file count, generic "timed out after N seconds" message). Now captures e.stdout / e.stderr (whatever gallery-dl produced before being killed), counts files written so far, surfaces a stderr tail-hint in the error message, and promotes to RATE_LIMITED if the partial stderr matches rate-limit patterns — so a rate-limit-induced timeout triggers the platform cooldown the same way a clean-exit rate limit does.

No alembic migrations, no schema changes, no new env vars.

Test plan

After :latest rebuilds and compose restarts:

Rich timeout diagnostics: A timed-out download event now shows populated stdout/stderr in the DownloadDetailModal; files_count > 0 if gallery-dl wrote anything before being killed; error summary reads Download timed out after 900s — N file(s) written; last stderr: <tail line> instead of the prior generic message.
Rate-limit-induced timeout → cooldown: A timeout whose partial stderr shows 429-style patterns flips to error_type: rate_limited and stamps the platform cooldown (visible at /api/system/activity/summary → scheduler.platform_cooldowns).
Most-overdue-first ordering: Under queue pressure, sources with older last_checked_at (or NULL) reach workers ahead of freshly-checked ones; no source can be starved indefinitely.

🤖 Generated with Claude Code

## Summary - **`77f7a23`** Order due sources by `last_checked_at ASC NULLS FIRST, id` in `select_due_sources`. Combined with Celery's FIFO `download` queue, the most-overdue source in each tick hits a worker first — closes the "freshly-rerun source keeps cutting in line ahead of a still-waiting one" anti-starvation gap when queue throughput is below the tick population. - **`99b66aa`** Preserve gallery-dl's partial output on timeout. The `TimeoutExpired` branch was returning an empty `DownloadResult` (no stdout/stderr/file count, generic "timed out after N seconds" message). Now captures `e.stdout` / `e.stderr` (whatever gallery-dl produced before being killed), counts files written so far, surfaces a stderr tail-hint in the error message, and promotes to `RATE_LIMITED` if the partial stderr matches rate-limit patterns — so a rate-limit-induced timeout triggers the platform cooldown the same way a clean-exit rate limit does. No alembic migrations, no schema changes, no new env vars. ## Test plan After `:latest` rebuilds and compose restarts: - [ ] **Rich timeout diagnostics:** A timed-out download event now shows populated stdout/stderr in the DownloadDetailModal; `files_count > 0` if gallery-dl wrote anything before being killed; error summary reads `Download timed out after 900s — N file(s) written; last stderr: <tail line>` instead of the prior generic message. - [ ] **Rate-limit-induced timeout → cooldown:** A timeout whose partial stderr shows 429-style patterns flips to `error_type: rate_limited` and stamps the platform cooldown (visible at `/api/system/activity/summary` → `scheduler.platform_cooldowns`). - [ ] **Most-overdue-first ordering:** Under queue pressure, sources with older `last_checked_at` (or NULL) reach workers ahead of freshly-checked ones; no source can be starved indefinitely. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledCurator#36