fix: dead external links are retried sparingly, not three times a day
CI and images / lint (push) Successful in 1s
CI and images / extension-version (push) Successful in 2s
CI and images / frontend-build (push) Successful in 20s
CI and images / extension-test (push) Successful in 23s
CI and images / backend-lint-and-test (push) Successful in 33s
CI and images / integration (push) Successful in 2m20s
CI and images / sign-extension (push) Successful in 2s
CI and images / build-agent (push) Successful in 5s
CI and images / build-web (push) Successful in 2m1s
CI and images / smoke-web (push) Successful in 49s
CI and images / promote (push) Successful in 1s
CI and images / refresh-dev (push) Skipped
CI and images / unattended-failure (push) Skipped

The daily recovery reset every dead link to a fresh three-attempt budget,
so a link that can never succeed (an archive too big to import inside the
fetch's hard limit, a file made private) was downloaded up to three times a
day, every day.

Recovery now gives a dead link one more attempt once it has been dead for
2**recoveries days (1, 2, 4, 8, 16), and stops after five. A recovered link
that fails again goes straight back to dead. Retention prunes only links
recovery has given up on, 30 days after their last attempt. A manual refetch
of a post restores the full budgets.

Migration 0118 adds external_link.recoveries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-05 19:13:46 -05:00
co-authored by Claude Opus 5.5
parent 157f2803c0
commit 5259caa846
5 changed files with 150 additions and 15 deletions
+4 -3
View File
@@ -21,9 +21,9 @@ log = logging.getLogger(__name__)
def refetch_links_for_post(session: Session, post_id: int) -> dict:
"""Reset every settled ExternalLink on a post to pending (fresh attempt
budget) and dispatch its fetch. In-flight ('downloading') links are left
alone. Commits. Returns {links_total, links_reset}."""
"""Reset every settled ExternalLink on a post to pending (fresh attempt and
recovery budgets) and dispatch its fetch. In-flight ('downloading') links
are left alone. Commits. Returns {links_total, links_reset}."""
links = session.execute(
select(ExternalLink).where(ExternalLink.post_id == post_id)
).scalars().all()
@@ -33,6 +33,7 @@ def refetch_links_for_post(session: Session, post_id: int) -> dict:
continue
link.status = "pending"
link.attempts = 0
link.recoveries = 0
link.last_error = None
link.completed_at = None
reset_ids.append(link.id)