feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2)
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
A2 of milestone 387. A1 made the gated-post count true; this makes it mean something. A native walk that reached the bottom returned `error_type=None` whether the creator had posted nothing or every post sat behind a tier we don't hold. `source.error_type` stayed NULL and the source read healthy and quiet. gallery-dl has classified this as TIER_LIMITED since the paywall-as-"needs attention" complaint; the native path never did. Two things had to move that the plan didn't foresee, both found by reading the consumers rather than by testing afterwards: The backfill lifecycle's completion test required `error_type is None`. Returning TIER_LIMITED naively would have dropped a fully-paywalled backfill into the not-finished branch — zero downloads means no progress, two strikes marks it "stalled" — so the creator we can see least would become the one we re-walk most. `walk_completed` now admits informational classes. `_update_source_health` only stamps `error_type` on status "error" and CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped by the very run that produced it — which is why FailingSourcesCard's `tier_limited` palette entry has never been reachable. An "ok" run now keeps an informational class while failures stay 0 and last_error stays clear: the run did not fail and must not earn a backoff. Deviation from the plan, deliberate: the filed step said classify only when `downloaded == 0`. gallery-dl doesn't condition on that, and diverging the two backends over the same concept is what rule 169 forbids — so the native path mirrors it. "There is content here you aren't paying for" is equally true in a week we also got the cheap posts. Pinned by a test, since the stricter rule looks more correct. The predicate, the wording and the completion test are defined once in gallery_dl.py and spread into both backends (snippet 3087), rather than re-derived per half — which is exactly how they drifted apart before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -35,7 +35,13 @@ from collections.abc import Callable
|
||||
from sqlalchemy import delete, func, select, text
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
|
||||
from .gallery_dl import DownloadResult, ErrorType, make_run_stats
|
||||
from .gallery_dl import (
|
||||
DownloadResult,
|
||||
ErrorType,
|
||||
classify_tier_gated,
|
||||
make_run_stats,
|
||||
tier_gated_message,
|
||||
)
|
||||
from .native_ingest_common import NativeAuthError, NativeDriftError
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -574,17 +580,33 @@ class Ingester:
|
||||
error_type=ErrorType.API_DRIFT, error_message=msg,
|
||||
)
|
||||
|
||||
# Normal success: reached the bottom, or a tick that early-outed. rc 0 +
|
||||
# error_type None is REQUIRED for a backfill/recovery walk that reached
|
||||
# the bottom to be marked COMPLETE by
|
||||
# download_service._apply_backfill_lifecycle — so we return None even
|
||||
# when downloaded == 0 (a re-confirming walk that found nothing new still
|
||||
# completed). success=True maps to status "ok" regardless. A tick that
|
||||
# early-outed also returns here; ticks never set backfill state so the
|
||||
# lifecycle is a no-op for them.
|
||||
# Normal success: reached the bottom, or a tick that early-outed. A
|
||||
# zero-download walk still returns success here — a re-confirming walk
|
||||
# that found nothing new genuinely completed. A tick that early-outed
|
||||
# also lands here; ticks never set backfill state so the lifecycle is a
|
||||
# no-op for them.
|
||||
#
|
||||
# success=True and return_code=0 are load-bearing, not cosmetic. They
|
||||
# are what make this a COMPLETE walk for
|
||||
# download_service._apply_backfill_lifecycle (via walk_completed) and
|
||||
# what map it to status "ok", so a walk that fetched nothing doesn't
|
||||
# accrue consecutive_failures or a backoff it hasn't earned.
|
||||
#
|
||||
# #874 follow-up: "nothing new" and "everything sat behind a tier you
|
||||
# don't hold" are different facts, and returning None for both made a
|
||||
# paywalled creator indistinguishable from a silent one. TIER_LIMITED is
|
||||
# classified LAST — every real failure has already returned above —
|
||||
# because tier-gating is the weakest signal and must never mask a
|
||||
# genuine error. It is informational, so walk_completed still counts
|
||||
# this walk as finished (see that predicate for why re-walking a
|
||||
# paywalled creator forever is the bug being avoided).
|
||||
gated_error = classify_tier_gated(gated_skipped)
|
||||
return _result(
|
||||
success=True, return_code=0,
|
||||
error_type=None, error_message=None,
|
||||
error_type=gated_error,
|
||||
error_message=(
|
||||
tier_gated_message(gated_skipped) if gated_error else None
|
||||
),
|
||||
)
|
||||
|
||||
# -- failure mapping (adapter overrides) -------------------------------
|
||||
|
||||
Reference in New Issue
Block a user