feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2)
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
A2 of milestone 387. A1 made the gated-post count true; this makes it mean something. A native walk that reached the bottom returned `error_type=None` whether the creator had posted nothing or every post sat behind a tier we don't hold. `source.error_type` stayed NULL and the source read healthy and quiet. gallery-dl has classified this as TIER_LIMITED since the paywall-as-"needs attention" complaint; the native path never did. Two things had to move that the plan didn't foresee, both found by reading the consumers rather than by testing afterwards: The backfill lifecycle's completion test required `error_type is None`. Returning TIER_LIMITED naively would have dropped a fully-paywalled backfill into the not-finished branch — zero downloads means no progress, two strikes marks it "stalled" — so the creator we can see least would become the one we re-walk most. `walk_completed` now admits informational classes. `_update_source_health` only stamps `error_type` on status "error" and CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped by the very run that produced it — which is why FailingSourcesCard's `tier_limited` palette entry has never been reachable. An "ok" run now keeps an informational class while failures stay 0 and last_error stays clear: the run did not fail and must not earn a backoff. Deviation from the plan, deliberate: the filed step said classify only when `downloaded == 0`. gallery-dl doesn't condition on that, and diverging the two backends over the same concept is what rule 169 forbids — so the native path mirrors it. "There is content here you aren't paying for" is equally true in a week we also got the cheap posts. Pinned by a test, since the stricter rule looks more correct. The predicate, the wording and the completion test are defined once in gallery_dl.py and spread into both backends (snippet 3087), rather than re-derived per half — which is exactly how they drifted apart before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -34,7 +34,9 @@ from .gallery_dl import (
|
||||
GalleryDLService,
|
||||
SourceConfig,
|
||||
extract_errors_warnings,
|
||||
is_informational,
|
||||
truncate_log,
|
||||
walk_completed,
|
||||
)
|
||||
from .importer import Importer
|
||||
from .platforms import auth_type_for
|
||||
@@ -552,11 +554,13 @@ class DownloadService:
|
||||
# page is no longer double-counted. new_overrides (read fresh above)
|
||||
# carries the ingester's committed value forward untouched.
|
||||
|
||||
completed = (
|
||||
dl_result.success
|
||||
and dl_result.error_type is None
|
||||
and dl_result.return_code == 0
|
||||
)
|
||||
# Shared with the result path so the two halves can't disagree about
|
||||
# what "finished" means. Note it admits an INFORMATIONAL error_type: a
|
||||
# fully-paywalled creator's backfill really did reach the bottom, and
|
||||
# treating it as unfinished would re-walk that wall every chunk until
|
||||
# the stall counter tripped — the creator we can see least becoming the
|
||||
# one we fetch most.
|
||||
completed = walk_completed(dl_result)
|
||||
if completed:
|
||||
new_overrides["_backfill_state"] = "complete"
|
||||
new_overrides.pop("_backfill_cursor", None)
|
||||
@@ -625,8 +629,17 @@ class DownloadService:
|
||||
if status == "ok":
|
||||
source.consecutive_failures = 0
|
||||
source.last_error = None
|
||||
# alembic 0032 — clear the failure-class chip on success.
|
||||
source.error_type = None
|
||||
# alembic 0032 — clear the failure-class chip on success, EXCEPT an
|
||||
# informational class. tier_limited rides an otherwise-successful
|
||||
# run: failures stay 0 and last_error stays clear (the run did not
|
||||
# fail and must not earn a backoff), but "there is content here we
|
||||
# aren't allowed to see" is a durable fact about the SOURCE, not
|
||||
# about this run. Clearing it here is what left FailingSourcesCard's
|
||||
# `tier_limited` palette entry unreachable — the chip was wiped by
|
||||
# the very success that produced it.
|
||||
source.error_type = (
|
||||
error_type if is_informational(error_type) else None
|
||||
)
|
||||
elif status == "error":
|
||||
source.consecutive_failures = (source.consecutive_failures or 0) + 1
|
||||
source.last_error = error_message
|
||||
|
||||
Reference in New Issue
Block a user