feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2)
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
A2 of milestone 387. A1 made the gated-post count true; this makes it mean something. A native walk that reached the bottom returned `error_type=None` whether the creator had posted nothing or every post sat behind a tier we don't hold. `source.error_type` stayed NULL and the source read healthy and quiet. gallery-dl has classified this as TIER_LIMITED since the paywall-as-"needs attention" complaint; the native path never did. Two things had to move that the plan didn't foresee, both found by reading the consumers rather than by testing afterwards: The backfill lifecycle's completion test required `error_type is None`. Returning TIER_LIMITED naively would have dropped a fully-paywalled backfill into the not-finished branch — zero downloads means no progress, two strikes marks it "stalled" — so the creator we can see least would become the one we re-walk most. `walk_completed` now admits informational classes. `_update_source_health` only stamps `error_type` on status "error" and CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped by the very run that produced it — which is why FailingSourcesCard's `tier_limited` palette entry has never been reachable. An "ok" run now keeps an informational class while failures stay 0 and last_error stays clear: the run did not fail and must not earn a backoff. Deviation from the plan, deliberate: the filed step said classify only when `downloaded == 0`. gallery-dl doesn't condition on that, and diverging the two backends over the same concept is what rule 169 forbids — so the native path mirrors it. "There is content here you aren't paying for" is equally true in a week we also got the cheap posts. Pinned by a test, since the stricter rule looks more correct. The predicate, the wording and the completion test are defined once in gallery_dl.py and spread into both backends (snippet 3087), rather than re-derived per half — which is exactly how they drifted apart before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -711,6 +711,61 @@ async def test_gated_posts_are_reported_in_run_stats(
|
||||
assert result.run_stats["downloaded_count"] == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fully_gated_walk_classifies_tier_limited_but_still_succeeds(
|
||||
source_id, sync_engine, tmp_path,
|
||||
):
|
||||
"""A walk where everything was paywalled is a SUCCESS with a reason.
|
||||
|
||||
Before this, it returned `error_type=None` — identical to a creator who
|
||||
simply hadn't posted. success/return_code are asserted deliberately: they
|
||||
are what keep the walk "completed" for the backfill lifecycle and status
|
||||
"ok" for source health, so a source we can't see never accrues
|
||||
consecutive_failures or a backoff it hasn't earned.
|
||||
"""
|
||||
gated = [(f"g{i}", [_media(f"g{i}", 1)]) for i in range(2)]
|
||||
client = _FakeClient([(None, gated)], gated={"g0", "g1"})
|
||||
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
|
||||
|
||||
result = ing.run(
|
||||
source_id=source_id, campaign_id="c1", artist_slug="ingest",
|
||||
url="https://patreon.com/ingest", mode="backfill",
|
||||
)
|
||||
|
||||
assert result.error_type == ErrorType.TIER_LIMITED
|
||||
assert result.success is True
|
||||
assert result.return_code == 0
|
||||
assert "2 posts" in result.error_message
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tier_limited_fires_even_when_some_media_downloaded(
|
||||
source_id, sync_engine, tmp_path,
|
||||
):
|
||||
"""Gated posts classify TIER_LIMITED even on a walk that DID download.
|
||||
|
||||
Mirrors gallery-dl's long-standing rule (`_categorize_error`), which does
|
||||
not condition on a zero download count either. The fact worth surfacing is
|
||||
"there is content here you aren't paying for", and that is equally true in a
|
||||
week we also got the cheaper posts. Pinned because the obvious-looking
|
||||
stricter rule (`downloaded == 0`) would silently diverge the two backends.
|
||||
"""
|
||||
client = _FakeClient(
|
||||
[(None, [("g0", [_media("g0", 1)]), ("open", [_media("open", 1)])])],
|
||||
gated={"g0"},
|
||||
)
|
||||
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
|
||||
|
||||
result = ing.run(
|
||||
source_id=source_id, campaign_id="c1", artist_slug="ingest",
|
||||
url="https://patreon.com/ingest", mode="backfill",
|
||||
)
|
||||
|
||||
assert result.files_downloaded == 1
|
||||
assert result.error_type == ErrorType.TIER_LIMITED
|
||||
assert result.success is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_stats_reports_zero_gated_when_nothing_is_gated(
|
||||
source_id, sync_engine, tmp_path,
|
||||
|
||||
Reference in New Issue
Block a user