feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2)
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s

A2 of milestone 387. A1 made the gated-post count true; this makes it
mean something.

A native walk that reached the bottom returned `error_type=None`
whether the creator had posted nothing or every post sat behind a tier
we don't hold. `source.error_type` stayed NULL and the source read
healthy and quiet. gallery-dl has classified this as TIER_LIMITED since
the paywall-as-"needs attention" complaint; the native path never did.

Two things had to move that the plan didn't foresee, both found by
reading the consumers rather than by testing afterwards:

The backfill lifecycle's completion test required `error_type is None`.
Returning TIER_LIMITED naively would have dropped a fully-paywalled
backfill into the not-finished branch — zero downloads means no
progress, two strikes marks it "stalled" — so the creator we can see
least would become the one we re-walk most. `walk_completed` now admits
informational classes.

`_update_source_health` only stamps `error_type` on status "error" and
CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped
by the very run that produced it — which is why FailingSourcesCard's
`tier_limited` palette entry has never been reachable. An "ok" run now
keeps an informational class while failures stay 0 and last_error stays
clear: the run did not fail and must not earn a backoff.

Deviation from the plan, deliberate: the filed step said classify only
when `downloaded == 0`. gallery-dl doesn't condition on that, and
diverging the two backends over the same concept is what rule 169
forbids — so the native path mirrors it. "There is content here you
aren't paying for" is equally true in a week we also got the cheap
posts. Pinned by a test, since the stricter rule looks more correct.

The predicate, the wording and the completion test are defined once in
gallery_dl.py and spread into both backends (snippet 3087), rather than
re-derived per half — which is exactly how they drifted apart before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
2026-09-09 15:40:19 -04:00
co-authored by Claude Opus 5
parent 173f4b00aa
commit 7751715b83
5 changed files with 281 additions and 24 deletions
+55
View File
@@ -711,6 +711,61 @@ async def test_gated_posts_are_reported_in_run_stats(
assert result.run_stats["downloaded_count"] == 1
@pytest.mark.asyncio
async def test_fully_gated_walk_classifies_tier_limited_but_still_succeeds(
source_id, sync_engine, tmp_path,
):
"""A walk where everything was paywalled is a SUCCESS with a reason.
Before this, it returned `error_type=None` — identical to a creator who
simply hadn't posted. success/return_code are asserted deliberately: they
are what keep the walk "completed" for the backfill lifecycle and status
"ok" for source health, so a source we can't see never accrues
consecutive_failures or a backoff it hasn't earned.
"""
gated = [(f"g{i}", [_media(f"g{i}", 1)]) for i in range(2)]
client = _FakeClient([(None, gated)], gated={"g0", "g1"})
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
result = ing.run(
source_id=source_id, campaign_id="c1", artist_slug="ingest",
url="https://patreon.com/ingest", mode="backfill",
)
assert result.error_type == ErrorType.TIER_LIMITED
assert result.success is True
assert result.return_code == 0
assert "2 posts" in result.error_message
@pytest.mark.asyncio
async def test_tier_limited_fires_even_when_some_media_downloaded(
source_id, sync_engine, tmp_path,
):
"""Gated posts classify TIER_LIMITED even on a walk that DID download.
Mirrors gallery-dl's long-standing rule (`_categorize_error`), which does
not condition on a zero download count either. The fact worth surfacing is
"there is content here you aren't paying for", and that is equally true in a
week we also got the cheaper posts. Pinned because the obvious-looking
stricter rule (`downloaded == 0`) would silently diverge the two backends.
"""
client = _FakeClient(
[(None, [("g0", [_media("g0", 1)]), ("open", [_media("open", 1)])])],
gated={"g0"},
)
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
result = ing.run(
source_id=source_id, campaign_id="c1", artist_slug="ingest",
url="https://patreon.com/ingest", mode="backfill",
)
assert result.files_downloaded == 1
assert result.error_type == ErrorType.TIER_LIMITED
assert result.success is True
@pytest.mark.asyncio
async def test_run_stats_reports_zero_gated_when_nothing_is_gated(
source_id, sync_engine, tmp_path,