feat: a paywalled creator is no longer indistinguishable from a silent one (milestone 387 step A2)
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
CI / extension-version (push) Successful in 5s
Build images / sign-extension (push) Successful in 5s
CI / lint (push) Successful in 6s
Build images / build-agent (push) Successful in 12s
CI / frontend-build (push) Successful in 36s
CI / backend-lint-and-test (push) Successful in 42s
Build images / build-web (push) Successful in 1m24s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m30s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m34s
A2 of milestone 387. A1 made the gated-post count true; this makes it mean something. A native walk that reached the bottom returned `error_type=None` whether the creator had posted nothing or every post sat behind a tier we don't hold. `source.error_type` stayed NULL and the source read healthy and quiet. gallery-dl has classified this as TIER_LIMITED since the paywall-as-"needs attention" complaint; the native path never did. Two things had to move that the plan didn't foresee, both found by reading the consumers rather than by testing afterwards: The backfill lifecycle's completion test required `error_type is None`. Returning TIER_LIMITED naively would have dropped a fully-paywalled backfill into the not-finished branch — zero downloads means no progress, two strikes marks it "stalled" — so the creator we can see least would become the one we re-walk most. `walk_completed` now admits informational classes. `_update_source_health` only stamps `error_type` on status "error" and CLEARS it on "ok". Since TIER_LIMITED is a success, the chip was wiped by the very run that produced it — which is why FailingSourcesCard's `tier_limited` palette entry has never been reachable. An "ok" run now keeps an informational class while failures stay 0 and last_error stays clear: the run did not fail and must not earn a backoff. Deviation from the plan, deliberate: the filed step said classify only when `downloaded == 0`. gallery-dl doesn't condition on that, and diverging the two backends over the same concept is what rule 169 forbids — so the native path mirrors it. "There is content here you aren't paying for" is equally true in a week we also got the cheap posts. Pinned by a test, since the stricter rule looks more correct. The predicate, the wording and the completion test are defined once in gallery_dl.py and spread into both backends (snippet 3087), rather than re-derived per half — which is exactly how they drifted apart before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -370,7 +370,9 @@ async def test_run_download_native_unresolvable_fails_loud(
|
||||
# --- FC-3d: finalize hook updates Source health columns -------------------
|
||||
|
||||
|
||||
async def _seed_source_with_health(db, *, failures=0, last_error=None, suffix="fz"):
|
||||
async def _seed_source_with_health(
|
||||
db, *, failures=0, last_error=None, suffix="fz", error_type=None,
|
||||
):
|
||||
artist = Artist(name=f"alice-{suffix}", slug=f"alice-{suffix}")
|
||||
db.add(artist)
|
||||
await db.flush()
|
||||
@@ -378,6 +380,7 @@ async def _seed_source_with_health(db, *, failures=0, last_error=None, suffix="f
|
||||
artist_id=artist.id, platform="patreon",
|
||||
url=f"https://patreon.com/alice-{suffix}", enabled=True,
|
||||
consecutive_failures=failures, last_error=last_error,
|
||||
error_type=error_type,
|
||||
)
|
||||
db.add(source)
|
||||
await db.flush()
|
||||
@@ -588,6 +591,104 @@ async def test_backfill_state_running_selects_backfill_mode_and_resumes(
|
||||
assert co.get("_backfill_cursor") == "03:RESUME2:next"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_backfill_completes_when_every_post_was_tier_gated(
|
||||
db, db_sync, tmp_path, seed_artist_and_source,
|
||||
):
|
||||
"""A fully-paywalled backfill chunk still COMPLETES.
|
||||
|
||||
The completion test used to require `error_type is None`, so classifying a
|
||||
gated walk as TIER_LIMITED would have dropped it into the not-finished
|
||||
branch: zero files downloaded means `advanced` is False, two strikes marks
|
||||
it 'stalled', and the creator we can see least becomes the one we re-walk
|
||||
most. `walk_completed` admits informational classes for exactly this.
|
||||
"""
|
||||
from backend.app.services.gallery_dl import ErrorType
|
||||
|
||||
_artist, source = seed_artist_and_source
|
||||
source.config_overrides = {
|
||||
"_backfill_state": "running", "_backfill_cursor": "03:NEAR:bottom",
|
||||
}
|
||||
source.backfill_runs_remaining = 5
|
||||
await db.commit()
|
||||
|
||||
svc, _ = _backfill_svc(db, db_sync, tmp_path, _make_fake_dl_result(
|
||||
success=True, written_paths=[], files_downloaded=0,
|
||||
error_type=ErrorType.TIER_LIMITED,
|
||||
error_message="Subscription tier does not grant access to 9 posts",
|
||||
))
|
||||
await svc.download_source(source.id)
|
||||
|
||||
co = (await db.execute(
|
||||
select(Source.config_overrides).where(Source.id == source.id)
|
||||
)).scalar_one()
|
||||
assert co.get("_backfill_state") == "complete"
|
||||
assert "_backfill_cursor" not in co
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ok_run_keeps_the_tier_limited_chip_but_not_the_failure_state(
|
||||
db,
|
||||
):
|
||||
"""An informational class survives a successful run; a failure class doesn't.
|
||||
|
||||
`status == "ok"` clears `error_type` (alembic 0032) because it is the
|
||||
failure-class chip. tier_limited rides an OK run, so that clear is what made
|
||||
FailingSourcesCard's `tier_limited` palette entry unreachable — the chip was
|
||||
wiped by the same success that produced it. It must persist while the
|
||||
failure bookkeeping stays clean: no accrued failures, no last_error.
|
||||
"""
|
||||
from backend.app.services.download_service import DownloadService
|
||||
|
||||
source_id, event_id = await _seed_source_with_health(
|
||||
db, failures=3, last_error="prev", suffix="gated",
|
||||
)
|
||||
svc = DownloadService(
|
||||
async_session=db, sync_session=None,
|
||||
gdl=None, importer=None, cred_service=None,
|
||||
)
|
||||
await svc._update_source_health(
|
||||
source_id=source_id, status="ok", error_message=None,
|
||||
error_type="tier_limited",
|
||||
)
|
||||
|
||||
row = (await db.execute(
|
||||
select(
|
||||
Source.error_type, Source.consecutive_failures, Source.last_error,
|
||||
).where(Source.id == source_id)
|
||||
)).one()
|
||||
assert row.error_type == "tier_limited"
|
||||
assert row.consecutive_failures == 0
|
||||
assert row.last_error is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ok_run_still_clears_a_real_failure_class(db):
|
||||
"""The exemption is narrow: a genuine failure class is still cleared on OK,
|
||||
so a recovered source stops showing a stale chip."""
|
||||
from backend.app.services.download_service import DownloadService
|
||||
|
||||
source_id, _event_id = await _seed_source_with_health(
|
||||
db, failures=2, last_error="boom", suffix="recovered",
|
||||
error_type="auth_error",
|
||||
)
|
||||
svc = DownloadService(
|
||||
async_session=db, sync_session=None,
|
||||
gdl=None, importer=None, cred_service=None,
|
||||
)
|
||||
await svc._update_source_health(
|
||||
source_id=source_id, status="ok", error_message=None,
|
||||
error_type=None,
|
||||
)
|
||||
|
||||
row = (await db.execute(
|
||||
select(Source.error_type, Source.consecutive_failures)
|
||||
.where(Source.id == source_id)
|
||||
)).one()
|
||||
assert row.error_type is None
|
||||
assert row.consecutive_failures == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_backfill_clean_exit_marks_complete(
|
||||
db, db_sync, tmp_path, seed_artist_and_source,
|
||||
|
||||
Reference in New Issue
Block a user