fix: the native path never reported tier-gated posts (milestone 387 step A1)
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 55s
CI / integration (push) Successful in 2m4s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Failing after 13s
Build images / build-web (push) Successful in 9s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 55s
CI / integration (push) Successful in 2m4s
Build images / sign-extension (push) Successful in 5s
Build images / build-agent (push) Successful in 9s
Build images / build-ml (push) Failing after 13s
Build images / build-web (push) Successful in 9s
Build images / smoke-web (push) Skipped
Build images / promote (push) Skipped
`make_run_stats` has always declared `tier_gated_count`, and DownloadDetailModal has always rendered it. gallery-dl populated it; `ingest_core._result` did not — it built run_stats with six keys and let the seventh default to 0, while the very same walk counted gated posts into `gated_skipped` and spent the number on a log line. So on Patreon, SubscribeStar and pixiv — the three platforms we now own — the Downloads modal read "Tier-gated: 0" for a walk that skipped N paywalled posts. A creator we've lost access to was indistinguishable from a creator who stopped posting. Migrating Patreon off gallery-dl is what dropped the signal. Pass the count through, and tick it in the live-progress payload too, so a long backfill on an inaccessible creator explains itself while it runs rather than only at finalization. ActiveDownloadsPanel renders it only when non-zero, coloured 'info' to match the severity FailingSourcesCard already assigns tier_limited — this is not a failure. Tests assert the run_stats key the UI actually reads rather than the ingester's internal counter, so the guard tracks the property and not a name. Falsification is structural: `make_run_stats` defaults the key to 0, so both assertions fail against the pre-fix call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
@@ -245,6 +245,12 @@ class Ingester:
|
||||
per_item_failures=errors,
|
||||
quarantined_count=quarantined,
|
||||
dead_lettered_count=dead_lettered,
|
||||
# #874 follow-up: the native path counted gated posts but
|
||||
# never reported them, so DownloadDetailModal's "Tier-gated"
|
||||
# field read 0 on every native walk while gallery-dl's read
|
||||
# true. A paywalled creator was indistinguishable from a
|
||||
# silent one.
|
||||
tier_gated_count=gated_skipped,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -464,6 +470,10 @@ class Ingester:
|
||||
"errors": errors,
|
||||
"quarantined": quarantined,
|
||||
"posts": posts_processed,
|
||||
# Ticks during the walk, not only at finalization: a
|
||||
# deep backfill on a creator we've lost access to is
|
||||
# otherwise a long run of zeros with no explanation.
|
||||
"gated": gated_skipped,
|
||||
})
|
||||
|
||||
if early_out:
|
||||
|
||||
@@ -32,6 +32,13 @@
|
||||
v-if="e.live.errors" class="fc-active__count fc-active__count--err"
|
||||
title="errors"
|
||||
>✕ {{ e.live.errors }}</span>
|
||||
<!-- Tier-gated posts (#874): not an error — the walk is working, the
|
||||
content just isn't ours. Shown only when non-zero so a healthy
|
||||
run stays uncluttered. -->
|
||||
<span
|
||||
v-if="e.live.gated" class="fc-active__count fc-active__count--gated"
|
||||
title="posts skipped — no access at your tier"
|
||||
>🔒 {{ e.live.gated }}</span>
|
||||
<span class="fc-active__count fc-active__count--posts" title="posts scanned">
|
||||
{{ e.live.posts }} posts</span>
|
||||
</span>
|
||||
@@ -141,6 +148,9 @@ function elapsed (startedIso) {
|
||||
}
|
||||
.fc-active__count { color: rgb(var(--v-theme-on-surface-variant)); }
|
||||
.fc-active__count--err { color: rgb(var(--v-theme-error)); }
|
||||
/* Matches FailingSourcesCard's severity map, which already colours
|
||||
tier_limited as 'info' — no-access is information, not a failure. */
|
||||
.fc-active__count--gated { color: rgb(var(--v-theme-info)); }
|
||||
.fc-active__count--posts { opacity: 0.7; }
|
||||
@keyframes fc-active-pulse {
|
||||
0%, 100% { opacity: 1; transform: scale(1); }
|
||||
|
||||
@@ -678,6 +678,56 @@ async def test_gated_post_skipped_entirely_no_media_no_record(
|
||||
assert _count_ledger(sync_engine, source_id) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gated_posts_are_reported_in_run_stats(
|
||||
source_id, sync_engine, tmp_path,
|
||||
):
|
||||
"""A gated post must be COUNTED, not just skipped.
|
||||
|
||||
`make_run_stats` has always declared `tier_gated_count` and
|
||||
DownloadDetailModal has always rendered it, but the native path never
|
||||
populated it — so a walk that skipped N paywalled posts reported 0, and a
|
||||
creator we've lost access to looked exactly like a creator who stopped
|
||||
posting. Asserted on the run_stats key the UI actually reads (not the
|
||||
ingester's internal counter) so the guard tracks the property, not a name.
|
||||
"""
|
||||
gated = [(f"g{i}", [_media(f"g{i}", 1)]) for i in range(3)]
|
||||
am = _media("open", 1)
|
||||
client = _FakeClient(
|
||||
[(None, [*gated, ("open", [am])])],
|
||||
gated={"g0", "g1", "g2"},
|
||||
)
|
||||
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
|
||||
|
||||
result = ing.run(
|
||||
source_id=source_id, campaign_id="c1", artist_slug="ingest",
|
||||
url="https://patreon.com/ingest", mode="backfill",
|
||||
)
|
||||
|
||||
assert result.run_stats["tier_gated_count"] == 3
|
||||
# Not conflated with the other skip tiers: seen/on-disk skips are a
|
||||
# different fact from "we aren't allowed to see this".
|
||||
assert result.run_stats["skipped_count"] == 0
|
||||
assert result.run_stats["downloaded_count"] == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_stats_reports_zero_gated_when_nothing_is_gated(
|
||||
source_id, sync_engine, tmp_path,
|
||||
):
|
||||
"""The counter reports a real zero on a clean walk — so a 0 in the UI means
|
||||
'nothing was gated', not 'this path never counted'."""
|
||||
client = _FakeClient([(None, [("p1", [_media("p1", 1)])])])
|
||||
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
|
||||
|
||||
result = ing.run(
|
||||
source_id=source_id, campaign_id="c1", artist_slug="ingest",
|
||||
url="https://patreon.com/ingest", mode="backfill",
|
||||
)
|
||||
|
||||
assert result.run_stats["tier_gated_count"] == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recapture_does_not_refetch_seen_media_missing_from_disk(
|
||||
source_id, sync_engine, tmp_path,
|
||||
|
||||
Reference in New Issue
Block a user