fix: the native path never reported tier-gated posts (milestone 387 step A1)
CI / lint (push) Successful in 4s
CI / extension-version (push) Successful in 4s
CI / frontend-build (push) Successful in 26s
CI / backend-lint-and-test (push) Successful in 55s
CI / integration (push) Successful in 2m4s
Build images / sign-extension (push) Successful in 11s
Build images / build-agent (push) Successful in 2m41s
Build images / build-web (push) Successful in 2m3s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 5m40s
Build images / promote (push) Skipped

`make_run_stats` has always declared `tier_gated_count`, and
DownloadDetailModal has always rendered it. gallery-dl populated it;
`ingest_core._result` did not — it built run_stats with six keys and let
the seventh default to 0, while the very same walk counted gated posts
into `gated_skipped` and spent the number on a log line.

So on Patreon, SubscribeStar and pixiv — the three platforms we now own —
the Downloads modal read "Tier-gated: 0" for a walk that skipped N
paywalled posts. A creator we've lost access to was indistinguishable
from a creator who stopped posting. Migrating Patreon off gallery-dl is
what dropped the signal.

Pass the count through, and tick it in the live-progress payload too, so
a long backfill on an inaccessible creator explains itself while it runs
rather than only at finalization. ActiveDownloadsPanel renders it only
when non-zero, coloured 'info' to match the severity FailingSourcesCard
already assigns tier_limited — this is not a failure.

Tests assert the run_stats key the UI actually reads rather than the
ingester's internal counter, so the guard tracks the property and not a
name. Falsification is structural: `make_run_stats` defaults the key to
0, so both assertions fail against the pre-fix call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
2026-09-09 12:34:08 -04:00
co-authored by Claude Opus 5
parent ad8392b790
commit 173f4b00aa
3 changed files with 70 additions and 0 deletions
+10
View File
@@ -245,6 +245,12 @@ class Ingester:
per_item_failures=errors,
quarantined_count=quarantined,
dead_lettered_count=dead_lettered,
# #874 follow-up: the native path counted gated posts but
# never reported them, so DownloadDetailModal's "Tier-gated"
# field read 0 on every native walk while gallery-dl's read
# true. A paywalled creator was indistinguishable from a
# silent one.
tier_gated_count=gated_skipped,
),
)
@@ -464,6 +470,10 @@ class Ingester:
"errors": errors,
"quarantined": quarantined,
"posts": posts_processed,
# Ticks during the walk, not only at finalization: a
# deep backfill on a creator we've lost access to is
# otherwise a long run of zeros with no explanation.
"gated": gated_skipped,
})
if early_out:
@@ -32,6 +32,13 @@
v-if="e.live.errors" class="fc-active__count fc-active__count--err"
title="errors"
> {{ e.live.errors }}</span>
<!-- Tier-gated posts (#874): not an error the walk is working, the
content just isn't ours. Shown only when non-zero so a healthy
run stays uncluttered. -->
<span
v-if="e.live.gated" class="fc-active__count fc-active__count--gated"
title="posts skipped — no access at your tier"
>🔒 {{ e.live.gated }}</span>
<span class="fc-active__count fc-active__count--posts" title="posts scanned">
{{ e.live.posts }} posts</span>
</span>
@@ -141,6 +148,9 @@ function elapsed (startedIso) {
}
.fc-active__count { color: rgb(var(--v-theme-on-surface-variant)); }
.fc-active__count--err { color: rgb(var(--v-theme-error)); }
/* Matches FailingSourcesCard's severity map, which already colours
tier_limited as 'info' no-access is information, not a failure. */
.fc-active__count--gated { color: rgb(var(--v-theme-info)); }
.fc-active__count--posts { opacity: 0.7; }
@keyframes fc-active-pulse {
0%, 100% { opacity: 1; transform: scale(1); }
+50
View File
@@ -678,6 +678,56 @@ async def test_gated_post_skipped_entirely_no_media_no_record(
assert _count_ledger(sync_engine, source_id) == 2
@pytest.mark.asyncio
async def test_gated_posts_are_reported_in_run_stats(
source_id, sync_engine, tmp_path,
):
"""A gated post must be COUNTED, not just skipped.
`make_run_stats` has always declared `tier_gated_count` and
DownloadDetailModal has always rendered it, but the native path never
populated it — so a walk that skipped N paywalled posts reported 0, and a
creator we've lost access to looked exactly like a creator who stopped
posting. Asserted on the run_stats key the UI actually reads (not the
ingester's internal counter) so the guard tracks the property, not a name.
"""
gated = [(f"g{i}", [_media(f"g{i}", 1)]) for i in range(3)]
am = _media("open", 1)
client = _FakeClient(
[(None, [*gated, ("open", [am])])],
gated={"g0", "g1", "g2"},
)
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
result = ing.run(
source_id=source_id, campaign_id="c1", artist_slug="ingest",
url="https://patreon.com/ingest", mode="backfill",
)
assert result.run_stats["tier_gated_count"] == 3
# Not conflated with the other skip tiers: seen/on-disk skips are a
# different fact from "we aren't allowed to see this".
assert result.run_stats["skipped_count"] == 0
assert result.run_stats["downloaded_count"] == 1
@pytest.mark.asyncio
async def test_run_stats_reports_zero_gated_when_nothing_is_gated(
source_id, sync_engine, tmp_path,
):
"""The counter reports a real zero on a clean walk — so a 0 in the UI means
'nothing was gated', not 'this path never counted'."""
client = _FakeClient([(None, [("p1", [_media("p1", 1)])])])
ing = _ingester(sync_engine, tmp_path, client, _FakeDownloader(tmp_path))
result = ing.run(
source_id=source_id, campaign_id="c1", artist_slug="ingest",
url="https://patreon.com/ingest", mode="backfill",
)
assert result.run_stats["tier_gated_count"] == 0
@pytest.mark.asyncio
async def test_recapture_does_not_refetch_seen_media_missing_from_disk(
source_id, sync_engine, tmp_path,