feat: say why the posts are invisible, without ever deciding they are (387 C5)
CI / lint (push) Successful in 3s
CI / extension-version (push) Successful in 3s
Build images / sign-extension (push) Successful in 4s
Build images / build-agent (push) Successful in 7s
CI / frontend-build (push) Successful in 21s
CI / backend-lint-and-test (push) Failing after 32s
Build images / build-web (push) Successful in 58s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 1m47s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m16s

A3 made a tier-gated source say "47 posts you can't see". The roster turns that into a reason: the membership ended, or the tier doesn't reach these posts, or it's a free follow. Rendered under A3's count in the health tooltip, quieter than the count it explains.

FREE is a fourth case the step didn't enumerate, and it earns its own sentence. has_paid_access collapses "former patron" and "current free follower" to the same False, so deriving the reason from that boolean would tell a free follower "you're not a patron any more" - a false statement about a state they were never in. gated_reason reads the status axis first, calling has_paid_access with is_free_member forced off, then splits on the free flag.

Silence is the default, and there are four ways into it: campaign absent from the roster, roster stale, platform never swept, status word not yet characterised. All four send null and the count stands alone. The frontend has no fallback sentence either - a default would turn "we don't know why" into a reason, which is the one thing this step must not do.

The line that must not be crossed is pinned structurally rather than by inspection: test_no_fetch_path_can_read_the_roster walks the transitive first-party imports from the fetch roots and asserts the roster is unreachable. FC runs no local verification (rule 85), so a guard cannot be falsified by hand before it lands - it carries two positive controls instead, proving the walker finds roster imports that ARE there, one direct and one through a hop, so the real assertion can never pass merely because the walk resolved nothing.

C4's identity loop moved to membership_roster.pair_sources_with_memberships when C5 became its second caller; two copies would let the Subscriptions row and the reconciliation card disagree about which creator a source IS. Three test files were each building PlatformMembership rows with their own drifting helper - consolidated into tests/roster_builders.py, same family as issue 3109.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHQB1YukL3VyvMK8rcbmV9
This commit is contained in:
2026-09-11 22:59:29 -04:00
co-authored by Claude Opus 5
parent de11c14448
commit aa765f0a72
9 changed files with 688 additions and 53 deletions
+7 -15
View File
@@ -49,7 +49,7 @@ from .membership_roster import (
get_sync_state,
has_paid_access,
identity_keys_for_source,
match_kind,
pair_sources_with_memberships,
roster_is_fresh,
url_tail,
)
@@ -113,20 +113,12 @@ async def reconcile(
.where(Source.platform == platform)
)).all()
# Nested loop rather than a SQL join: the match is a predicate over a JSON
# blob and a derived URL handle, neither of which is indexable, and both
# sides are tens of rows on any real library. Keeping it in Python means ONE
# definition of identity (`match_kind`) instead of a second one in SQL that
# could drift from it.
pairs: dict[int, tuple[PlatformMembership, str]] = {}
matched_membership_ids: set[int] = set()
for source, _artist in rows:
for m in memberships:
kind = match_kind(source, m)
if kind:
pairs[source.id] = (m, kind)
matched_membership_ids.add(m.id)
break
# The join itself lives in `membership_roster` beside `match_kind`, so C5's
# gated-reason annotation pairs sources with memberships by exactly the same
# rule this card sorts them by. Two copies would let the Subscriptions row
# and this card disagree about which creator a source IS.
pairs = pair_sources_with_memberships([s for s, _a in rows], memberships)
matched_membership_ids = {m.id for m, _kind in pairs.values()}
subscribed_not_tracked = []
for m in memberships:
+121
View File
@@ -352,6 +352,127 @@ def match_kind(source: Source, membership: PlatformMembership) -> str | None:
return None
def pair_sources_with_memberships(
sources: list[Source], memberships: list[PlatformMembership],
) -> dict[int, tuple[PlatformMembership, str]]:
"""Source id -> the membership it is the same creator as, and how we know.
Extracted from C4's reconcile loop when C5 became its second caller. It is
a nested loop rather than a SQL join because the match is a predicate over
a JSON blob and a derived URL handle, neither of which is indexable, and
both sides are tens of rows on any real library. Keeping it in Python means
ONE definition of identity (`match_kind`) instead of a second one in SQL
that could drift from it.
First match wins, which is `match_kind`'s ordering doing its job: a source
with a cached campaign id can only pair with the membership holding that
id, so an ambiguous handle never outvotes it.
"""
pairs: dict[int, tuple[PlatformMembership, str]] = {}
for source in sources:
for m in memberships:
kind = match_kind(source, m)
if kind:
pairs[source.id] = (m, kind)
break
return pairs
# ---------------------------------------------------------------------------
# Why the posts are invisible (#387 C5)
# ---------------------------------------------------------------------------
#
# A3 made a tier-gated source say "47 posts you can't see". These are the words
# the roster is allowed to add to that count — and ONLY to that count.
#
# THE LINE: the roster ANNOTATES the gated flag, it never produces it.
# `current_user_can_view` (read per post by `patreon_client.post_is_gated`) is
# the authoritative per-post signal, and entitled-tier data cannot stand in for
# it — a creator can gate a post behind an access rule that maps onto no tier
# name at all. So nothing here may suppress a download, skip a walk, or decide
# a post is inaccessible. It explains a skip that ALREADY happened. Getting
# that backwards would make FC silently stop fetching content the operator is
# paying for, which is the worst failure available in this milestone.
# `test_no_fetch_path_can_read_the_roster` pins that structurally.
GATED_LAPSED = "lapsed" # the membership ended — resubscribe, or disable
GATED_TIER = "tier" # paying, but this tier doesn't reach these posts
GATED_FREE = "free" # a current FREE follow — nobody is paying for access
def gated_reason(
platform: str, status: str | None, *, is_free_member: bool = False,
) -> str | None:
"""Why a tier-gated source's posts are out of reach, if the roster knows.
None means "no words beyond the count" and is the answer for every case
where the roster is not evidence: a status this code has not been taught,
and (at the call site) a campaign absent from the roster or a roster too
stale to trust. Absence is not evidence — the same discipline as
`test_post_is_gated_only_on_explicit_false`.
`is_free_member` is read AFTER the status axis, not folded into it, which
is why `has_paid_access` is called here with it forced off. The two axes
are independent in Patreon's payload, and collapsing them loses a real
distinction: a current free follower has not lost anything, so telling them
"you're not a patron any more" would be a false sentence about a state they
were never in.
"""
by_status = has_paid_access(platform, status, is_free_member=False)
if by_status is None:
return None
if not by_status:
return GATED_LAPSED
return GATED_FREE if is_free_member else GATED_TIER
async def gated_reasons_for_sources(
session: AsyncSession, sources: list[Source], *, now: datetime | None = None,
) -> dict[int, str]:
"""The reason word for each of these sources, where the roster has one.
Callers pass ONLY the sources already known to be tier-gated: the question
"why can't I see these posts" is meaningless for a source whose posts are
all visible, and asking it anyway would put roster data on rows that have
no gated state for it to annotate.
Sources with no entry in the result get A3's bare count, which is the
correct degraded rendering for all three of: platform never swept, roster
stale, campaign not in the roster.
"""
if not sources:
return {}
platforms = {s.platform for s in sources}
# Per platform, because freshness is per platform: a working Patreon sweep
# must not lend its credibility to a SubscribeStar roster that has never
# run. Same gate as C4's `tracked_not_subscribed`, for the same reason.
fresh = {
p for p in platforms
if roster_is_fresh(await get_sync_state(session, p), now=now)
}
if not fresh:
return {}
memberships = (await session.execute(
select(PlatformMembership).where(
PlatformMembership.platform.in_(sorted(fresh))
)
)).scalars().all()
pairs = pair_sources_with_memberships(
[s for s in sources if s.platform in fresh], memberships,
)
reasons: dict[int, str] = {}
for source_id, (m, _kind) in pairs.items():
reason = gated_reason(
m.platform, m.status,
is_free_member=bool((m.details or {}).get("is_free_member")),
)
if reason is not None:
reasons[source_id] = reason
return reasons
async def source_for_membership(
session: AsyncSession, membership: PlatformMembership,
) -> Source | None:
+19 -6
View File
@@ -19,6 +19,7 @@ from ..models import (
)
from .db_helpers import failing_sources_clause
from .gallery_dl import ErrorType
from .membership_roster import gated_reasons_for_sources
from .platforms import known_platform_keys
from .scheduler_service import compute_next_check_at
@@ -92,6 +93,12 @@ class SourceRecord:
# source, so it is joined in by `list()` only — None everywhere else, which
# the UI renders as the bare no-access state with no fabricated number.
tier_gated_count: int | None = None
# Milestone #387 C5: WHY those posts are out of reach, when the learned
# roster can say — "lapsed" / "tier" / "free", or None for "no words beyond
# the count". Joined in by `list()` beside the count, and for the same
# reason: it annotates a gated state rather than producing one. Nothing on
# a fetch path may read it (see `membership_roster.gated_reason`).
gated_reason: str | None = None
def to_dict(self) -> dict:
return {
@@ -116,6 +123,7 @@ class SourceRecord:
"backfill_recapture": self.backfill_recapture,
"backfill_posts": self.backfill_posts,
"tier_gated_count": self.tier_gated_count,
"gated_reason": self.gated_reason,
}
@@ -243,6 +251,7 @@ class SourceService:
def _build_record(
self, source: Source, artist: Artist, settings: ImportSettings,
gated_counts: dict[int, int] | None = None,
gated_reasons: dict[int, str] | None = None,
) -> SourceRecord:
nxt = compute_next_check_at(source, artist, settings)
co = source.config_overrides or {}
@@ -268,6 +277,7 @@ class SourceService:
backfill_recapture=bool(co.get("_backfill_recapture")),
backfill_posts=int(co.get("_backfill_posts", 0)),
tier_gated_count=(gated_counts or {}).get(source.id),
gated_reason=(gated_reasons or {}).get(source.id),
)
async def _row_to_record(self, source: Source) -> SourceRecord:
@@ -302,12 +312,15 @@ class SourceService:
stmt = stmt.order_by(Artist.name.asc(), Source.id.asc())
rows = (await self.session.execute(stmt)).all()
settings = await self._load_settings()
# Only tier-gated rows need the join — on a healthy library that is an
# empty list and _tier_gated_counts short-circuits without a query.
gated_counts = await self._tier_gated_counts(
[s.id for s, _a in rows if s.error_type == ErrorType.TIER_LIMITED]
)
return [self._build_record(s, a, settings, gated_counts) for s, a in rows]
# Only tier-gated rows need either join — on a healthy library that is
# an empty list and both helpers short-circuit without a query.
gated = [s for s, _a in rows if s.error_type == ErrorType.TIER_LIMITED]
gated_counts = await self._tier_gated_counts([s.id for s in gated])
gated_reasons = await gated_reasons_for_sources(self.session, gated)
return [
self._build_record(s, a, settings, gated_counts, gated_reasons)
for s, a in rows
]
async def get(self, source_id: int) -> SourceRecord | None:
source = (await self.session.execute(