feat: the membership sweep, and the state that makes its failures readable (387 C3)
CI / lint (push) Failing after 3s
Build images / sign-extension (push) Successful in 4s
CI / extension-version (push) Successful in 2s
Build images / build-agent (push) Successful in 8s
CI / frontend-build (push) Successful in 28s
CI / backend-lint-and-test (push) Successful in 38s
Build images / build-web (push) Successful in 1m28s
Build images / smoke-web (push) Skipped
Build images / build-ml (push) Successful in 2m35s
Build images / promote (push) Skipped
CI / integration (push) Successful in 2m55s

A daily sweep that walks each platform's roster into `platform_membership`.
Daily because memberships change on a BILLING cycle, not a download cadence.

**`membership_sync` is the part that earns its keep.** Without it three very
different situations are one indistinguishable state — the account subscribes
to nothing, the sweep never ran, the sweep failed — and all three leave zero
rows in `platform_membership`. "You are tracking 12 sources you no longer
subscribe to" is correct in the first case and an invitation to cancel things
the operator is actively paying for in the other two. So C4 gates its
CONCLUSIONS on `last_success_at`, not merely its display, and `roster_is_fresh`
is computed server-side so no caller can forget to.

Two timestamps rather than one: `last_attempt_at` moves every run,
`last_success_at` only on a clean walk. The gap between them is the signal —
a sweep hammering a broken credential every day must not look healthy because
it ran recently, and there is a test for exactly that.

Rejected shortcuts, both tempting: `MAX(platform_membership.last_seen_at)`
cannot tell "synced fine, found nothing" from "never synced"; `task_run` is
worse, since its retention prunes ok rows after 24h and a sweep that last
succeeded three days ago would leave no trace at all.

**The fetch completes before anything is written.** That ordering is the safety
property: a walk that dies mid-pagination writes nothing, so a failure can
never leave a roster half this week's and half last week's. `touch_membership`
never deletes, so a failure cannot empty the roster either — but "intact"
should mean intact, not merely non-empty.

Rule 89's four, each where it actually lives: recovery is "run it again"
(upsert, no deletes); retention is C1's age-out-never-delete, because
disappearing IS the signal; the wall-clock deadline is per-platform and
distinct from the per-REQUEST timeout the client already has (rule 156 — a
paginated roster answering every page slowly-but-within-timeout would never
trip that one and would sit on a worker indefinitely); duration comes from the
existing TaskRun signal plumbing.

**A bug caught in review, not production:** the broad `except Exception` would
have swallowed Celery's SoftTimeLimitExceeded — which is an ORDINARY Exception
subclass, not a BaseException — letting the sweep run past the soft limit into
the hard one, where it is SIGKILLed mid-transaction. A sweep that cannot be
stopped is worse than one that fails. Now re-raised explicitly, with a test
that also asserts SoftTimeLimitExceeded is still an Exception, so the re-raise
cannot quietly become dead code.

Rule 164 is why this ships with UI rather than backend-only: a roster that
never synced must be VISIBLE as such. The card says "never synced" in words and
states no count at all — rendering it as 0 is the precise conflation the whole
step exists to prevent — while a real zero behind a real sync is reported as
zero, because that one IS an answer. Pinned in both directions.

Three independent gates decide whether a platform is swept — registered here,
client exposes `iter_memberships`, credential exists — each silent, so adding
SubscribeStar (D1) is one line and nothing else. A missing credential is not an
error: recording a failure would light up the UI for a feature never enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNXXULQDjVZmbuNa2G9mD9
This commit is contained in:
2026-09-10 23:40:48 -04:00
co-authored by Claude Opus 5
parent afcde8e457
commit 751e7ddb9f
12 changed files with 946 additions and 4 deletions
+57 -2
View File
@@ -1,10 +1,11 @@
"""FC-3a: CRUD over Source rows. FC-3c adds POST /<id>/check."""
from quart import Blueprint, jsonify, request
from sqlalchemy import select
from sqlalchemy import func, select
from ..extensions import get_session
from ..models import DownloadEvent, Source
from ..models import DownloadEvent, MembershipSync, PlatformMembership, Source
from ..services.membership_roster import roster_is_fresh
from ..services.scheduler_service import active_platform_cooldowns, scheduler_status
from ..services.source_service import (
KNOWN_PLATFORMS,
@@ -288,3 +289,57 @@ async def check_source(source_id: int):
download_source.delay(source_id)
return jsonify({"download_event_id": event_id, "status": "pending"}), 202
# --- #387 C3: the membership roster's sync state --------------------------
#
# Rule 164's visibility requirement lives here. A roster that failed to sync,
# or never has, must be DISTINGUISHABLE from an account that subscribes to
# nothing — otherwise the reconciliation this unlocks would tell the operator
# to cancel sources they are actively paying for.
@sources_bp.route("/membership-sync", methods=["GET"])
async def membership_sync_status():
async with get_session() as session:
rows = (await session.execute(select(MembershipSync))).scalars().all()
counts = dict(
(await session.execute(
select(PlatformMembership.platform, func.count())
.group_by(PlatformMembership.platform)
)).all()
)
return jsonify({"platforms": [
{
"platform": r.platform,
"last_attempt_at": r.last_attempt_at.isoformat() if r.last_attempt_at else None,
# NULL here means NEVER, and the UI must say so in words. Rendering
# it as 0 or as "-" is the exact conflation this endpoint exists to
# prevent.
"last_success_at": r.last_success_at.isoformat() if r.last_success_at else None,
"last_count": r.last_count,
"last_error_type": r.last_error_type,
"last_error_message": r.last_error_message,
# Whether a CONCLUSION may be drawn from this roster — not merely
# whether it looks recent. C4 gates on this, and it is computed
# server-side so no caller can forget to.
"fresh": roster_is_fresh(r),
"known_memberships": counts.get(r.platform, 0),
}
for r in sorted(rows, key=lambda r: r.platform)
]})
@sources_bp.route("/membership-sync", methods=["POST"])
async def trigger_membership_sync():
"""Run the roster sweep now.
The beat schedule runs daily, which is right for a billing-cycle fact but
far too slow when the operator has just connected a credential and wants to
see whether it works. Queued rather than run inline: it crosses the network
to an external service and the request path is not where that belongs.
"""
from ..tasks.maintenance import sync_memberships
sync_memberships.delay()
return jsonify({"queued": True})