feat(ledger): coverage self-seeds — enter_project background refresh + refresh_pattern_coverage tool + shape-accounting skill (#2802, milestone 294)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Failing after 33s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 23s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Failing after 33s
CI & Build / Build & push image (push) Skipped
The UI Refresh button must not be the only seed path (operator directive, hit live: the P7 backfill stalled waiting for a click). Three parts: - enter_project fire-and-forgets refresh_if_stale on the project OWNER — absent or day-old readouts recompute in the background (same spawn the webhook path uses), the enter stays fast, forge-less owners exit quietly (rule #115 baseline), and an in-flight guard keeps concurrent enters from fetching the same tarball N times. - refresh_pattern_coverage(project_id): the synchronous agent-facing form — write-gated, owner-keyring resolution, and ValueError messages that name the fix (add a connection / bind_repo) instead of measuring nothing silently. - plugin 0.1.33 ships the shape-accounting skill: the five statuses, the seed/todo/judge loop, and the derive-first rule, triggered by the coverage line or any proved code-to-canon relationship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -26,6 +26,7 @@ from scribe.services import projects as projects_svc
|
||||
from scribe.services import rulebooks as rulebooks_svc
|
||||
from scribe.services import systems as systems_svc
|
||||
from scribe.services import trash as trash_svc
|
||||
from scribe.services.background import spawn
|
||||
from scribe.services.note_usage import record_surfaced
|
||||
|
||||
|
||||
@@ -68,7 +69,9 @@ async def enter_project(project_id: int) -> dict:
|
||||
unclassified, largest: internal/api". Unclassified IS the todo: as you
|
||||
touch code in those areas, classify the shapes you can (instances of
|
||||
recorded canon, deliberate variants, one-off exemptions) and record the
|
||||
canon that's missing with create_snippet.
|
||||
canon that's missing with create_snippet. A null line on a forge-served
|
||||
project usually means the ledger is seeding in the background (entering
|
||||
triggers it); refresh_pattern_coverage computes it on the spot.
|
||||
|
||||
`systems` is the project's vocabulary of named subsystems/areas. It is
|
||||
returned here so you can TAG as you write: when creating or meaningfully
|
||||
@@ -164,6 +167,16 @@ async def enter_project(project_id: int) -> dict:
|
||||
coverage = await coverage_svc.cached_coverage(
|
||||
project.user_id or uid, project_id
|
||||
)
|
||||
# Arrival self-seed (#2802): a ledger that is absent or stale refreshes in
|
||||
# the BACKGROUND — entering is the moment the number is wanted, and the
|
||||
# UI button must not be the only path. This enter stays fast; the next
|
||||
# one carries the line. Forge-less projects exit the seed quietly.
|
||||
spawn(
|
||||
coverage_svc.refresh_if_stale(
|
||||
project.user_id or uid, project_id, cached=coverage
|
||||
),
|
||||
site="enter_project.coverage_seed",
|
||||
)
|
||||
|
||||
out = {
|
||||
"project": project.to_dict(),
|
||||
|
||||
@@ -9,6 +9,7 @@ refresh; these tools only ever judge what the sync has seen.
|
||||
from __future__ import annotations
|
||||
|
||||
from scribe.mcp._context import current_user_id
|
||||
from scribe.services import coverage as coverage_svc
|
||||
from scribe.services import shape_ledger as shape_ledger_svc
|
||||
|
||||
|
||||
@@ -90,6 +91,34 @@ async def list_shapes(
|
||||
return {"shapes": [r.to_dict() for r in rows], "total": total}
|
||||
|
||||
|
||||
async def refresh_pattern_coverage(project_id: int) -> dict:
|
||||
"""Seed or refresh the project's shape ledger NOW, and return the readout.
|
||||
|
||||
The synchronous form of the arrival-moment background seed (#2802):
|
||||
downloads the bound repos through the OWNER's forge connections, upserts
|
||||
every extracted shape into the ledger (new shapes arrive `unclassified`),
|
||||
re-stamps snippet reference locations as canonical, and recomputes the
|
||||
accounting. Reach for it when the ledger must be current before you act —
|
||||
a classification batch about to run, a coverage question asked directly —
|
||||
rather than waiting on the background seed an enter_project triggers.
|
||||
|
||||
Takes seconds, not milliseconds (it moves repo archives). Requires write
|
||||
access to the project. Errors name the fix: no forge connection → the
|
||||
owner adds one (Settings → Integrations → Git Forges); no served repo →
|
||||
bind_repo on a host a connection serves.
|
||||
|
||||
Returns the accounting payload — total, accounted, counts by status,
|
||||
unclassified, repos, largest_gaps — plus `pattern_coverage`, the same
|
||||
one-line summary enter_project carries.
|
||||
"""
|
||||
uid = current_user_id()
|
||||
coverage = await coverage_svc.refresh_for_caller(uid, project_id)
|
||||
return {
|
||||
"pattern_coverage": coverage_svc.coverage_line(coverage),
|
||||
**coverage,
|
||||
}
|
||||
|
||||
|
||||
def register(mcp) -> None:
|
||||
for fn in (classify_shapes, list_shapes):
|
||||
for fn in (classify_shapes, list_shapes, refresh_pattern_coverage):
|
||||
mcp.tool(name=fn.__name__)(fn)
|
||||
|
||||
@@ -31,7 +31,7 @@ import logging
|
||||
import posixpath
|
||||
import re
|
||||
import tarfile
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from scribe.services.forge import ForgeSelector, get_forges
|
||||
from scribe.services.repo_bindings import keys_for_project
|
||||
@@ -323,6 +323,90 @@ async def refresh_coverage(
|
||||
return coverage
|
||||
|
||||
|
||||
# The arrival-moment self-seed (#2802). A ledger that has never been computed
|
||||
# used to wait for someone to find the UI Refresh button; entering the project
|
||||
# is the moment the number is wanted, so entering is the moment it seeds.
|
||||
_SEED_MAX_AGE = timedelta(hours=24)
|
||||
|
||||
# Projects with a refresh already running — concurrent enters must not fetch
|
||||
# the same tarball N times. In-process on purpose: the cost being bounded is
|
||||
# per-process forge traffic, and a rare double-fetch across workers is
|
||||
# harmless (the upsert is idempotent).
|
||||
_inflight_seed: set[int] = set()
|
||||
|
||||
|
||||
async def refresh_if_stale(
|
||||
user_id: int, project_id: int, cached: dict | None = None
|
||||
) -> None:
|
||||
"""Background-refresh a project's ledger when its readout is absent or
|
||||
older than a day. Fire-and-forget material (background.spawn): every exit
|
||||
is quiet, every failure a WARNING — a seed must never surface as an
|
||||
enter_project error. ``user_id`` is the project OWNER (whose keyring and
|
||||
cache this is); pass ``cached`` when the caller already read it.
|
||||
"""
|
||||
try:
|
||||
if cached is None:
|
||||
cached = await cached_coverage(user_id, project_id)
|
||||
if cached:
|
||||
try:
|
||||
computed = datetime.fromisoformat(cached.get("computed_at") or "")
|
||||
if datetime.now(timezone.utc) - computed < _SEED_MAX_AGE:
|
||||
return
|
||||
except ValueError:
|
||||
pass # an unreadable stamp reads as stale
|
||||
if project_id in _inflight_seed:
|
||||
return
|
||||
selector = await get_forges(user_id, project_id)
|
||||
if not selector.configured:
|
||||
return # rule #115: forge-less stays exactly as it was
|
||||
_inflight_seed.add(project_id)
|
||||
try:
|
||||
await refresh_coverage(user_id, project_id, selector=selector)
|
||||
finally:
|
||||
_inflight_seed.discard(project_id)
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"background coverage seed failed for project %s", project_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
async def refresh_for_caller(caller_id: int, project_id: int) -> dict:
|
||||
"""The explicit agent-facing refresh (#2802) — synchronous, named errors.
|
||||
|
||||
Write-gated: recomputing spends the owner's forge API budget and rewrites
|
||||
ledger rows' seen-markers, so a read share doesn't grant it. Resolution
|
||||
runs on the OWNER's keyring like every other forge read. Raises
|
||||
ValueError with a fixable message instead of silently measuring nothing —
|
||||
the caller is an agent mid-task, and "None" would strand it exactly the
|
||||
way the button-only path did.
|
||||
"""
|
||||
from scribe.models import async_session
|
||||
from scribe.models.project import Project
|
||||
from scribe.services import access
|
||||
|
||||
if not await access.can_write_project(caller_id, project_id):
|
||||
raise ValueError(f"project {project_id} not found or no write access")
|
||||
async with async_session() as session:
|
||||
project = await session.get(Project, project_id)
|
||||
if project is None:
|
||||
raise ValueError(f"project {project_id} not found")
|
||||
owner_id = project.user_id or caller_id
|
||||
selector = await get_forges(owner_id, project_id)
|
||||
if not selector.configured:
|
||||
raise ValueError(
|
||||
"No forge connection serves this project — the owner adds one "
|
||||
"under Settings → Integrations → Git Forges"
|
||||
)
|
||||
coverage = await refresh_coverage(owner_id, project_id, selector=selector)
|
||||
if coverage is None:
|
||||
raise ValueError(
|
||||
"No bound repo is served by the owner's forge connections — "
|
||||
"bind_repo the project's repo on a host a connection serves"
|
||||
)
|
||||
return coverage
|
||||
|
||||
|
||||
async def cached_coverage(user_id: int, project_id: int) -> dict | None:
|
||||
"""The last computed summary, or None — never computes."""
|
||||
raw = await get_setting(user_id, f"{_CACHE_KEY_PREFIX}{project_id}", "")
|
||||
|
||||
Reference in New Issue
Block a user