Milestone 288 — pattern library + forge integration #111

Merged

bvandeusen merged 10 commits from dev into main 2026-08-16 16:37:12 -04:00

Conversation 0 Commits 10 Files Changed 30

+3475 -77

bvandeusen commented 2026-08-16 16:37:07 -04:00

Owner

Copy Link

Copy Source

All 8 steps of milestone 288 (decision #2686), each CI-green on its head SHA (runs 3782–3819):

1. Instruction surfaces — all-shapes recording doctrine in hook context, reusing-code skill, and MCP tool docstrings (2d58e74)
2. All-code detector — the prior-art hook's local arm covers every keyword-announced definition family, not a language shortlist (8407368)
3. Provenance — snippets carry the commit their code was confirmed at; the stamp lives and dies with the code (1e7f66e, 7d26a3f)
4. Forge adapter — optional read-only Gitea integration + admin settings card with test button; secrets follow the smtp_password convention and the /api/settings leak is fixed (13e428c)
5. Pull-time freshness — every snippet pull reports current/diverged/missing/unreachable against the source, budget-bounded, never clobbering (2fce578, eb760eb)
6. Push webhook — /api/webhooks/forge flags exactly the verified snippets a push touched into the attention queue; clears on re-verify by construction (89b07f7)
7. Coverage measurement — recorded shapes vs. shapes in the bound repo, estimate-labeled: project-page card + one line in enter_project; cached, recomputed on push or refresh (cbccb6b)
8. GitHub adapter — second ForgeAdapter implementation (github.com + GHE), X-Hub-Signature-256 webhook mapping, cached-SHA short-circuit for rate limits (765635b)

Everything is optional per instance (rule #115): a forge-less install is byte-identical to before. Plugin version bumps to 0.1.30.

🤖 Generated with Claude Code

All 8 steps of milestone 288 (decision #2686), each CI-green on its head SHA (runs 3782–3819): 1. **Instruction surfaces** — all-shapes recording doctrine in hook context, reusing-code skill, and MCP tool docstrings (2d58e74) 2. **All-code detector** — the prior-art hook's local arm covers every keyword-announced definition family, not a language shortlist (8407368) 3. **Provenance** — snippets carry the commit their code was confirmed at; the stamp lives and dies with the code (1e7f66e, 7d26a3f) 4. **Forge adapter** — optional read-only Gitea integration + admin settings card with test button; secrets follow the smtp_password convention and the /api/settings leak is fixed (13e428c) 5. **Pull-time freshness** — every snippet pull reports current/diverged/missing/unreachable against the source, budget-bounded, never clobbering (2fce578, eb760eb) 6. **Push webhook** — `/api/webhooks/forge` flags exactly the verified snippets a push touched into the attention queue; clears on re-verify by construction (89b07f7) 7. **Coverage measurement** — recorded shapes vs. shapes in the bound repo, estimate-labeled: project-page card + one line in enter_project; cached, recomputed on push or refresh (cbccb6b) 8. **GitHub adapter** — second ForgeAdapter implementation (github.com + GHE), X-Hub-Signature-256 webhook mapping, cached-SHA short-circuit for rate limits (765635b) Everything is optional per instance (rule #115): a forge-less install is byte-identical to before. Plugin version bumps to 0.1.30. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

bvandeusen added 10 commits 2026-08-16 16:37:07 -04:00

feat(reuse): recording model becomes the pattern library — every shape at first build (#2687) ... 2d58e74ec7

Decision #2686: snippets are the project's pattern library, not a dedup
net. The floor, the reusing-code skill, and the snippet tool docstrings
now state the proactive model — record every shape the first time it is
built, with no will-it-recur judgment, and start later instances from
the recorded shape; second-copy consolidation stays as the backstop.
The floor guard test pins all three elements (tool, first-build trigger,
backstop) so the model cannot silently regress to the reactive wording.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(hooks): definition detector covers all code, not a language shortlist (#2682) ... 8407368c0c

ARM 1 extracted definitions with patterns for CSS/JS/TS/Python only —
the languages of the repo it was born in — so the local duplication
proof, and the #2664 record nudge gated on it, were structurally
unreachable in Go/Kotlin/Rust projects (Minstrel, FabledExchange):
precisely where recording was observed never to happen. One
modifier-strip plus a definition-keyword family (func/fun/fn/function/
def/sub, struct/trait/interface/enum/object/protocol/type, plus Go
method receivers) now covers them all; impl is excluded because several
impl blocks per type is normal Rust, and keyword-less declarations
(C/Java/Dart) are documented out of scope. Grep patterns mirror the
same forms so hits are definitions, never call sites. Parameterized
tests pin the coverage per language family. Plugin 0.1.30.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(snippets): body provenance — the cache-with-provenance half of the pointer model (#2688) ... 1e7f66e72d

Decision #2686: the recorded location is the source of truth for a
snippet's code; the stored body is a cache of it. data.provenance now
records what the cache is a cache OF — commit_sha + fetched_at — as a
carried JSONB field following the verification precedent, so no
migration is needed and absence keeps today's exact semantics.

The rules: provenance follows the code (fresh SHA restamps it, a code
edit without one drops it, a metadata edit carries it); writes ABOUT
the code carry it — record_verification rebuilds data from scratch and
would otherwise erase it silently; an ok verdict at a known commit
restamps it, since the checker just proved the cache matches the source
there. verify_snippet verdicts also record the commit they ran at,
making "the repo moved on since the check" computable once the forge
integration lands. create/update/verify MCP tools take commit_sha
(git rev-parse HEAD — free for any session with a checkout).

Unit tests pin the compose/carry logic; real-Postgres integration tests
run create→verify→update end-to-end (#2663: DB paths get no
mocked-only coverage).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(tests): provenance itest user fixture is get-or-create — the lane DB persists across tests ... 7d26a3fc6a

Both integration tests built the same username; the second insert died
on users_username_key. 19 passed, 1 error on run 3802.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(forge): adapter seam + Gitea implementation — optional read access to the operator's forge (#2689) ... 13e428c596

Step 4 of milestone 288 (decision #2686). services/forge.py defines the
contract steps 5-7 consume — read_file (content + last_commit_sha, the
provenance stamp), default_branch, resolve_repo, check — with GiteaForge
as the first implementation over the REST contents/repo/version/user
endpoints. Repo identity reuses normalize_repo_key: the host segment
selects whether this forge serves a recorded repo, the remainder is the
API path, so no new identity scheme exists. Read-only by construction;
errors never carry the token; first outbound-HTTP timeout convention
(5s total, no retries — the consumer's fallback is the retry policy).

OPTIONAL per instance (rule #115): get_forge() returns None when
unconfigured and every consumer treats None as today's behavior. Config
lives in admin settings (Settings → Config → Git Forge: kind/base
URL/token, save + test-connection probe reporting version + identity),
with FORGE_* env / Docker-secret fallbacks; DB wins so a UI edit can't
silently lose to an env var. Token treatment follows the smtp_password
convention (masked on read, mask-sentinel skipped on write, absent from
audit details) — and wiring it surfaced that the generic GET/PUT
/api/settings dump bypassed that masking for the owning admin's raw KV
rows, so secret keys are now masked there too (fixes the same exposure
for smtp_password).

Contract tests run against httpx.MockTransport as the fake forge — the
reference behaviors the GitHub adapter (step 8) must reproduce — plus
the off-by-default gate, partial-config-is-off, env-vs-DB precedence,
and route/mask structural checks. Also: the step-2 definition detector
learned to skip dunders after flagging __init__ as 'already defined in
4 files' on this step's own build — guaranteed noise for a hint that
must stay trustworthy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(snippets): pull-time freshness — the forge confirms the cache at the moment it's trusted (#2690) ... 2fce57847b

First consumer of the forge adapter. attach_live_body decorates both
pull surfaces (MCP get_snippet, REST detail) with body_source +
body_freshness when the instance has a forge: 'current' means the
cached code was just found verbatim (whitespace-normalized, the same
normalization the verdict hash uses) in the fetched file, and
provenance restamps to the file's last commit — reflected in the
response and persisted in the background. A snippet body is a FRAGMENT
of its file, so a fetch can honestly CONFIRM the cache or report
divergence, never clobber the record with the whole file: 'diverged'
is the reader's information, and a 404 stamps the mechanically-true
'missing' verdict into the existing attention state — once, not on
every pull of an already-flagged record.

The probe never raises and never blocks past 2.5s (tighter than the
adapter's own timeout — the pull is where a session decides whether
pulling is worth it, #2663's finding); a hung forge costs bounded time
and the cache serves. A no-forge instance's response stays
byte-identical to today's (rule #115 baseline, pinned by test).

services/background.py is the new one home for fire-and-forget tasks
with strong references (the #2663 GC footgun) — telemetry's two copies
predate it and keep their bespoke canaries; new callers use this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(snippets): read the stored provenance before writing the fresh stamp into the response ... eb760eb440

fields aliases data['snippet'], so stamping the response first made the
staleness check compare the new stamp to itself — the background persist
never fired. Caught by test_current_code_confirms_and_refreshes_provenance
on run 3811, which exists for exactly this write.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(forge): push webhook flags drift at the moment the repo moves (#2691) ... 89b07f7857

Second adapter consumer. POST /api/webhooks/forge validates Gitea's
X-Gitea-Signature (HMAC-SHA256, constant-time; no secret configured =
the endpoint 404s out of existence), extracts changed/removed paths,
and flags matched snippets by writing verification.invalidated_by
{commit_sha, at, path, removed} — the existing attention vocabulary
extended, not a new flag: needs_attention includes it, both filter
dialects (Python + jsonpath SQL) include it in 'attention' and exclude
it from 'ok', and recording ANY fresh verdict clears it by construction
because compose_verification builds a new dict. Unverified snippets are
skipped (already in their own bucket); replayed deliveries at the same
head commit are no-ops; processing failures return 200 with a WARNING +
AppLog canary so the forge never marks deliveries failed and operators
never disable the hook over a transient (#2663's lesson).

Matching goes through repo BINDINGS: recorded location repos are
free-form names ('Scribe') that cannot address a forge, so a snippet
reaches its forge repo through its project's binding — which also fixes
step 5's pull-time resolution for every real record via the same
fallback. O(bindings + snippets-in-project + changed files).

Settings: webhook secret beside the forge config (masked, sentinel-
skipped, Docker-secret env channel, endpoint documented in the UI).
Tests: signature gate, payload parsing, path semantics, both filter
dialects extended in the drift-check guard file, and real-Postgres
end-to-end (flag lands, attention lists it, replay quiet, re-verify
clears, unbound repo untouched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(coverage): pattern-library coverage measurement (#2692, milestone 288 step 7) ... cbccb6bd5d

Server-side shape enumeration per bound repo — one archive download via the
forge adapter, definitions extracted with a Python mirror of the write-path
hook's awk rules (shared test vectors pin the two together) — compared
against recorded snippet locations by path+symbol. Summary is cached in the
settings KV with a freshness stamp; recomputed on webhook push (spawned off
the delivery path) or explicit refresh, never in a request path.

Surfaces: GET/POST /api/projects/<id>/coverage[/refresh], a project-page
card (estimate-labeled, largest-gaps chips), and a one-line evidence-carrying
entry in enter_project read from cache only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(forge): GitHub adapter — second implementation keeps the seam a contract (#2693, milestone 288 step 8) ... 765635bbf2

ForgeAdapter is now a named base class carrying the shared plumbing
(host join, error taxonomy, contents decoding, archive, default_branch,
latest_commit); GiteaForge keeps its exact behavior and GitHubForge joins
with the real differences: api.github.com / GHE /api/v3 host mapping,
Bearer auth, a commits call for the provenance stamp (GitHub's contents
payload only carries the blob sha), and the codeload tarball redirect.

The contract grew latest_commit, and with it the cached-SHA short-circuit
in pull-time freshness: a stored provenance commit that still heads the
recorded path confirms 'current' without a content transfer — the economy
that fits pulls inside GitHub's rate limits; every surprise falls back to
the full fetch. Webhook deliveries now also accept X-Hub-Signature-256
(sha256=<hex>); the payload shape was already common. Settings card copy
covers both forges' token scopes; the kind selector already flowed from
the server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

bvandeusen merged commit 9f57cfad71 into main 2026-08-16 16:37:12 -04:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate-bot bvandeusen

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#111