1314 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
bvandeusenandClaude Fable 5	d50ebbad66	fix(mcp): classify list_shapes as read-only for read-scoped API keys ... The auth guard test caught it: a read-shaped tool in neither set fails closed for read keys, silently. list_shapes reads the ledger and writes nothing; classify_shapes stays write-scoped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 20:02:12 -04:00
bvandeusenandClaude Fable 5	942edd1eb5	feat(ledger): classify_shapes + list_shapes MCP tools; get_snippet carries the consumer map (#2789, milestone 294 step 3) ... The judgment write path. classify_shapes applies a batch of classifications to a project's live ledger rows — all-or-nothing (#2709's lesson: the whole batch is validated, write-ACL'd, and every snippet target proven readable before any row is touched); rows match by exact (path, symbol), kind narrows, and shapes no live row matches come back as 'unmatched' rather than errors. variant/exempt REQUIRE the reason — the why is the record (note 2786) — and 'unclassified' deliberately withdraws a judgment back to the todo. The 'via' channel is caller-restricted to agent|audit|import; hook and mechanical stay server-internal so a caller can't launder judgment as machinery. list_shapes is the todo query (status=unclassified) with composable filters: path is exact-or-under like recorded locations, snippet_id reads a consumer map, include_vanished reads history; paged with the true total. get_snippet now attaches and — the structured consumer map, filtered to projects the CALLER can read so a shared snippet never side- channels another project's file layout; attached only when non-empty (#2483). Integration tests pin the batch atomicity, ACL gates, filter composition, the consumer map on the MCP pull, and the SET NULL companion: a judgment whose snippet was purged rejoins the todo on the next sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:55:47 -04:00
bvandeusenandClaude Fable 5	9d92df2825	fix(tests): drop the pre-ledger line-ending assertion the step-2 edit missed ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:43:11 -04:00
bvandeusenandClaude Fable 5	9b1597a3c9	feat(ledger): coverage refresh feeds the shape ledger; the readout inverts to accounting (#2788, milestone 294 step 2) ... compute_coverage is now the ledger's sync point: every walk upserts the extracted shapes (new → unclassified, the todo state; surviving → last-seen bump; vanished → stamped, kept as history), re-files judgments whose snippet target went away, and mechanically stamps snippet reference locations as canonical — the one always-safe rule, self-healing only for its own stamps (an agent's judgment is never unwound by machinery). The covering predicate moves to shape_ledger.location_covers as the single home (match_shapes retired with its consumer); coverage's payload and line invert from 'N/M shapes recorded' to shape ACCOUNTING per note 2786: accounted/total with a canonical·instance·variant·exempt breakdown, and unclassified — THE todo — with its largest directories. Cache key bumps to v2 so pre-ledger blobs honestly read 'not measured yet' instead of rendering in a shape no longer spoken. Readout is deliberately project-wide (all repos' live rows), while the walk serves whichever repos the owner's keyring reaches this refresh. Integration tests pin the new contract: rows for every extracted shape, mechanical canonical stamps carrying snippet ids, idempotent recompute, agent judgments surviving recompute AND vanish/return, vanished rows leaving the readout but keeping their history. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:37:18 -04:00
bvandeusenandClaude Fable 5	19fdc9aa89	feat(ledger): code_shapes — the shape ledger schema (#2787, milestone 294 step 1) ... The accounting half of the pattern system (governing note 2786): the snippet library records canon (small), this table accounts for EVERY extracted shape (total). Identity is (project, repo_key, path, symbol, kind) — kind included because one file can define '.foo' (css) and 'foo' (sym) as distinct shapes. Status vocabulary: canonical / instance / variant / exempt / unclassified, with unclassified as the default and THE todo state; classifications carry who judged (agent|audit|hook|mechanical|import), when, and the why for variants/exemptions. first/last-seen commits + vanished_at keep history instead of deleting it; a rename reads as vanish+new (accepted for v1). snippet_id is SET NULL on snippet deletion so accounting rows outlive their target and rejoin the todo via the step-2 sync, never dangle silently. Backups: v7 carries code_shapes (judgment data, worth moving) — full and per-user export sections, and a restore that keeps a judgment only when its snippet survives the id re-mapping, downgrading to unclassified otherwise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 19:25:25 -04:00
bvandeusen	c1cb8fb183	Merge pull request 'Per-user forge connections — keyring, host-keyed resolution, project pin (#2778)' (#114) from dev into main	2026-08-19 15:24:47 -04:00
bvandeusenandClaude Fable 5	1faf8f3ece	feat(forge): per-user forge connections — keyring, host-keyed resolution, project pin (#2778) ... A forge token is a user's credential, not an instance's. The single admin-settings config is replaced by per-user keyring rows (one per forge host), and every server-side forge read runs on the PROJECT OWNER's keyring: - forge_connections table + projects.forge_connection_id pin (migration 0078, which also carries the existing admin config into the first admin's row and deletes the old setting keys — no legacy dual-read) - get_forge() replaced by get_forges(owner_id, project_id) -> ForgeSelector; resolve(repo) picks the connection whose host serves the repo. A pinned project uses ONLY its pinned connection; a stale pin (ownership moved) is ignored, never honored across users - env FORGE_* config survives as an implicit entry for admin owners only; a stored row for the same host beats it - consumers threaded: pull-time freshness (owner of the note), coverage (owner of the project), coverage routes' configured flag - routes: /api/settings/forge-connections CRUD + per-connection test (own-rows only, tokens never returned); /api/admin/forge shrinks to /api/admin/forge-webhook (secret only); PUT /api/projects/<id>/forge pins, owner-or-admin asking, owner's connections only - UI: Git Forges card moves to Settings -> Integrations as a connection list; webhook secret stays in the admin Config tab; owner-only forge select on the project coverage card - backups exclude forge_connections (credentials, api_keys precedent) and the pin, so restores fall back to keyring resolution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-19 11:23:22 -04:00
bvandeusen	5c14c2621c	Merge pull request 'Strict MCP arguments (#2709) + zero-Systems bootstrap ask (#2683)' (#113) from dev into main	2026-08-17 16:26:32 -04:00
bvandeusenandClaude Fable 5	7a5e2b18d9	feat(systems): evidence-carrying bootstrap ask for mature zero-Systems projects (#2683) ... The generic zero-state systems_hint never converts: identical on every record, maximal in scope, asked at wrap-up time — Minstrel reached 282 records with zero Systems while vocabularied projects grew organically. What converts is the project's own evidence at the moment of action. bootstrap_systems_ask (mcp/tools/systems.py) fires only in a project with >=20 records and no Systems: it names the record count and recent titles, and asks for a concrete deliverable — propose 3-6 Systems, confirm with the operator, create_system the set. Self-retiring: the first System ends it everywhere. Wired at both moments the task named: untagged_systems_hint escalates to it at write time, and enter_project carries it as systems_bootstrap at arrival (attached only when it applies). Young projects keep the mild question; populated vocabularies never pay the count query. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-17 14:53:46 -04:00
bvandeusenandClaude Fable 5	d6c9f08a59	fix(mcp): reject undeclared tool arguments instead of silently dropping them (#2709) ... FastMCP validates tool arguments with a pydantic model whose extra-field policy is 'ignore', so create_note(content=...) — a plausible near-miss for body=, primed by add_task_log's content — ran successfully, stored body: '', and left a record embedding/search cannot see. Two real notes were persisted body-less before the pattern was noticed; create_task only 'worked' because those calls happened to use the right name. StrictArgsFastMCP rejects any tool call carrying arguments the tool does not declare, before dispatch, with a did-you-mean hint when one is close and the declared list when none is. Applied at the dispatch seam so every tool gets the guarantee — an error the caller sees once beats data half-written forever. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-17 12:54:58 -04:00
bvandeusen	25411cf223	Merge pull request 'Edit-time record-sync nudge — the sync class (#2708)' (#112) from dev into main	2026-08-16 20:09:38 -04:00
bvandeusenandClaude Fable 5	3162332a13	feat(prior-art): edit-time record-sync nudge — the sync class (#2708) ... A snippet recorded AT the exact file being edited is not a reuse suggestion — it IS the record of the file being changed. The write-path hint now renders those as their own SYNC class: 'snippet #N records this file — updating the record is part of the edit (update_snippet / verify_snippet)'. Nearby and semantic hits stay the reuse menu. The two classes dedup on separate per-session channels (exclude_ids vs exclude_sync_ids, .ids vs .sync.ids in the hook), so a reuse hint shown early in a session can no longer silence the record-sync nudge when the recorded file itself is edited later. Sync surfacing is measured under its own note_usage source (write_path_sync) — its pull-through rate is the scoreboard for whether edit-time sync actually happens, per decision #2707 (no forge connection; records stay current in the session that has the context). Plugin 0.1.31. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 20:05:01 -04:00
bvandeusen	9f57cfad71	Merge pull request 'Milestone 288 — pattern library + forge integration' (#111) from dev into main	2026-08-16 16:37:12 -04:00
bvandeusenandClaude Fable 5	765635bbf2	feat(forge): GitHub adapter — second implementation keeps the seam a contract (#2693, milestone 288 step 8) ... ForgeAdapter is now a named base class carrying the shared plumbing (host join, error taxonomy, contents decoding, archive, default_branch, latest_commit); GiteaForge keeps its exact behavior and GitHubForge joins with the real differences: api.github.com / GHE /api/v3 host mapping, Bearer auth, a commits call for the provenance stamp (GitHub's contents payload only carries the blob sha), and the codeload tarball redirect. The contract grew latest_commit, and with it the cached-SHA short-circuit in pull-time freshness: a stored provenance commit that still heads the recorded path confirms 'current' without a content transfer — the economy that fits pulls inside GitHub's rate limits; every surprise falls back to the full fetch. Webhook deliveries now also accept X-Hub-Signature-256 (sha256=<hex>); the payload shape was already common. Settings card copy covers both forges' token scopes; the kind selector already flowed from the server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 16:18:16 -04:00
bvandeusenandClaude Fable 5	cbccb6bd5d	feat(coverage): pattern-library coverage measurement (#2692, milestone 288 step 7) ... Server-side shape enumeration per bound repo — one archive download via the forge adapter, definitions extracted with a Python mirror of the write-path hook's awk rules (shared test vectors pin the two together) — compared against recorded snippet locations by path+symbol. Summary is cached in the settings KV with a freshness stamp; recomputed on webhook push (spawned off the delivery path) or explicit refresh, never in a request path. Surfaces: GET/POST /api/projects/<id>/coverage[/refresh], a project-page card (estimate-labeled, largest-gaps chips), and a one-line evidence-carrying entry in enter_project read from cache only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 16:05:43 -04:00
bvandeusenandClaude Fable 5	89b07f7857	feat(forge): push webhook flags drift at the moment the repo moves (#2691) ... Second adapter consumer. POST /api/webhooks/forge validates Gitea's X-Gitea-Signature (HMAC-SHA256, constant-time; no secret configured = the endpoint 404s out of existence), extracts changed/removed paths, and flags matched snippets by writing verification.invalidated_by {commit_sha, at, path, removed} — the existing attention vocabulary extended, not a new flag: needs_attention includes it, both filter dialects (Python + jsonpath SQL) include it in 'attention' and exclude it from 'ok', and recording ANY fresh verdict clears it by construction because compose_verification builds a new dict. Unverified snippets are skipped (already in their own bucket); replayed deliveries at the same head commit are no-ops; processing failures return 200 with a WARNING + AppLog canary so the forge never marks deliveries failed and operators never disable the hook over a transient (#2663's lesson). Matching goes through repo BINDINGS: recorded location repos are free-form names ('Scribe') that cannot address a forge, so a snippet reaches its forge repo through its project's binding — which also fixes step 5's pull-time resolution for every real record via the same fallback. O(bindings + snippets-in-project + changed files). Settings: webhook secret beside the forge config (masked, sentinel- skipped, Docker-secret env channel, endpoint documented in the UI). Tests: signature gate, payload parsing, path semantics, both filter dialects extended in the drift-check guard file, and real-Postgres end-to-end (flag lands, attention lists it, replay quiet, re-verify clears, unbound repo untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 13:05:00 -04:00
bvandeusenandClaude Fable 5	eb760eb440	fix(snippets): read the stored provenance before writing the fresh stamp into the response ... fields aliases data['snippet'], so stamping the response first made the staleness check compare the new stamp to itself — the background persist never fired. Caught by test_current_code_confirms_and_refreshes_provenance on run 3811, which exists for exactly this write. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:53:34 -04:00
bvandeusenandClaude Fable 5	2fce57847b	feat(snippets): pull-time freshness — the forge confirms the cache at the moment it's trusted (#2690) ... First consumer of the forge adapter. attach_live_body decorates both pull surfaces (MCP get_snippet, REST detail) with body_source + body_freshness when the instance has a forge: 'current' means the cached code was just found verbatim (whitespace-normalized, the same normalization the verdict hash uses) in the fetched file, and provenance restamps to the file's last commit — reflected in the response and persisted in the background. A snippet body is a FRAGMENT of its file, so a fetch can honestly CONFIRM the cache or report divergence, never clobber the record with the whole file: 'diverged' is the reader's information, and a 404 stamps the mechanically-true 'missing' verdict into the existing attention state — once, not on every pull of an already-flagged record. The probe never raises and never blocks past 2.5s (tighter than the adapter's own timeout — the pull is where a session decides whether pulling is worth it, #2663's finding); a hung forge costs bounded time and the cache serves. A no-forge instance's response stays byte-identical to today's (rule #115 baseline, pinned by test). services/background.py is the new one home for fire-and-forget tasks with strong references (the #2663 GC footgun) — telemetry's two copies predate it and keep their bespoke canaries; new callers use this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:46:45 -04:00
bvandeusenandClaude Fable 5	13e428c596	feat(forge): adapter seam + Gitea implementation — optional read access to the operator's forge (#2689) ... Step 4 of milestone 288 (decision #2686). services/forge.py defines the contract steps 5-7 consume — read_file (content + last_commit_sha, the provenance stamp), default_branch, resolve_repo, check — with GiteaForge as the first implementation over the REST contents/repo/version/user endpoints. Repo identity reuses normalize_repo_key: the host segment selects whether this forge serves a recorded repo, the remainder is the API path, so no new identity scheme exists. Read-only by construction; errors never carry the token; first outbound-HTTP timeout convention (5s total, no retries — the consumer's fallback is the retry policy). OPTIONAL per instance (rule #115): get_forge() returns None when unconfigured and every consumer treats None as today's behavior. Config lives in admin settings (Settings → Config → Git Forge: kind/base URL/token, save + test-connection probe reporting version + identity), with FORGE_* env / Docker-secret fallbacks; DB wins so a UI edit can't silently lose to an env var. Token treatment follows the smtp_password convention (masked on read, mask-sentinel skipped on write, absent from audit details) — and wiring it surfaced that the generic GET/PUT /api/settings dump bypassed that masking for the owning admin's raw KV rows, so secret keys are now masked there too (fixes the same exposure for smtp_password). Contract tests run against httpx.MockTransport as the fake forge — the reference behaviors the GitHub adapter (step 8) must reproduce — plus the off-by-default gate, partial-config-is-off, env-vs-DB precedence, and route/mask structural checks. Also: the step-2 definition detector learned to skip dunders after flagging __init__ as 'already defined in 4 files' on this step's own build — guaranteed noise for a hint that must stay trustworthy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:37:27 -04:00
bvandeusenandClaude Fable 5	7d26a3fc6a	fix(tests): provenance itest user fixture is get-or-create — the lane DB persists across tests ... Both integration tests built the same username; the second insert died on users_username_key. 19 passed, 1 error on run 3802. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:06:19 -04:00
bvandeusenandClaude Fable 5	1e7f66e72d	feat(snippets): body provenance — the cache-with-provenance half of the pointer model (#2688) ... Decision #2686: the recorded location is the source of truth for a snippet's code; the stored body is a cache of it. data.provenance now records what the cache is a cache OF — commit_sha + fetched_at — as a carried JSONB field following the verification precedent, so no migration is needed and absence keeps today's exact semantics. The rules: provenance follows the code (fresh SHA restamps it, a code edit without one drops it, a metadata edit carries it); writes ABOUT the code carry it — record_verification rebuilds data from scratch and would otherwise erase it silently; an ok verdict at a known commit restamps it, since the checker just proved the cache matches the source there. verify_snippet verdicts also record the commit they ran at, making "the repo moved on since the check" computable once the forge integration lands. create/update/verify MCP tools take commit_sha (git rev-parse HEAD — free for any session with a checkout). Unit tests pin the compose/carry logic; real-Postgres integration tests run create→verify→update end-to-end (#2663: DB paths get no mocked-only coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:00:40 -04:00
bvandeusenandClaude Fable 5	8407368c0c	fix(hooks): definition detector covers all code, not a language shortlist (#2682) ... ARM 1 extracted definitions with patterns for CSS/JS/TS/Python only — the languages of the repo it was born in — so the local duplication proof, and the #2664 record nudge gated on it, were structurally unreachable in Go/Kotlin/Rust projects (Minstrel, FabledExchange): precisely where recording was observed never to happen. One modifier-strip plus a definition-keyword family (func/fun/fn/function/ def/sub, struct/trait/interface/enum/object/protocol/type, plus Go method receivers) now covers them all; impl is excluded because several impl blocks per type is normal Rust, and keyword-less declarations (C/Java/Dart) are documented out of scope. Grep patterns mirror the same forms so hits are definitions, never call sites. Parameterized tests pin the coverage per language family. Plugin 0.1.30. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 11:33:24 -04:00
bvandeusenandClaude Fable 5	2d58e74ec7	feat(reuse): recording model becomes the pattern library — every shape at first build (#2687) ... Decision #2686: snippets are the project's pattern library, not a dedup net. The floor, the reusing-code skill, and the snippet tool docstrings now state the proactive model — record every shape the first time it is built, with no will-it-recur judgment, and start later instances from the recorded shape; second-copy consolidation stays as the backstop. The floor guard test pins all three elements (tool, first-build trigger, backstop) so the model cannot silently regress to the reactive wording. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 11:33:24 -04:00
bvandeusen	e05a660471	Merge pull request 'Usage telemetry alive — readout fix, canaries, and the snippet-recording seam' (#110) from dev into main	2026-08-14 22:36:43 -04:00
bvandeusenandClaude Fable 5	4107b17727	fix(telemetry): usage readout grouped by a rebuilt CASE — group by the label instead (#2663 root cause) ... CI's new integration tests reproduced the outage on a clean database and named the half: the writes land fine, and usage_for_notes fails on EVERY call. The GROUP BY rebuilt the ambient case() expression, and asyncpg's expanding IN-parameters give each instance its own bind names — so Postgres sees a SELECT expression the GROUP BY doesn't cover and rejects the query with a GroupingError, which the old code swallowed into zeros. One labelled expression, grouped by its label. The deployed table has been accumulating events all along; history appears as soon as this deploys. Also: the two hook-execution tests now run with the real PATH and skip when the hook's tools are absent (the restricted-PATH convention next door is for silence contracts, where empty-for-the-wrong-reason still passes) — and the unit lane installs jq so 'skip' never quietly becomes 'nowhere'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:49:48 -04:00
bvandeusenandClaude Fable 5	fb757fb4ba	feat(reuse): recording gets a seam — the prior-art hook asks for create_snippet when duplication is proven and unrecorded (#2664) ... Zero snippets were ever recorded outside sessions already thinking about snippets: the read side had a real seam (the PreToolUse hook) and the record side had a trailing clause of a floor bullet. The trigger moment — 'I just wrote the second copy' — is mid-Write/Edit, so the nudge now rides the same hook: when the local arm proves the definition exists elsewhere in the repo AND Scribe returned no record of it, the context block asks for create_snippet. Both gates or silence, so a brand-new helper and an already-recorded one stay nudge-free. Floor bullet promoted to name the trigger moments (extract, hoist, second copy); guarded by test the same way the Systems reflex is. Plugin 0.1.29 so the cache picks up the hook (#2209). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:45:32 -04:00
bvandeusenandClaude Fable 5	77acee9239	fix(telemetry): usage counters get canaries, task references, and the missing integration tests (#2663) ... The deployed instance ran with every usage counter at zero while surfacing demonstrably fired. Every unit test was green because every unit test mocked either _schedule or the session — the two functions that touch the database ran against real Postgres nowhere. Both telemetry writers also held no reference to their fire-and-forget tasks (the loop keeps only weak ones), and swallowed every failure into logger.debug, so a total outage was indistinguishable from an unused corpus. - note_usage + retrieval_telemetry keep strong task references until done - failures log at WARNING; note_usage additionally drops one AppLog error row per process per site, so the admin UI shows the outage without host access - integration tests cover _insert_events -> usage_for_notes and the full record_pulled chain on a running loop, splitting the write and read halves so a failure names its side Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:45:32 -04:00
bvandeusen	5503580875	Merge pull request 'Process contracts — invoke by name, compose with the conversation, author as a shape' (#109) from dev into main	2026-08-10 09:51:47 -04:00
bvandeusenandClaude Fable 5	5cb7cfe706	feat(processes): authoring contract — a process is a shape, never a script ... Operator's generalization of #2582: the benefit of nearly every stored process is the accumulated SHAPE — steps, taxonomy, quality bar — and a process must not force anything or overwrite the intent of the request that invoked it. create_process now states the authoring side of the composition contract: no embedded approach mandates, no pre-granted approvals, clarify steps seed from the conversation instead of re-asking it. (All three stored processes were also audited: DRY Pass and Rulebook Review were already propose-approve-apply with no forcing language; both gained the seeded-clarify line — data changes, live already.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 09:40:23 -04:00
bvandeusenandClaude Fable 5	9c68faa0bd	feat(processes): composition contract — the process is the skeleton, the conversation supplies the parameters ... Extends #2582's fix per the operator's direction: a stored Process must incorporate the context it was triggered with, not displace it. No new plumbing needed — the live context is already in the session; what was missing is the stated contract. get_process now carries it: live constraints/scope/focus fold into the procedure and override its defaults where they disagree; a clarify step asks only what the conversation has NOT already answered (confirm interpretations, don't re-ask); stated concerns become lenses the procedure applies, not text it discards. The sync-generated skill stubs state the short form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 08:39:02 -04:00
bvandeusenandClaude Fable 5	66c4fd1b88	fix(sync): process-skill triggers invoke by name only — resemblance offers, never substitutes ... Scribe issue #2582: the sync wrote "Use when {title}-type work is requested" into every generated process skill, so a bespoke live prompt that merely resembled a stored Process got the Process's canned procedure — and inherited approvals embedded in its body (the Drift Audit's fan-out opt-in) that the conversation never granted. Observed live: an explicit systems-review request with its own constraints was answered with the Drift Audit clarify menu recommending ~15 parallel auditors. New trigger contract, both variants: follow verbatim only when invoked by name; on mere resemblance the live instructions govern — offer the process, ask before following, and never inherit embedded approvals the operator hasn't granted in this conversation. (The Drift Audit process record itself was also rewritten to a sequential in-context walk — data change, live already, recorded in the same issue.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 08:33:40 -04:00
bvandeusen	266d8b8117	Merge pull request 'The Systems question rides every read and write — unified seam, create_system dedup gate, sweeps-are-discovery prose' (#107) from dev into main	2026-08-09 16:04:58 -04:00
bvandeusenandClaude Fable 5	75bdb9c148	test(systems): fix two mock bugs in the seam tests — unset .id on the fake, shared to_dict dict leaking across creates ... Runs 3562/3565: _fake_system never sets .id so the dup-gate assertion compared a MagicMock to 7; and a shared to_dict return_value dict let the tagged create's mutation leak into the orphan create's response. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:55:56 -04:00
bvandeusenandClaude Fable 5	3455f9cb9a	refactor(systems): one seam — the Systems question rides every read and write of a record ... Scribe issue #2570, from the operator's challenge: the invariant is "always be asking whether what you're touching is a System's territory and whether the work is filed there" — not a nudge in one corner. The prior shape failed it twice: the hint fired only on creates (with a special-cased zero-Systems branch), and get_task/get_note returned records WITHOUT their Systems, so the read-side reflex had nothing to fire on (same per-kind asymmetry as #2481). - attach_systems(): single helper used by get/create/update for tasks, notes, and snippets, plus add_task_log. Tagged records always show `systems`; an untagged project record carries the `systems_hint` question instead. Neither field attaches empty (#2483). Hint is owner-only; everything fail-open (#2109). - untagged_systems_hint unified to ONE question — the vocabulary listing varies, the question doesn't; the zero-Systems branch stops being special text. - Docstrings state the uniform contract; floor prose now names the read-side reflex (systems visible -> list_system_records the pile). - Plugin 0.1.27 -> 0.1.28. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:48:41 -04:00
bvandeusenandClaude Fable 5	56b1952de5	fix(systems): sweeps are the discovery moment — zero-Systems hint, create_system dedup gate, prose inverted ... First real-world test of the #2562 fixes (Scribe issue #2569): a Forge session ran a whole-codebase audit and created zero Systems — endorsed by the shipped guidance, whose "no particular area takes none" clause read as an exemption for exactly the record type that enumerates the subsystem vocabulary. And the systems_hint was silent for a zero-Systems project, the one state nothing else nudges out of. - systems_hint gains a zero-Systems branch: prompt the FIRST create_system instead of going quiet. - create_system is duplicate-gated like the other creates (normalized name, archived included, fail-open) — liberal creation becomes safe by construction, so the guidance can stop preaching restraint. - Prose inverted on every surface (hint text, create_system docstring, floor bullet, using-scribe step 7): audits/sweeps take several tags and mint the Systems they name; the gate is the guardrail against sprawl, not holding back; only a record genuinely about no particular area goes untagged. - Plugin 0.1.26 -> 0.1.27. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:35:59 -04:00
bvandeusen	e80f2f233b	Merge pull request 'Instruction delivery fits the fold — 2k server map, floor Systems reflex, write-time systems_hint' (#106) from dev into main	2026-08-09 13:11:20 -04:00
bvandeusenandClaude Fable 5	3ff8803593	feat(instructions): fit the delivery fold — 2k server map, floor Systems reflex, write-time systems_hint ... Claude Code injects only the first ~2,048 chars of an MCP server's instructions and silently cuts the rest mid-word (#2562, observed live): _INSTRUCTIONS was 20,002 chars, so ~90% — including all Systems tagging guidance — never reached any session. Rearchitect delivery around what each surface actually delivers: - _INSTRUCTIONS becomes a 1,997-char purpose-sorted map, with a header comment stating the budget and where detail belongs instead. - Tool docstrings keep the per-tool HOW (audit: nearly all displaced topics were already duplicated there); backfill the four gaps — enter_project session scoping + project bootstrap, create_rule entity-vs-rule test, create_design_system not-a-rulebook, create_system two-records test. - The plugin static context (the delivery floor) gains the tag-to-Systems reflex and a surfaces-layering statement; plugin 0.1.25 -> 0.1.26 so the executing cache refreshes (#2209). - create_task / create_note / create_snippet return a systems_hint when a record is created untagged in a project that has Systems — in-band at the exact write it applies to, fail-open like the dedup gate. - Guards: _INSTRUCTIONS length budget, floor-states-the-reflex, and a displaced-topics sweep asserting every cut topic still lives on a delivered surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 12:53:35 -04:00
bvandeusen	854aae252a	Merge pull request 'Per-kind duplicate-report floors — notes/tasks default 0.93' (#105) from dev into main	2026-08-09 10:37:06 -04:00
bvandeusenandClaude Fable 5	272b7dbddf	feat(dedup): per-kind duplicate-report floors — notes/tasks default 0.93 ... At chunk grain (#280) a note-pair's similarity is its closest chunk pair, so the shared 0.82 floor saturated the note/task reports with related families (38 note / 155 task groups against the 200-pair cap, measured 2026-08-09). Split kb_duplicate_threshold into per-kind settings keys with per-kind defaults: snippet 0.82 (single-chunk, scale unchanged), note/task 0.93 (points the report at genuinely-alike records). Settings UI grows the two new knobs; report entrypoints inherit the change via get_duplicate_threshold(user_id, kind). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-09 10:33:11 -04:00
bvandeusen	063494094a	Merge pull request 'Chunked embeddings — no record content invisible to search (#280)' (#104) from dev into main	2026-08-08 23:54:34 -04:00
bvandeusenandClaude Fable 5	23cb396e65	fix(recurrence): drop the now-unused child binding ... The spawn's explicit embed call went with #280 step 3; the create_note result had no other reader. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:52:33 -04:00
bvandeusenandClaude Fable 5	041d8defbc	feat(embeddings): best-chunk-per-note on every retrieval surface (#280 step 4) ... A note's relevance is now its best chunk's similarity, everywhere: - semantic_search_notes keeps the indexed raw-distance top-k and over-fetches chunk rows (x4, composing with the x3 supersession over-fetch), then collapses to first-appearance-per-note — rows arrive distance-ordered, so first is best. Every ranked consumer (MCP/REST search, Browse, auto-inject, write-path, gate) inherits through the one function. - list_notes semantic q swaps its join for a correlated MIN-distance subquery — the join would have repeated a long note once per matching chunk and made total count chunks. - the duplicate report groups its self-join by note pair on MIN(distance): pair similarity = closest chunk pair, and the < join now also drops cross-chunk self-pairs that would flag every long note against itself. - the write gate queries once per chunk of the candidate (capped at 8), so a note duplicating an existing record in ONE SECTION is caught — the whole-document query diluted exactly the section that mattered. Integration test now seeds a two-chunk note and pins the collapse against real pgvector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:51:01 -04:00
bvandeusenandClaude Fable 5	0e70a3896b	feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3) ... note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus chunk_text (what this vector actually encodes) and chunker_version. Migration 0077 clears the table — embeddings are derived (0067 precedent) and the old whole-document rows are indistinguishable from single-chunk notes, so the startup backfill regenerates the corpus at the new shape. The backfill is now version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe. upsert_note_embedding takes (title, body) and chunks internally — one path for the write path, the recurrence spawn and the backfill. The recurrence spawn's own embed call is deleted outright: create_note already embeds via embed_note (#2056), so the spawn was a second copy of the rule. An emptied record now CLEARS its stale vectors instead of leaving them findable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:47:34 -04:00
bvandeusenandClaude Fable 5	6b5043a69c	feat(embeddings): chunk_document — the chunked document shape (#280 step 1) ... bge-small reads 512 tokens and fastembed truncates silently, so a single whole-document vector permanently lost everything past ~400 words. The new shape: split at markdown headings (fence-aware), merge small sections, split oversize ones at paragraph boundaries, title-anchor every chunk, repeat the section heading on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical title\nbody shape, so the corpus's sharpest records are byte-for-byte unaffected. CHUNKER_VERSION added so later shape changes re-embed by version comparison instead of a table wipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:43:51 -04:00
bvandeusen	875f6a7b5e	Queue clear — shared ACL on lists, semantic q, full telemetry ledger, --color-* retired (#103)	2026-08-08 22:45:26 -04:00
bvandeusen	4ba544e2af	refactor(theme): retire the --color-* shim — the sweep it promised, run ... #2533. theme.css claimed "removing this block is a rename sweep across the components, tracked separately" — written in 67a529a, never filed, which made the comment itself an instance of the survey's presence-without-reference pattern. This is that sweep. 73 alias declarations deleted; 69 files rewritten; every --color-*-style name now references its --fs-* token directly. Mechanical by construction: the map IS the alias block, applied longest-name-first with a boundary guard so --color-text never matched inside --color-text-muted. Zero survivors outside theme.css, verified by grep rather than assumed. One deliberate survivor: --color-shadow stays DECLARED, because it was never an alias — it is a literal value the design system has no token for. Marked in place as a recorded gap: promote it to an --fs-* token when a second app needs it, don't copy the line. Nothing is lost mode-wise: the aliases' resolve-at-use-time trick (which absorbed 48 dark-mode overrides) lives one layer down in the --fs-* tokens' own derivations, which is why the sweep is a pure rename. Both CSS checkers green. Why now rather than never: check_snippets_against_design_system reports every --color-* reference as "unknown — renders as NOTHING", and nine recipe snippets recorded from components.css carried the deprecated names, making them prior art pointing the wrong way. With the sweep in, the checker's report over re-recorded snippets should be EMPTY — the acceptance test that proves the checker was right all along (#2517's correction). Refs #2533	2026-08-08 22:42:37 -04:00
bvandeusen	2d1e26f38f	feat(telemetry): ambient surfacings count, apart — enter_project and the skill sync emit ... #2477, option (a) as decided, with the readout changed in the same commit. ## The two silent surfaces enter_project returns open tasks + recent notes on every project entry — probably the largest surfacing by volume — and emitted nothing, so the pulls it caused floated unattributed and the surfaced:pulled ratio ran against a denominator missing its biggest contributor. Now source "enter_project". build_process_manifest installs every reachable Process as an auto-surfacing skill on the operator's machine — its own docstring calls it the most consequential passive surface Scribe has — and emitted nothing, so a Process matched on every relevant turn and never opened was indistinguishable from one never installed. Now source "process_skill_sync": the honest event is "installed", which is a surfacing in effect since the description sits in front of the model each session. ## The readout, same commit — the condition option (a) carried Both surfaces are AMBIENT: top-N-by-recency and install-everything are not ranked choices. Pooling them into surfaced_count would make a note's number dominated by "recently updated in a project you opened", and dead-weight detection would read that as popularity — the wrong number read confidently, which is the corrupts-data tier the survey ranked above everything else. So usage_for_notes splits: surfaced_count stays RANKED-ONLY (every existing consumer's reading — "surfaced often, never pulled → dead weight" — keeps meaning what it meant), and ambient_count is new. Classified in SQL via a CASE on AMBIENT_SOURCES so the group count stays three rows per note, not one per distinct source. Pulls stay pooled: "did anyone ever open this?" does not depend on how it was found. #1038 and #2085 read agent pulls and ranked surfacings; both are unaffected by ambient volume, which is the point. Refs #2477	2026-08-08 22:39:13 -04:00
bvandeusen	9b3874b657	test: the auto-inject path now logs two retrievals, and that is the point ... 45ba4aa made the reuse slot log its query (source: reuse_slot). The margin-gate test pinned record_retrieval to exactly one call, which was asserting the very asymmetry #2463 fixed — the displaced hit logged, the displacing query not. Assert both sources in order instead. Refs #2463	2026-08-08 22:32:58 -04:00
bvandeusen	45ba4aab25	fix(telemetry): the two invisible retrievals log, and /api/search takes scope ... #2463, the remaining findings (finding 3 landed with f11a547). ## The reuse slot logs (source: reuse_slot) A real semantic query competing for an auto-inject menu slot, and the ledger was asymmetric: the scored hit it DISPLACED was in retrieval_logs, the query that displaced it was not — so the slot could never be evaluated against what it replaced. #1038 and #2085 are gated on this ledger being complete. ## Browse search logs (source: browse_search) The human's main search surface, and it logged nothing — so retrieval_logs claimed the web's search was /api/search. Measured while fixing: /api/search has ZERO frontend consumers; the web UI searches through /api/knowledge exclusively. The table wasn't just under-describing the UI, it was describing a surface the UI never touches. The task's check — does folding human queries into the corpus skew the precision signal thresholds are tuned against? — is answered by the source column: distinct values (browse_search, reuse_slot beside the existing four) mean tuning includes or excludes human traffic deliberately rather than by accident. Same resolution as the mcp_/rest_ split in note_usage_events. ## /api/search takes project_id (and logs it) The route logged project_id=None unconditionally while the MCP tool insists the agent pass the active project. Now optional, default global: with no frontend consumer, this route serves API callers, and an API caller states its scope explicitly — the default-scope UI decision the task flagged is moot until a UI actually consumes the route, which is recorded rather than guessed. Refs #2463	2026-08-08 20:00:22 -04:00
bvandeusen	f11a547cd2	fix(lists): shared-project records appear, and a list's q means what search means ... #2462, both decided halves. ## The ACL defect list_notes filtered on Note.user_id == user_id with no scope parameter at all — never a per-call decision, the capability was absent. query_knowledge beside it was deliberately browse-scoped with a comment saying why. So a task in a shared project was invisible in list_tasks, enter_project's open-task list, the SessionStart todo count and the web UI's task views, while the same project's notes and snippets appeared. Now the shared clause: notes_visibility_clause(user_id, "browse"). Browse and not read, per decision #2094 — an ambient list must never surface a record someone shared one-to-one; those stay search-only. This is the remaining half of a fix made twice before (#2159 widened fetch, #2092 widened meaning), and the guard below is what stops a fourth half appearing. Guarded by source inspection in test_retrieval_scopes: every list-shaped service references a shared visibility clause AND carries no bare Note.user_id comparison that would quietly re-narrow it. An owner-only list returns correct-looking rows and simply omits the shared ones — the shape no behavioural test catches. ## q is semantic (operator: "make it match") The UI's note list keyword-matched while the UI's Browse search semantic-matched, over the same records, with nothing saying so. Now one meaning: q joins the embedding index and orders by cosine distance at the interactive floor, with every lifecycle filter still applied in the same indexed query. Relevance ordering wins over `sort` when q is present — a query is a relevance claim, and sorting its results by date would shuffle the answer. ILIKE survives only as the embedder-down fallback: degraded, never empty. Stated position: superseded records are NOT demoted in list-q. The penalty reorders a top-k; reordering a paginated, counted list would make page boundaries lie. The search surfaces carry the demotion. ## The interactive floor becomes one constant INTERACTIVE_SEARCH_THRESHOLD = 0.3 in embeddings.py, consumed by routes/search.py (was a commented constant), knowledge.py (was a bare literal), and the new list-q. This closes #2463's finding 3 early — the same number lived in two files with the reasoning attached to only one. ## Deferred, deliberately The return-shape unification (ORM objects vs dicts) stays undone. It is a 14-caller refactor whose motivation — callers being unable to swap paths — shrinks now that both paths share the clause and the meaning of q. If swap pressure recurs it deserves its own change, not a rider on an ACL fix. Refs #2462, #2463	2026-08-08 19:58:01 -04:00