251 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Fable 5	3162332a13	feat(prior-art): edit-time record-sync nudge — the sync class (#2708) ... A snippet recorded AT the exact file being edited is not a reuse suggestion — it IS the record of the file being changed. The write-path hint now renders those as their own SYNC class: 'snippet #N records this file — updating the record is part of the edit (update_snippet / verify_snippet)'. Nearby and semantic hits stay the reuse menu. The two classes dedup on separate per-session channels (exclude_ids vs exclude_sync_ids, .ids vs .sync.ids in the hook), so a reuse hint shown early in a session can no longer silence the record-sync nudge when the recorded file itself is edited later. Sync surfacing is measured under its own note_usage source (write_path_sync) — its pull-through rate is the scoreboard for whether edit-time sync actually happens, per decision #2707 (no forge connection; records stay current in the session that has the context). Plugin 0.1.31. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 20:05:01 -04:00
bvandeusenandClaude Fable 5	765635bbf2	feat(forge): GitHub adapter — second implementation keeps the seam a contract (#2693, milestone 288 step 8) ... ForgeAdapter is now a named base class carrying the shared plumbing (host join, error taxonomy, contents decoding, archive, default_branch, latest_commit); GiteaForge keeps its exact behavior and GitHubForge joins with the real differences: api.github.com / GHE /api/v3 host mapping, Bearer auth, a commits call for the provenance stamp (GitHub's contents payload only carries the blob sha), and the codeload tarball redirect. The contract grew latest_commit, and with it the cached-SHA short-circuit in pull-time freshness: a stored provenance commit that still heads the recorded path confirms 'current' without a content transfer — the economy that fits pulls inside GitHub's rate limits; every surprise falls back to the full fetch. Webhook deliveries now also accept X-Hub-Signature-256 (sha256=<hex>); the payload shape was already common. Settings card copy covers both forges' token scopes; the kind selector already flowed from the server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 16:18:16 -04:00
bvandeusenandClaude Fable 5	cbccb6bd5d	feat(coverage): pattern-library coverage measurement (#2692, milestone 288 step 7) ... Server-side shape enumeration per bound repo — one archive download via the forge adapter, definitions extracted with a Python mirror of the write-path hook's awk rules (shared test vectors pin the two together) — compared against recorded snippet locations by path+symbol. Summary is cached in the settings KV with a freshness stamp; recomputed on webhook push (spawned off the delivery path) or explicit refresh, never in a request path. Surfaces: GET/POST /api/projects/<id>/coverage[/refresh], a project-page card (estimate-labeled, largest-gaps chips), and a one-line evidence-carrying entry in enter_project read from cache only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 16:05:43 -04:00
bvandeusenandClaude Fable 5	89b07f7857	feat(forge): push webhook flags drift at the moment the repo moves (#2691) ... Second adapter consumer. POST /api/webhooks/forge validates Gitea's X-Gitea-Signature (HMAC-SHA256, constant-time; no secret configured = the endpoint 404s out of existence), extracts changed/removed paths, and flags matched snippets by writing verification.invalidated_by {commit_sha, at, path, removed} — the existing attention vocabulary extended, not a new flag: needs_attention includes it, both filter dialects (Python + jsonpath SQL) include it in 'attention' and exclude it from 'ok', and recording ANY fresh verdict clears it by construction because compose_verification builds a new dict. Unverified snippets are skipped (already in their own bucket); replayed deliveries at the same head commit are no-ops; processing failures return 200 with a WARNING + AppLog canary so the forge never marks deliveries failed and operators never disable the hook over a transient (#2663's lesson). Matching goes through repo BINDINGS: recorded location repos are free-form names ('Scribe') that cannot address a forge, so a snippet reaches its forge repo through its project's binding — which also fixes step 5's pull-time resolution for every real record via the same fallback. O(bindings + snippets-in-project + changed files). Settings: webhook secret beside the forge config (masked, sentinel- skipped, Docker-secret env channel, endpoint documented in the UI). Tests: signature gate, payload parsing, path semantics, both filter dialects extended in the drift-check guard file, and real-Postgres end-to-end (flag lands, attention lists it, replay quiet, re-verify clears, unbound repo untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 13:05:00 -04:00
bvandeusenandClaude Fable 5	2fce57847b	feat(snippets): pull-time freshness — the forge confirms the cache at the moment it's trusted (#2690) ... First consumer of the forge adapter. attach_live_body decorates both pull surfaces (MCP get_snippet, REST detail) with body_source + body_freshness when the instance has a forge: 'current' means the cached code was just found verbatim (whitespace-normalized, the same normalization the verdict hash uses) in the fetched file, and provenance restamps to the file's last commit — reflected in the response and persisted in the background. A snippet body is a FRAGMENT of its file, so a fetch can honestly CONFIRM the cache or report divergence, never clobber the record with the whole file: 'diverged' is the reader's information, and a 404 stamps the mechanically-true 'missing' verdict into the existing attention state — once, not on every pull of an already-flagged record. The probe never raises and never blocks past 2.5s (tighter than the adapter's own timeout — the pull is where a session decides whether pulling is worth it, #2663's finding); a hung forge costs bounded time and the cache serves. A no-forge instance's response stays byte-identical to today's (rule #115 baseline, pinned by test). services/background.py is the new one home for fire-and-forget tasks with strong references (the #2663 GC footgun) — telemetry's two copies predate it and keep their bespoke canaries; new callers use this. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:46:45 -04:00
bvandeusenandClaude Fable 5	13e428c596	feat(forge): adapter seam + Gitea implementation — optional read access to the operator's forge (#2689) ... Step 4 of milestone 288 (decision #2686). services/forge.py defines the contract steps 5-7 consume — read_file (content + last_commit_sha, the provenance stamp), default_branch, resolve_repo, check — with GiteaForge as the first implementation over the REST contents/repo/version/user endpoints. Repo identity reuses normalize_repo_key: the host segment selects whether this forge serves a recorded repo, the remainder is the API path, so no new identity scheme exists. Read-only by construction; errors never carry the token; first outbound-HTTP timeout convention (5s total, no retries — the consumer's fallback is the retry policy). OPTIONAL per instance (rule #115): get_forge() returns None when unconfigured and every consumer treats None as today's behavior. Config lives in admin settings (Settings → Config → Git Forge: kind/base URL/token, save + test-connection probe reporting version + identity), with FORGE_* env / Docker-secret fallbacks; DB wins so a UI edit can't silently lose to an env var. Token treatment follows the smtp_password convention (masked on read, mask-sentinel skipped on write, absent from audit details) — and wiring it surfaced that the generic GET/PUT /api/settings dump bypassed that masking for the owning admin's raw KV rows, so secret keys are now masked there too (fixes the same exposure for smtp_password). Contract tests run against httpx.MockTransport as the fake forge — the reference behaviors the GitHub adapter (step 8) must reproduce — plus the off-by-default gate, partial-config-is-off, env-vs-DB precedence, and route/mask structural checks. Also: the step-2 definition detector learned to skip dunders after flagging __init__ as 'already defined in 4 files' on this step's own build — guaranteed noise for a hint that must stay trustworthy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:37:27 -04:00
bvandeusenandClaude Fable 5	7d26a3fc6a	fix(tests): provenance itest user fixture is get-or-create — the lane DB persists across tests ... Both integration tests built the same username; the second insert died on users_username_key. 19 passed, 1 error on run 3802. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:06:19 -04:00
bvandeusenandClaude Fable 5	1e7f66e72d	feat(snippets): body provenance — the cache-with-provenance half of the pointer model (#2688) ... Decision #2686: the recorded location is the source of truth for a snippet's code; the stored body is a cache of it. data.provenance now records what the cache is a cache OF — commit_sha + fetched_at — as a carried JSONB field following the verification precedent, so no migration is needed and absence keeps today's exact semantics. The rules: provenance follows the code (fresh SHA restamps it, a code edit without one drops it, a metadata edit carries it); writes ABOUT the code carry it — record_verification rebuilds data from scratch and would otherwise erase it silently; an ok verdict at a known commit restamps it, since the checker just proved the cache matches the source there. verify_snippet verdicts also record the commit they ran at, making "the repo moved on since the check" computable once the forge integration lands. create/update/verify MCP tools take commit_sha (git rev-parse HEAD — free for any session with a checkout). Unit tests pin the compose/carry logic; real-Postgres integration tests run create→verify→update end-to-end (#2663: DB paths get no mocked-only coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 12:00:40 -04:00
bvandeusenandClaude Fable 5	8407368c0c	fix(hooks): definition detector covers all code, not a language shortlist (#2682) ... ARM 1 extracted definitions with patterns for CSS/JS/TS/Python only — the languages of the repo it was born in — so the local duplication proof, and the #2664 record nudge gated on it, were structurally unreachable in Go/Kotlin/Rust projects (Minstrel, FabledExchange): precisely where recording was observed never to happen. One modifier-strip plus a definition-keyword family (func/fun/fn/function/ def/sub, struct/trait/interface/enum/object/protocol/type, plus Go method receivers) now covers them all; impl is excluded because several impl blocks per type is normal Rust, and keyword-less declarations (C/Java/Dart) are documented out of scope. Grep patterns mirror the same forms so hits are definitions, never call sites. Parameterized tests pin the coverage per language family. Plugin 0.1.30. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 11:33:24 -04:00
bvandeusenandClaude Fable 5	2d58e74ec7	feat(reuse): recording model becomes the pattern library — every shape at first build (#2687) ... Decision #2686: snippets are the project's pattern library, not a dedup net. The floor, the reusing-code skill, and the snippet tool docstrings now state the proactive model — record every shape the first time it is built, with no will-it-recur judgment, and start later instances from the recorded shape; second-copy consolidation stays as the backstop. The floor guard test pins all three elements (tool, first-build trigger, backstop) so the model cannot silently regress to the reactive wording. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-16 11:33:24 -04:00
bvandeusenandClaude Fable 5	4107b17727	fix(telemetry): usage readout grouped by a rebuilt CASE — group by the label instead (#2663 root cause) ... CI's new integration tests reproduced the outage on a clean database and named the half: the writes land fine, and usage_for_notes fails on EVERY call. The GROUP BY rebuilt the ambient case() expression, and asyncpg's expanding IN-parameters give each instance its own bind names — so Postgres sees a SELECT expression the GROUP BY doesn't cover and rejects the query with a GroupingError, which the old code swallowed into zeros. One labelled expression, grouped by its label. The deployed table has been accumulating events all along; history appears as soon as this deploys. Also: the two hook-execution tests now run with the real PATH and skip when the hook's tools are absent (the restricted-PATH convention next door is for silence contracts, where empty-for-the-wrong-reason still passes) — and the unit lane installs jq so 'skip' never quietly becomes 'nowhere'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:49:48 -04:00
bvandeusenandClaude Fable 5	fb757fb4ba	feat(reuse): recording gets a seam — the prior-art hook asks for create_snippet when duplication is proven and unrecorded (#2664) ... Zero snippets were ever recorded outside sessions already thinking about snippets: the read side had a real seam (the PreToolUse hook) and the record side had a trailing clause of a floor bullet. The trigger moment — 'I just wrote the second copy' — is mid-Write/Edit, so the nudge now rides the same hook: when the local arm proves the definition exists elsewhere in the repo AND Scribe returned no record of it, the context block asks for create_snippet. Both gates or silence, so a brand-new helper and an already-recorded one stay nudge-free. Floor bullet promoted to name the trigger moments (extract, hoist, second copy); guarded by test the same way the Systems reflex is. Plugin 0.1.29 so the cache picks up the hook (#2209). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:45:32 -04:00
bvandeusenandClaude Fable 5	77acee9239	fix(telemetry): usage counters get canaries, task references, and the missing integration tests (#2663) ... The deployed instance ran with every usage counter at zero while surfacing demonstrably fired. Every unit test was green because every unit test mocked either _schedule or the session — the two functions that touch the database ran against real Postgres nowhere. Both telemetry writers also held no reference to their fire-and-forget tasks (the loop keeps only weak ones), and swallowed every failure into logger.debug, so a total outage was indistinguishable from an unused corpus. - note_usage + retrieval_telemetry keep strong task references until done - failures log at WARNING; note_usage additionally drops one AppLog error row per process per site, so the admin UI shows the outage without host access - integration tests cover _insert_events -> usage_for_notes and the full record_pulled chain on a running loop, splitting the write and read halves so a failure names its side Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-14 21:45:32 -04:00
bvandeusenandClaude Fable 5	75bdb9c148	test(systems): fix two mock bugs in the seam tests — unset .id on the fake, shared to_dict dict leaking across creates ... Runs 3562/3565: _fake_system never sets .id so the dup-gate assertion compared a MagicMock to 7; and a shared to_dict return_value dict let the tagged create's mutation leak into the orphan create's response. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:55:56 -04:00
bvandeusenandClaude Fable 5	3455f9cb9a	refactor(systems): one seam — the Systems question rides every read and write of a record ... Scribe issue #2570, from the operator's challenge: the invariant is "always be asking whether what you're touching is a System's territory and whether the work is filed there" — not a nudge in one corner. The prior shape failed it twice: the hint fired only on creates (with a special-cased zero-Systems branch), and get_task/get_note returned records WITHOUT their Systems, so the read-side reflex had nothing to fire on (same per-kind asymmetry as #2481). - attach_systems(): single helper used by get/create/update for tasks, notes, and snippets, plus add_task_log. Tagged records always show `systems`; an untagged project record carries the `systems_hint` question instead. Neither field attaches empty (#2483). Hint is owner-only; everything fail-open (#2109). - untagged_systems_hint unified to ONE question — the vocabulary listing varies, the question doesn't; the zero-Systems branch stops being special text. - Docstrings state the uniform contract; floor prose now names the read-side reflex (systems visible -> list_system_records the pile). - Plugin 0.1.27 -> 0.1.28. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:48:41 -04:00
bvandeusenandClaude Fable 5	56b1952de5	fix(systems): sweeps are the discovery moment — zero-Systems hint, create_system dedup gate, prose inverted ... First real-world test of the #2562 fixes (Scribe issue #2569): a Forge session ran a whole-codebase audit and created zero Systems — endorsed by the shipped guidance, whose "no particular area takes none" clause read as an exemption for exactly the record type that enumerates the subsystem vocabulary. And the systems_hint was silent for a zero-Systems project, the one state nothing else nudges out of. - systems_hint gains a zero-Systems branch: prompt the FIRST create_system instead of going quiet. - create_system is duplicate-gated like the other creates (normalized name, archived included, fail-open) — liberal creation becomes safe by construction, so the guidance can stop preaching restraint. - Prose inverted on every surface (hint text, create_system docstring, floor bullet, using-scribe step 7): audits/sweeps take several tags and mint the Systems they name; the gate is the guardrail against sprawl, not holding back; only a record genuinely about no particular area goes untagged. - Plugin 0.1.26 -> 0.1.27. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 15:35:59 -04:00
bvandeusenandClaude Fable 5	3ff8803593	feat(instructions): fit the delivery fold — 2k server map, floor Systems reflex, write-time systems_hint ... Claude Code injects only the first ~2,048 chars of an MCP server's instructions and silently cuts the rest mid-word (#2562, observed live): _INSTRUCTIONS was 20,002 chars, so ~90% — including all Systems tagging guidance — never reached any session. Rearchitect delivery around what each surface actually delivers: - _INSTRUCTIONS becomes a 1,997-char purpose-sorted map, with a header comment stating the budget and where detail belongs instead. - Tool docstrings keep the per-tool HOW (audit: nearly all displaced topics were already duplicated there); backfill the four gaps — enter_project session scoping + project bootstrap, create_rule entity-vs-rule test, create_design_system not-a-rulebook, create_system two-records test. - The plugin static context (the delivery floor) gains the tag-to-Systems reflex and a surfaces-layering statement; plugin 0.1.25 -> 0.1.26 so the executing cache refreshes (#2209). - create_task / create_note / create_snippet return a systems_hint when a record is created untagged in a project that has Systems — in-band at the exact write it applies to, fail-open like the dedup gate. - Guards: _INSTRUCTIONS length budget, floor-states-the-reflex, and a displaced-topics sweep asserting every cut topic still lives on a delivered surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 12:53:35 -04:00
bvandeusenandClaude Fable 5	272b7dbddf	feat(dedup): per-kind duplicate-report floors — notes/tasks default 0.93 ... At chunk grain (#280) a note-pair's similarity is its closest chunk pair, so the shared 0.82 floor saturated the note/task reports with related families (38 note / 155 task groups against the 200-pair cap, measured 2026-08-09). Split kb_duplicate_threshold into per-kind settings keys with per-kind defaults: snippet 0.82 (single-chunk, scale unchanged), note/task 0.93 (points the report at genuinely-alike records). Settings UI grows the two new knobs; report entrypoints inherit the change via get_duplicate_threshold(user_id, kind). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-09 10:33:11 -04:00
bvandeusenandClaude Fable 5	041d8defbc	feat(embeddings): best-chunk-per-note on every retrieval surface (#280 step 4) ... A note's relevance is now its best chunk's similarity, everywhere: - semantic_search_notes keeps the indexed raw-distance top-k and over-fetches chunk rows (x4, composing with the x3 supersession over-fetch), then collapses to first-appearance-per-note — rows arrive distance-ordered, so first is best. Every ranked consumer (MCP/REST search, Browse, auto-inject, write-path, gate) inherits through the one function. - list_notes semantic q swaps its join for a correlated MIN-distance subquery — the join would have repeated a long note once per matching chunk and made total count chunks. - the duplicate report groups its self-join by note pair on MIN(distance): pair similarity = closest chunk pair, and the < join now also drops cross-chunk self-pairs that would flag every long note against itself. - the write gate queries once per chunk of the candidate (capped at 8), so a note duplicating an existing record in ONE SECTION is caught — the whole-document query diluted exactly the section that mattered. Integration test now seeds a two-chunk note and pins the collapse against real pgvector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:51:01 -04:00
bvandeusenandClaude Fable 5	0e70a3896b	feat(embeddings): per-chunk rows — schema, write path, version-aware backfill (#280 steps 2+3) ... note_embeddings becomes one row per chunk: PK (note_id, chunk_index), plus chunk_text (what this vector actually encodes) and chunker_version. Migration 0077 clears the table — embeddings are derived (0067 precedent) and the old whole-document rows are indistinguishable from single-chunk notes, so the startup backfill regenerates the corpus at the new shape. The backfill is now version-aware: a future shape change is a CHUNKER_VERSION bump that re-embeds exactly the stale notes, not another wipe. upsert_note_embedding takes (title, body) and chunks internally — one path for the write path, the recurrence spawn and the backfill. The recurrence spawn's own embed call is deleted outright: create_note already embeds via embed_note (#2056), so the spawn was a second copy of the rule. An emptied record now CLEARS its stale vectors instead of leaving them findable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:47:34 -04:00
bvandeusenandClaude Fable 5	6b5043a69c	feat(embeddings): chunk_document — the chunked document shape (#280 step 1) ... bge-small reads 512 tokens and fastembed truncates silently, so a single whole-document vector permanently lost everything past ~400 words. The new shape: split at markdown headings (fence-aware), merge small sections, split oversize ones at paragraph boundaries, title-anchor every chunk, repeat the section heading on continuation pieces. A record that fits the window yields exactly one chunk identical to the historical title\nbody shape, so the corpus's sharpest records are byte-for-byte unaffected. CHUNKER_VERSION added so later shape changes re-embed by version comparison instead of a table wipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-08 23:43:51 -04:00
bvandeusen	2d1e26f38f	feat(telemetry): ambient surfacings count, apart — enter_project and the skill sync emit ... #2477, option (a) as decided, with the readout changed in the same commit. ## The two silent surfaces enter_project returns open tasks + recent notes on every project entry — probably the largest surfacing by volume — and emitted nothing, so the pulls it caused floated unattributed and the surfaced:pulled ratio ran against a denominator missing its biggest contributor. Now source "enter_project". build_process_manifest installs every reachable Process as an auto-surfacing skill on the operator's machine — its own docstring calls it the most consequential passive surface Scribe has — and emitted nothing, so a Process matched on every relevant turn and never opened was indistinguishable from one never installed. Now source "process_skill_sync": the honest event is "installed", which is a surfacing in effect since the description sits in front of the model each session. ## The readout, same commit — the condition option (a) carried Both surfaces are AMBIENT: top-N-by-recency and install-everything are not ranked choices. Pooling them into surfaced_count would make a note's number dominated by "recently updated in a project you opened", and dead-weight detection would read that as popularity — the wrong number read confidently, which is the corrupts-data tier the survey ranked above everything else. So usage_for_notes splits: surfaced_count stays RANKED-ONLY (every existing consumer's reading — "surfaced often, never pulled → dead weight" — keeps meaning what it meant), and ambient_count is new. Classified in SQL via a CASE on AMBIENT_SOURCES so the group count stays three rows per note, not one per distinct source. Pulls stay pooled: "did anyone ever open this?" does not depend on how it was found. #1038 and #2085 read agent pulls and ranked surfacings; both are unaffected by ambient volume, which is the point. Refs #2477	2026-08-08 22:39:13 -04:00
bvandeusen	9b3874b657	test: the auto-inject path now logs two retrievals, and that is the point ... 45ba4aa made the reuse slot log its query (source: reuse_slot). The margin-gate test pinned record_retrieval to exactly one call, which was asserting the very asymmetry #2463 fixed — the displaced hit logged, the displacing query not. Assert both sources in order instead. Refs #2463	2026-08-08 22:32:58 -04:00
bvandeusen	f11a547cd2	fix(lists): shared-project records appear, and a list's q means what search means ... #2462, both decided halves. ## The ACL defect list_notes filtered on Note.user_id == user_id with no scope parameter at all — never a per-call decision, the capability was absent. query_knowledge beside it was deliberately browse-scoped with a comment saying why. So a task in a shared project was invisible in list_tasks, enter_project's open-task list, the SessionStart todo count and the web UI's task views, while the same project's notes and snippets appeared. Now the shared clause: notes_visibility_clause(user_id, "browse"). Browse and not read, per decision #2094 — an ambient list must never surface a record someone shared one-to-one; those stay search-only. This is the remaining half of a fix made twice before (#2159 widened fetch, #2092 widened meaning), and the guard below is what stops a fourth half appearing. Guarded by source inspection in test_retrieval_scopes: every list-shaped service references a shared visibility clause AND carries no bare Note.user_id comparison that would quietly re-narrow it. An owner-only list returns correct-looking rows and simply omits the shared ones — the shape no behavioural test catches. ## q is semantic (operator: "make it match") The UI's note list keyword-matched while the UI's Browse search semantic-matched, over the same records, with nothing saying so. Now one meaning: q joins the embedding index and orders by cosine distance at the interactive floor, with every lifecycle filter still applied in the same indexed query. Relevance ordering wins over `sort` when q is present — a query is a relevance claim, and sorting its results by date would shuffle the answer. ILIKE survives only as the embedder-down fallback: degraded, never empty. Stated position: superseded records are NOT demoted in list-q. The penalty reorders a top-k; reordering a paginated, counted list would make page boundaries lie. The search surfaces carry the demotion. ## The interactive floor becomes one constant INTERACTIVE_SEARCH_THRESHOLD = 0.3 in embeddings.py, consumed by routes/search.py (was a commented constant), knowledge.py (was a bare literal), and the new list-q. This closes #2463's finding 3 early — the same number lived in two files with the reasoning attached to only one. ## Deferred, deliberately The return-shape unification (ORM objects vs dicts) stays undone. It is a 14-caller refactor whose motivation — callers being unable to swap paths — shrinks now that both paths share the clause and the meaning of q. If swap pressure recurs it deserves its own change, not a rider on an ACL fix. Refs #2462, #2463	2026-08-08 19:58:01 -04:00
bvandeusen	52bf40de4f	test: the empty duplicate report carries suggestion now — assert shape, not the exact dict ... The step-5 field addition (d7039dc) broke a test pinning the error-path return literally. The field is deliberate: a failed scan must match a clean scan in shape, or every consumer grows a second code path for the degraded case. Assert the parts instead. Refs #2547	2026-08-08 18:58:37 -04:00
bvandeusen	d7039dc17c	feat(dedup): the duplicate report reaches notes and tasks, with per-kind cures ... Step 5 of #278, folding in #2534. The operator's no-gate decision for the web UI (#2482 — "an llm attached to this surface is the corrections system") has a precondition nobody had built: the corrector has to be able to SEE what needs correcting. find_duplicate_snippets had no equivalent for notes or tasks, so a duplicate note was only ever noticed by accident. find_duplicate_records(kind="snippet"|"note"|"task") — the same indexed self-join, parameterised. Tasks are notes with a status, not a note_type, so the kind split is a status predicate; mixing them would propose folding a to-do into a write-up. find_duplicate_snippets stays as a wrapper because both surfaces and SnippetListView consume it by name. What differs by kind is the CURE, and the report says so in a `suggestion` field rather than leaving the caller to guess: snippet merge — lossless, the survivor keeps every call site note NEVER merge. A correction pair → supersedes on the newer; state smeared across dated records → extract to the System's reference note; genuinely parallel → leave alone. Choosing needs the records READ, which is the agent's job — so non-snippet groups carry `members` with dates and any `existing_supersessions` already declared inside the group. A pair someone ruled on is not an open question. task usually the same work opened twice — keep the one with the history, cancel the other with a pointer. The snippet sibling filter stays snippet-only: it keys on symbol/code_sha, which other kinds don't carry — and for them a look-alike is a finding. Surfaces: MCP find_duplicate_records (classified into _READ_ONLY_TOOLS — the completeness test would have caught the omission), REST /api/notes/duplicates, and a KnowledgeView panel mirroring SnippetListView's — links only, no merge button, because for notes the report proposes and the correction is a read- and-decide act. The panel follows the type filter and clears when it changes, so a note report can't linger under a task view. Correcting the task's own premise: it claimed the snippet report had "no view consuming it" — stale; SnippetListView has consumed it since it shipped. The UI gap was only ever notes/tasks. Answers the question carried from #2482: yes, the update routes on BOTH surfaces can turn a record into a duplicate — the gate is create-time by design. This report is the mechanism that catches it after the fact, which is the model the operator chose. Refs #278, #2547	2026-08-08 18:51:49 -04:00
bvandeusen	3f1523b19f	feat(systems): read-side teeth — the vocabulary at session start, a search filter, and the state/chronicle instructions ... Step 4 of #278, product half. The audit that motivated it: one System in project 2, thirty records tagged, nothing since July 28 — three days after the feature landed. Not a discipline failure; retrieval was completely blind to the association (zero references in embeddings, knowledge, search, auto-inject, or enter_project), so tagging was a write-side label with no read-side payoff, and labels nobody reads don't get maintained. Three changes, ordered by what makes the others workable: 1. enter_project returns the project's Systems (id, name, first line of the charter). Load-bearing for the tagging instruction: you cannot ask an agent to check a record against a vocabulary it never sees. Trimmed because it rides on every session start; the full charter stays get_system's job. Present-and-empty rather than absent when a project has none — "no named areas yet" is information the create-the-System instruction acts on. 2. search accepts system_id, MCP and REST (#33). Implemented once in semantic_search_notes as an EXISTS against record_systems — an association filter deciding candidate-set membership before scoring, like project_id, not a ranking signal. The REST route's missing project filter stays #2463's: it carries a default-scope UI decision this change must not preempt. 3. The instructions (#119, _INSTRUCTIONS + using-scribe skill; plugin 0.1.25 for the cache): - Tag as you write, with an executable test — "would someone investigating that subsystem want this in the pile list_system_records returns?" — rather than "tag appropriately", which is what died. - Create the System when the area has no record: the two-or-more test snippets use, plus "don't wait to be asked to name an area that plainly exists", because the agent's default was leaving un-modelled areas un-modelled forever. - State vs chronicle: dev-logs are written once and never rewritten; durable findings live in the System's reference note, updated in place — safe because note versions are the changelog, which has existed since the feature shipped and was never named as one. list_system_records' docstring now sells it as the way to READ a subsystem, reference note first. No auto-inject boost by System — vocabulary and filter first, measure before adding ranking behaviour (the #2486 lesson). Refs #278, #2546	2026-08-08 18:19:58 -04:00
bvandeusen	6c4c1bccfc	test: stub the auto-inject supersession lookup where there is no database ... CI on f20c019: nine failures across three files, all on the auto-inject path. Step 3's own tests passed; these are the same shape as the step 2 breakage — a real database call added to a path whose unit tests run without one. Stubbed per file, each saying why, rather than once in conftest. A global stub would hide the dependency from every future test on these paths too, which is the same "make the code lie" trade refused in 984407f, one level up. The auto-inject query is kept separate from the ranker's rather than threaded through, and that is deliberate: `_reserve_slot_for_reuse` runs a SECOND search and can add a hit to the menu, so the final `kept` set is not a subset of what the ranker scored. Labelling whatever actually reached the menu needs its own lookup over that final set — one indexed query on a handful of ids. Refs #278	2026-08-08 02:07:14 -04:00
bvandeusen	f20c019f2a	feat(supersession): demote what a later note overtook, and label it ... Step 3 of #278. First step with visible effect. ## Where the demotion happens, and why not in SQL Applied AFTER the pgvector fetch, over an over-fetched candidate set, not as part of the ORDER BY. Ordering by `distance + penalty` would be exact and would turn an indexed top-k into a scan-and-sort of every embedded note — the HNSW index from migration 0067 can only serve a raw-distance ordering. So the query fetches 3x the requested rows by raw distance and the re-rank happens in Python. Demoting after a LIMIT k with no over-fetch would have been theatre: the cut already happened, so a superseded record pushed down still sits in the results and the live record that should have replaced it was never fetched. The cost is stated in the code: a live record outside the over-fetch window cannot be promoted in. With a 0.05 penalty against neighbours ~0.014 apart, that needs the true answer more than three ranks down, which no observed query approaches. ## Demote, never hide — enforced in three places The penalty applies to the RANKING score, not to the relevance threshold. The floor decides whether a record is relevant at all; the penalty decides which relevant record comes first. Applying it to the floor would drop a superseded record out of the results entirely, which is the one thing this must not do. It is small on purpose. Supersession is a claim about SOME of a record's content, so one that strongly answers a question nothing else answers still surfaces — just behind anything comparable that is current. Its test asserts both bounds, the upper one citing the operator's constraint rather than an optimisation. And every test here that could be satisfied by dropping a record instead asserts the record is still present. ## The dedup gate opts out A superseded record is still a duplicate of what you are about to write — the claim is that it is no longer current, not that it is gone. Demoting it there would let the same note be recorded a second time, and the second copy would be the one nothing warns about. ## The label Auto-inject marks a superseded line SUPERSEDED with a pointer to check the later record. One query for the whole menu. An agent handed stale material with nothing marking it acts on it with full confidence, which is worse than never having surfaced it — the ranking is only half the fix. Fails open: a supersession lookup error returns unpenalised results rather than none, because unpenalised ranking is the behaviour that shipped for months and a broken search is not. Refs #278	2026-08-08 02:00:06 -04:00
bvandeusen	984407f931	fix(supersession): one query for both directions, not two per note read ... CI failed on 8d9e96c — eight tests in test_mcp_tool_notes.py, all "Connect call failed (127.0.0.1, 5432)". The proximate cause is that `_attach_supersession` runs on every note read/write and those are unit tests of the tool layer with no database. But the test failure exposed a worse decision underneath it. I had written the two directions as two service calls, so every `get_note` made TWO extra round trips plus TWO ACL checks — on the hottest path in the product — to save a two-line partition in Python. That is the wrong trade whether or not a test noticed. `get_relations` replaces both: one OR query, one ACL check, partitioned by which column holds the note's id. Its test asserts `execute.await_count == 1`, so the collapse can't quietly come apart later. The tests then get an autouse stub rather than the code getting a swallow. The tool genuinely has a new dependency; hiding that behind a try/except to keep unit tests green would be arranging for the code to lie about what it does. This file already records the same hazard for note 2109, so the stub sits next to that precedent. Added the test that matters, which the first pass missed: a superseded record still surfaces, so an agent WILL read stale material — and it must arrive with a plain-language warning, not just a numeric field to notice. Also pinned that both keys are ABSENT rather than present-and-empty when there are no relations. Refs #278	2026-08-07 22:45:56 -04:00
bvandeusen	8d9e96cc6d	feat(supersession): declare it — supersedes on both write paths ... Step 2 of #278. Records and reads the claim; the demotion that makes it matter is step 3. `services/supersession.py` with set/get on both directions, following the set_record_systems shape since this is the same kind of mutable M2M at the tool/route layer rather than inside notes_svc. ## Both directions are exposed, and only one is obvious `supersedes` is what the author claimed. `superseded_by` is what a READER needs and what the note itself cannot know — a stale record handed over with no marker gets acted on confidently, which is worse than never surfacing it. So get_note carries it, says so in its docstring, and adds a plain-language line telling the reader to open the newer note first. Both are OMITTED when empty rather than serialised as empty lists. A field that always says nothing trains readers to skip fields — the lesson consolidated_at cost, removed in the previous commit. ## Refuse vs drop, which is the one real judgement here Dropped silently: a target that doesn't exist, is trashed, is the note itself, or would close a cycle. Each is a claim with no subject or no meaning; none is something the caller can act on. REFUSED with PermissionError: a target the caller can read but not write. That is the single case where the caller could believe they succeeded and be wrong in a way that matters — demoting someone else's record out of their retrieval is damage invisible from the outside, with no symptom for the owner to trace. Rule #47, and PermissionError because services/snippets.py already uses it for read-but-not-write with both surfaces catching it. The PATCH/PUT routes scope by the CALLER, not owner_uid: an editor-share holder may edit the note and must not thereby inherit the owner's write access to whatever they name as superseded. ## Cycles A ring claims every member is obsolete. Under flat demotion that demotes them all equally, so the set drops out of ranked retrieval together with nothing in the data saying why. Refused by walking the existing graph from the proposed target — iteratively with a visited set, because the graph is user-supplied and a deep chain must not become a stack overflow on a write path. The visited set also makes the walk terminate on a ring that already exists, which is pinned by its own test rather than trusted. Both surfaces (#33), the instruction surface per #119 — framed as the third answer beside update-instead and force=true: not everything resembling an existing record should be folded into it, and not everything distinct should compete with it forever. Refs #278	2026-08-07 22:38:09 -04:00
bvandeusen	5dcb738ce8	fix(backup): carry note_supersessions — the coverage guard caught the omission ... CI failed on 45c6b1c. `tests/test_services_backup.py` asserts every table is either in `_BACKED_UP` or explicitly in `_NOT_INCLUDED`, and the new table was in neither. That guard exists for exactly this: a new table gets a model and a migration — both of which fail loudly if wrong — and then silently never gets a backup section. No error, no warning, and a restore that reports success. Its comment says it was written to "stop the seventh" table slipping through. This was the seventh. Supersession claims are backed up rather than excluded because they are a JUDGEMENT. Someone decided this note replaced that one, and nothing in either note's text records the decision. Lose them and a restored corpus silently returns to ranking stale material alongside current material — with no symptom that says why. Scoping differs by backup kind, deliberately. A whole-instance export takes every row. A single-user export takes only claims where BOTH ends are that user's notes: a claim spanning out to someone else's record cannot be restored into a single-user import, since the far id is not in the map, so exporting it would write a row that silently vanishes on the way back in. Restore maps both ids and skips the row unless both resolve. A claim is about a PAIR — half of one is not a weaker claim, it is a dangling row pointing at whatever note holds that id next. Guarded by `data.get` like every post-v2 section, so v5 and older payloads restore cleanly without it. BACKUP_VERSION 5 -> 6, and the version test moved with it. A payload section added without moving the version produces backups that are structurally different and indistinguishable by inspection. Refs #278	2026-08-07 21:49:08 -04:00
bvandeusen	bbba0b3ae3	refactor(embeddings): one definition of the document a record is embedded as ... `f"{title}\n{body}"` was written out four times. #2486 found three — the write path, the recurring-task spawn, the startup backfill. The guard added here found the fourth immediately, and it was the one that mattered most. `dedup.find_duplicate_note` built the same string as a QUERY, compared against embedded documents. Shaped differently from the corpus it searches, the gate degrades silently: it still returns neighbours, just less apt ones, and nothing says the query and the index stopped agreeing. The spawn path has the same shape of risk — a recurring task embedded differently from everything else is ranked against documents it doesn't match. None of the four had diverged. That is what makes this worth doing now rather than after: they are identical today, so collapsing them is a no-op, and the whole point is that the next change to the shape can't hit three of four. Which is imminent. #2486 measured a dev-log separating from five unrelated dev-logs by 0.023 where a snippet separates by 0.153 — the difference being that a snippet states its purpose twice in a short document. Whether that shape is right is the open question; testing an alternative against four copies would mean testing a shape that isn't the one in production. This is the precondition the issue named. The guard is source inspection, matching the f-string pattern rather than a variable name, so a copy that renames its locals is still caught. A behavioural test cannot see this: an inlined copy produces the same string today and diverges the day the shape changes. Refs #2486	2026-08-07 13:01:45 -04:00
bvandeusen	24d071619b	fix(dedup): compare the artefact, not the prose describing it ... The snippet gate was reading the wrong field, and #2464's UI recipes made it measurable in both directions at once: .btn-danger vs .btn-danger-outline 0.92 siblings, BLOCKED .btn-primary re-recorded verbatim under a different name <0.90 a literal copy, ALLOWED The second is what settles it. Identical code at an identical repo·path·symbol sailed through because the description differed, while two deliberately parallel variants were refused because theirs did not. A snippet's embedded document is mostly prose ABOUT the code, so no threshold fixes this: lowering it blocks more siblings, raising it admits more copies. So structure decides. Two exact signals, both index-served off the notes.data mirror that already exists, no migration and no backfill: location the same named thing in the same file. Requires BOTH path and symbol — a path alone is a directory of artefacts, and matching on it would refuse every second recipe from one stylesheet. code byte-identical code anywhere, via the same fingerprint the drift check uses. The semantic arm survives as a backstop for a genuine reword that shares neither, raised to 0.96 so it sits above the 0.92 band where real variants live. Structural hits say what they matched instead of hedging with "similar", and point at merge rather than update — two records of one artefact is what merge exists to fold back together. find_duplicate_snippets gets the same correction: pairs where both snippets name a symbol, name DIFFERENT symbols, and hold different code are variants, not copies. Without it a design system's button family reports as one merge set — eight recipes, every direct pair over the floor, top score 0.92, one click from collapsing a component family. The cost is real and stated in the code: a helper recorded twice under two names no longer reports. That trade favours the report being usable, and same-symbol and unnamed duplicates — how re-recording usually looks — still surface. The filter fails open, so a lookup failure degrades to the old unfiltered report rather than to a reassuring empty one. resolve_locations extracted: compose_body, create_snippet and now the gate each had their own copy of the repo/path/symbol shorthand fallback, and the gate is the one where a disagreement would mean matching a location the record won't be stored with. Applied to both create surfaces (#33) — the web UI must not be the way to record what the agent was stopped from writing. Refs #2518, #2464	2026-08-06 11:31:02 -04:00
bvandeusen	c18139622c	fix(telemetry): the human half of the pull ledger recorded one kind in three ... Opening a snippet in the UI recorded rest_snippet. Opening a note or a task recorded nothing — so the most direct evidence the product has that anyone cares about a record existed for one kind out of three, and the other two sat at zero pulls looking like dead weight beside a kind that merely had a counter. Not an open question about intent: models/note_usage.py already documented 'rest_note' as a source value. Nothing wrote it. The design named it and the implementation stopped at snippets. Adds rest_note and rest_task. Tagged by SURFACE rather than by the record's kind, matching rest_snippet — the kind is a join away, but which surface asked is not recoverable after the fact. The mcp_/rest_ split stays load-bearing: "is this dead weight" is served by any pull, "was that injected line useful" by agent pulls alone, and a human clicking a link would inflate exactly the number #1038 and #2085 gate on. The vocabulary comment in the model was itself the stale-enumeration shape this survey keeps finding — it named a source nothing wrote while omitting sources that existed. Replaced with the naming CONVENTION plus a pointer to grep, which cannot drift, rather than a longer list that would go stale the same way. Guard extended to the REST surface, same derivation as the MCP half: a route registered at exactly /<int:x> for GET is a detail view, and one reaching a note-backed loader must record. Handler source is expanded one level through module-private helpers, without which get_snippet_route — the route that already got this right — would drop out of the check by loading via _load_snippet. Verified the guard fires when a call is removed. Renamed test_mcp_pull_telemetry.py -> test_pull_telemetry.py; it is no longer only about MCP. Closes #2476	2026-08-06 08:51:21 -04:00
bvandeusen	ac1ce0a7f0	fix(mcp): a read key could read notes but not snippets or design systems ... _READ_ONLY_TOOLS fails closed, which is the right design — but the list had gone stale, so a read-only key could get_note and not get_snippet, both pure reads of the same table, and could not read a design system at all. That inverts the sensitivity ordering: the free-text records were reachable and the structured, low-sensitivity ones were not. `find_duplicate_snippets` sitting in the list was the tell — someone classified the report and missed the getters beside it. Adds the twelve reads that were missing: snippets, processes, the six design system tools, and list_repo_bindings. Each verified to mutate nothing rather than assumed — this is a security boundary, and a wrong entry does not cost what a missing one costs. record_pulled on four getters is telemetry about the read, not a change to what was read, and get_note already carried it inside the boundary. The list stays explicit. Deriving it from the name would be worse than staleness: it makes the boundary follow a naming convention, so any future get_* grants itself access. list_starter_role_groups is the live illustration — it reads a constant, but names create_design_system in its docstring, so a pattern-matcher flags it. So derive the CANDIDATES and keep the DECISION explicit: a new test asserts every read-shaped tool appears in _READ_ONLY_TOOLS or in a declared _DELIBERATELY_WRITE_SCOPED, and that neither set names a tool that no longer exists. Adding a getter now forces a classification at review time instead of denying it silently. The second set is empty and stays declared — otherwise a future get_or_create_* would be pushed into the allow-list to make the test pass, which is the wrong way to satisfy it. Third instance of the same shape, after #2476 and #2444: a hand-written enumeration that missed the members added after it was written. Refs #2496	2026-08-06 08:43:10 -04:00
bvandeusen	ffd08507f1	fix(instructions): the push is an optimisation, not the bridge ... _INSTRUCTIONS told an agent the SessionStart hook was how rules reach a session, and used that as the argument against a host-memory pointer. The using-scribe skill said the opposite — pull them yourself, treat any push as a bonus. Nothing said which wins, and #119 makes these surfaces the specification, so this was the product behaving two ways. #2198 is the case that settles it: every plugin hook was silently inert for an extended period. An agent trusting the push would have run with no binding rules and no signal, while those rules govern branch, commit and push. So: _INSTRUCTIONS now leads with the explicit pull and names the hook as a delivery optimisation. The argument against a host-memory pointer survives — it never needed the hook to be reliable, because the pull IS the bridge and it is written into every surface a session already loads. The static context gains the tiebreaker for the next disagreement: follow the surface that assumes least about its own delivery. "Most detailed wins" is wrong precisely because the most detailed surface is the one with a delivery precondition. It goes there by its own logic — a tiebreaker arriving over MCP cannot arbitrate what to do when MCP is absent. Guarded by tests/test_instruction_surfaces_agree.py: every session-start surface states the pull, and no surface names the push without it. Plugin version bumped so the cache that executes actually picks the file up (#2209). Refs #2497	2026-08-06 08:23:36 -04:00
bvandeusenandClaude Opus 5	63c213b617	fix(processes): the least-equipped kind is the one that gets followed ... Survey pass 3 (#2250) tabulated capabilities per record kind. Processes came out lowest on every column, and they are the kind with the most authority: build_process_manifest turns each one into a skill file on the operator's machine that auto-surfaces and is followed as written — its own docstring calls it "the most consequential passive surface Scribe has." Three gaps closed. NO PULL TELEMETRY (#2476). get_process recorded nothing, while the auto-inject menu header names get_process as the way to open that kind. Every note is embedded regardless of note_type, so a Process is surfaceable — and the getter the product points at was the one getter that recorded nothing, leaving every Process permanently at zero pulls and looking like dead weight beside kinds that merely had a counter. get_note's own comment already listed processes as a reason to record pulls. The fix for #2245 covered notes, tasks and snippets: it enumerated the kinds someone thought of rather than the kinds that exist. NO DEDUP GATE. create_process had no near-duplicate check and no force flag, while notes, tasks, snippets and rules all have both. It matters more here than elsewhere: two near-identical procedures don't just bloat the corpus, they compete to be followed, and which one wins is decided by a slug collision. NO DELETE. list/create/get/update, no delete — a kind that reads as one you cannot retire. Deletion was always possible via delete_note, since a Process is a note and the trash is kind-agnostic, so this was discoverability rather than capability. delete_process checks note_type before trashing: the tool is reached for by name, and letting it destroy an ordinary note whose id happened to resolve would be a destructive action taken on a mistyped argument. THE GUARD, which is the part that stops a fourth repeat. tests/test_mcp_pull_telemetry.py discovers every get_* MCP tool by AST and requires a record_pulled from any that loads a single note. Not a list of getters — a get_<newkind> added tomorrow is covered the moment it loads a note the way the others do. get_milestone is correctly excluded: it calls list_notes for a milestone's steps, which is a surfacing, not an opening. The loader NAMES are a list, and that residual weakness is pinned against a rename rather than papered over. An earlier draft tried to discover new loaders by return annotation and would have failed on create_note — which also returns a Note. Readers and writers aren't distinguishable by type, so the honest version is a pinned list, a non-empty assertion, and a docstring saying which hole remains. test_register_attaches_four_tools became a derived check of the module's public coroutines, so the next tool added can't be left unregistered. MCP _INSTRUCTIONS updated: product behaviour belongs in the instruction surfaces, not in a rule (rule #119). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-05 16:32:05 -04:00
bvandeusenandClaude Opus 5	dcd4efcea0	refactor(design): retire /design — a surface that could only inspect itself ... The design surface is for the projects an install tracks. /design read the running app's own stylesheet — names out of a bundled theme.css, values out of getComputedStyle(document.documentElement) — so it could only ever describe the instance serving the page. Scribe is one project among the projects Scribe tracks; it gets no view hardcoded into every install. The mechanism that makes this a tool rather than a mirror already existed and already covers Scribe: scripts/check_design_tokens.py runs in CI against a sheet path it knows nothing about, using check_code_against_tokens — the same engine behind check_snippets_against_system. /design was redundant even here. Removed: DesignView, DesignTabs (nothing left to tab between), api/design.ts, routes/design.py and its blueprint, the /design route, ui_design_system() and its setting, and the Settings picker that designated "this app's UI". utils/designTokens.ts and utils/designDrift.ts go with it — between them they were the browser-reading half. What survives is utils/designValues.ts, which works on a record rather than a document: valueForMode, modesPresent, and resolveDeclared. resolveDeclared gained real isolation in the move. Custom properties inherit and `all: initial` does not reset them, so a probe sitting in this page would resolve any reference a record leaves undeclared against the SURROUNDING app's tokens — previewing another project's system would quietly borrow this one's palette wherever that system was incomplete, and a token already reported under unknown_refs would render as though it were fine. Undeclared references are now blanked on the probe first, so they resolve to nothing, which is what the record says they are. Migration 0075 absorbs ui_design_system_id alongside design_rulebook_id rather than an 0076 undoing it: 0075 has not run anywhere, since dev is unmerged and deploys come from main. Both keys named a design source for the running install, and a project already carries its own pointer. This retires the agreement panel shipped yesterday. It asked whether the sheet was actually loaded and applied — the one question a record cannot answer about itself — but only ever about the app you are already inside. Nothing replaces it; recorded in #2430 rather than quietly dropped. Step 1 of milestone #274. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-04 10:35:52 -04:00
bvandeusenandClaude Opus 5	5b824c1626	feat(design): the panel now asks whether the app agrees with its own sheet ... Retiring rulebook #2 left the /design drift panel with no data source, and because its empty state was well-written the feature read as working while it could only ever render "nothing designated" (#2419). The original question is genuinely gone: theme.css is generated from design system 2, so checking the system against a sheet derived from it would be a tautology. The question that survives is the one no server can answer. A generated sheet still has to be LOADED and APPLIED, and nothing checked that it was: absent the record declares a token the app doesn't have — the sheet was never regenerated after the record changed, or never loaded differs the app has it with another value — a stale sheet, or a later rule that overrode it unrecorded the app declares a token in the record's own family that the record has never heard of Both sides go through the same engine so the comparison is honest: declared values are set on an offscreen probe and read back, which performs the same var() substitution the browser already did to the live values. Comparing raw strings would mark every derived token as drift. The designation moved with the feature — design_rulebook_id becomes ui_design_system_id, with a migration deleting the retired key rather than leaving an inert row. The prose extractor it fed goes too (#2288 said its runtime role ended when the import landed). Three orphans of the same shape, found alongside and fixed here: - darkOverriddenNames hardcoded [data-theme="dark"]. The sheet went dark-first months ago, so it matched nothing and the "mode-aware" flag silently left the gallery. Now matches the SHAPE of a mode selector, which also holds for an install whose modes aren't light and dark. - groupFor's prefix table never heard of --fs-, so 110 tokens sat under "other". Groups now come from the record where there is one; the table can only know families that shipped with the product (rule #115). - The type scale was a hand-written table of nine sizes marked "no token", true when written and false since the scale was recorded. Now rendered from whatever size tokens the sheet declares, so it can't go stale twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-03 20:50:03 -04:00
bvandeusenandClaude Opus 5	4852b0d3df	test(design): add the starter-roles endpoint to the URL enumeration ... CI caught it: tests/test_routes_design_systems.py enumerates every routed rule, and I added a handler without adding its rule. The guard doing exactly what its docstring says it is for. Two enumerations govern this blueprint and I had only extended one — the parity list (handlers exist on both surfaces) but not the URL list (handlers are actually routed). They catch different failures, which is why both exist. Noted in place: /api/design-systems/starter-roles is a static segment sharing a prefix with /api/design-systems/<int:design_system_id>. That pairing is where a silently-shadowed route hides, so it is worth being explicit that the int converter cannot match "starter-roles" — verified rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-03 11:42:11 -04:00
bvandeusenandClaude Opus 5	22f907c44d	feat(design): offer starter token ROLES at creation, never values ... A literal gets written into a stylesheet when there is no role to reach for. This codebase demonstrated it: the house style had no "text on a filled colour" role, so 76 call sites wrote a pure-white literal — not out of defiance, but because nothing existed to write instead. The correction was not a better ban list; it was declaring the missing role (#2275, #2349). So the useful moment is creation. A system whose roles are named on day one never presents the occasion. Ten groups, ~40 roles: surface, text, action, semantic, border, accent, radius, space, motion, state. Operator's call was one flat list, every group individually skippable — presets keyed to app shape (web / CLI / docs) were rejected because they need the product to hold opinions about app categories, and a wrong category is worse than a list someone prunes once. TWO BOUNDARIES THIS HAS TO HOLD, both rule #115: - The ROLES ship; the VALUES never do. Every seeded token has an empty value_by_mode, so a fresh system is a set of named, deliberately-unanswered questions. A test asserts no hex appears anywhere in the module — not just that tokens are blank, but that no palette hides in a comment waiting to be pasted in. - The PREFIX is the install's. `--fs-` is FabledSword's convention, not the product's; the default is a neutral `--ds-` and callers pass their own. Valueless roles are already legible downstream — render_stylesheet emits them as commented-out declarations and stylesheet_for_system reports them under `valueless` (#2299) — so "declared but undecided" reads correctly with nothing new built. Both surfaces, per rule #33: MCP gains starter_role_groups/token_prefix plus list_starter_role_groups(); REST gains the same on POST plus GET /api/design-systems/starter-roles. The parity enumeration is extended rather than loosened. Note create_design_system treats None and [] alike (seed nothing), while starter_tokens treats None as "all". Deliberate: creation must never write 40 rows into a system whose caller never asked, and the everything-checked default belongs in the UI where the operator can see it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-03 11:37:20 -04:00
bvandeusenandClaude Opus 5	be3a0ffaf9	fix(projects): batch the summary queries — the fan-out was exhausting the pool ... Reported live: Projects and Snippets showed skeletons that never resolved, /knowledge worked intermittently. The logs named it exactly: QueuePool limit of size 5 overflow 10 reached, connection timed out, 30.00 GET /api/settings 500 30584.0ms GET /api/projects 200 30882.9ms /api/projects was not hanging — it was waiting out the 30-second checkout timeout and then returning 200 with summaries silently missing, because _attach swallowed the TimeoutError. Nobody waits 31 seconds, so it read as a hang. THE SHAPE: routes/projects.py ran asyncio.gather over every project. Each _attach called get_project_summary, which opened its own session for three queries and then called get_project_milestone_summary — which opened one more session PER MILESTONE. So 25 projects asked for roughly 250 concurrent checkouts against a pool of 15 (SQLAlchemy's default 5 + 10 overflow). That is why unrelated routes failed too. Snippets and /knowledge were never broken; they queued behind the burst and inherited its timeout. /api/settings returning 500 while /api/projects returned 200 is the same cause wearing two faces. The comment above the gather said "one backend pass instead of N+1 frontend calls". It did remove the N+1 from the network — and recreated it against the connection pool, where it is worse, because the browser had at least been serialising those calls. Now: get_project_summaries() does all projects in four queries and one session, and get_project_milestone_summaries() does all milestones in two. Two sessions total for the whole page, independent of how many projects exist. The progress calculation is extracted to _progress_from_counts and shared by both the batch and single paths, so the cancelled-exclusion rule cannot drift into two versions that disagree about whether a milestone is finished. Tests assert the SESSION COUNT, not just the values. An implementation that returned identical output while opening a session per project would pass a correctness test and reproduce the outage. Deliberately NOT done: raising pool_size. It would move the cliff rather than remove it, and this endpoint now needs two connections regardless of scale. Closes #2384. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-08-02 19:09:36 -04:00
bvandeusenandClaude Opus 5	2cc9e1380e	test(backup): assert the version constant, not a copy of it ... The export test pinned `out["version"] == 4`, so bumping BACKUP_VERSION broke a test that was only ever checking the payload carries the version — which it still did. Asserts against backup.BACKUP_VERSION now, and covers the six v5 sections alongside the v3 ones. The guard itself passed on the first run: every table in Base.metadata was accounted for, in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:13:16 -04:00
bvandeusenandClaude Opus 5	84541f392b	fix(backup): six tables were silently absent, and nothing would catch a seventh ... services/backup.py enumerated its tables as hand-maintained literals with nothing tying them to the schema. Tables added since that list was last extended were absent from every backup — no error, no warning, and a restore that reports success. Missing: systems, record_systems (0065), note_usage_events (0071), design_systems, design_tokens (0072), and repo_bindings — which the issue itself had not spotted, found only by diffing the model tablenames against the two lists instead of trusting either. _NOT_INCLUDED was worse than incomplete: it named "embeddings", "invitations" and "password_resets", none of which are tables. It read as coverage while naming nothing the schema could confirm. Now real names, plus retrieval_logs — observational telemetry that grows per query and that nothing reads for correctness. THE DELIVERABLE IS THE GUARD, not the six sections. Extending a list fixes today and changes nothing about the next table; a new one now fails a test until someone either backs it up or states that it shouldn't be. It checks both directions — an unaccounted table, and a listed name that no longer exists, which is what the three phantom entries above would have tripped. Design systems need ordering care: parent_id is a self-FK. The export orders parent-first (parent_id NULLS FIRST, then id — a parent always has the smaller id), so restore resolves each parent from the map as it goes, with no second pass. A child whose parent is missing lands as a root rather than failing the whole restore. Usage events are kept because pull-through is the evidence base for whether recall works, and it only ever accumulates — a restore that dropped it would reset that measurement to zero while everything still looked fine. BACKUP_VERSION 4 -> 5. Every new restore section is data.get()-guarded, so v2/v3/v4 payloads restore unchanged. Closes #2293. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:09:25 -04:00
bvandeusenandClaude Opus 5	da2383b079	fix(retrieval): verify the reserved slot's kind instead of trusting the query ... CI caught six failures on f8522fb. Five were fixtures; one was a real assumption. THE REAL ONE: _reserve_slot_for_reuse trusted that a query filtered by note_type could only return reuse kinds. It now checks _record_kind on the way in. That slot exists FOR snippets and processes — one silently spent on something else is worse than no slot at all, because the resulting line is indistinguishable from one that earned its place on score. THE FIXTURES, all the same shape: MagicMock notes with is_task left to auto-create. It is truthy, and _record_kind reads task-ness FIRST — so every mock snippet in three test modules was rendering as "task". Two of those fixtures already carried a comment explaining this exact hazard about `.data`; the same reasoning applies to `.is_task` and nobody had needed it until the menu started naming kinds. One assertion was genuinely stale rather than broken: test_write_path_trigger pinned note_type == "snippet", which was the behaviour the widening replaced. Updated to the new contract, including the task_kind="issue" filter that keeps the open to-do list out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 23:01:31 -04:00
bvandeusenandClaude Opus 5	f8522fb28f	fix(retrieval): give reuse a slot, and let experience reach the write path ... Two mirror-image scoping mistakes, neither deliberate (#2246). AUTO-INJECT let every kind compete on raw cosine. That is fatal rather than merely imperfect here, because Scribe's project records are ABOUT software work: a task titled "surface snippets before the agent writes code" is a near-perfect lexical match for "write a function…" while answering none of it. Measured live, a prompt asking for a helper returned three records about BUILDING the retrieval system and zero snippets. Snippets are ~0.5% of the corpus, and the ratio worsens as the project record grows — which is the direction Scribe is meant to grow, so no threshold tuning fixes it. Now the best snippet or process takes the LAST slot when none won on score. Deliberately NOT held to the margin band: that band measures distance from the top overall score, and the top score is the very thing snippets lose to. It still must clear the configured threshold, so a weak snippet cannot buy the slot — silence stays the default. Skipped entirely when reuse already won, so the fix is invisible in the case it isn't needed. WRITE-PATH was snippets-only — the same mistake inverted. An issue recording "we tried this and it deadlocked" could never reach the moment that code was about to be written, though it is arguably the better prior art: it says what NOT to do. Widened to snippets plus recorded experience. That needed a filter the search layer couldn't express. "Experience" is issues plus dev-logs, which differ on is_task, so neither note_type nor is_task alone covers it. semantic_search_notes gains task_kind, which restricts TASKS to the given kinds while leaving non-task notes untouched — so note_type=("snippet", "note") + task_kind="issue" yields snippets, fixed problems and durable notes, without the open to-do list. note_type now accepts a sequence too. Non-snippet hits are labelled with their kind, because an unlabelled issue on that menu reads as "here is code to reuse", the opposite of what it says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:56:31 -04:00
bvandeusenandClaude Opus 5	5c51e29f26	fix(embeddings): embed in the service, so every caller gets it (#2056) ... A note or task created through MCP was not semantically searchable until the next restart's backfill ran. Embedding fired at the five REST route handlers and nowhere else; the MCP tools call the service directly, so they skipped it. The shape of this bug is the reason to care: it is invisible on an instance that redeploys constantly (this one does, per rule 46) and permanent on one that doesn't. Rule 115 — the product has to stand up for the install that restarts twice a year, not just for the one that restarts hourly. Moved to services/notes.embed_note(), called from create_note and update_note, and deleted from all five routes. Every caller — REST, MCP, recurrence, snippets — now gets it by construction rather than by remembering. Two things fall out of having one implementation instead of six: - It uses note.user_id, the OWNER. The routes were inconsistent: some passed the caller's uid, some the owner's. On a shared record the caller's id mints a second embedding row that nothing reads. - services/snippets.py's _embed_snippet existed only because snippets are created via MCP and the routes couldn't cover them. Every one of its four call sites goes through notes_svc, so the helper and its four calls are gone, along with the eight test patches that existed to neutralise it. RuntimeError (no running loop — unit tests, scripts) and any indexing failure are both swallowed: a write that succeeded must not be failed by its index. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:46:57 -04:00
bvandeusenandClaude Opus 5	731ca284c3	feat(design-systems): give a design system a way to reach the session ... Storing a design system never made a session aware of one. Rules get pushed into every session by the SessionStart hook and returned by enter_project; a design system had neither, so its standards were reachable only by an agent that already knew to call resolve_design_system — the same silent failure as a token nobody declares. That gap was invisible while the operator's visual standards also lived in a rulebook. Retiring that rulebook (which is what this unblocks) would have deleted design guidance from every session with nothing to say so. - services/design_systems.design_context() — the delivery side. Guidance is chain-merged ANCESTOR-FIRST: a child system holds only what it CHANGES, so its own guidance describes a departure from a house style it never restates, and the leaf alone is a fragment. Tokens are summarised (count + group names), not listed — a hundred declarations would crowd out the context they are meant to inform. - enter_project returns `design_system`, null when the project has none. - The SessionStart context gains a Design system block with pointers to the values, alongside the always-on rules. - server.py's entity list gains Design system, including the negative: do NOT record one as a rulebook, because a token kept as prose cannot be resolved, inherited, rendered or checked. - The rulebook-tier passage used "a design-system rulebook" as its worked example of a subscribed rulebook — it now teaches the opposite, plus a new "is this a rule at all?" test pointing at design systems, processes and snippets. - using-scribe gains a section on building UI against the project's system. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-31 22:10:09 -04:00
bvandeusen	378a4b8f99	feat(ci): check the app's own components against the tokens, not just snippets ... Closes the gap the theme.css repoint exposed (#2319, part of #2277). `check_code_against_tokens` could always answer "does this code use the sheet correctly?" — it was only ever fed recorded SNIPPETS. The app's own components, where sixteen unresolvable references were living quietly, were checked by nothing at all. That was structural rather than an oversight: the drift panel runs in the browser and cannot read source files, and the server has no repo access. CI is the only place holding both the sources and the ability to run the check — and it only became cheap once theme.css became a generated artifact, so the source of truth is a committed file with no network and no credentials. **The sheet now carries its own SUPERSEDES block.** That is what keeps the checker instance-agnostic (rule #115): it knows nothing about any palette, and reads both the declarations and the discouraged literals out of whatever stylesheet it is pointed at. Hardcoding "#fff means use the text token" would have baked one install's kit into the tool. Two severities, split on whether the count is already zero: FAIL an unresolvable var() reference — zero today, so this is a ratchet holding a line already reached. It cannot false-positive either: the name is declared or it is not. REPORT superseded literals (32 files) and raw colour literals (246). Gating those means a permanently-red job, and a check nobody reads is worse than no check. **Comments are stripped before scanning, and that fired on the first real run.** A comment explaining why a literal is avoided necessarily contains that literal — DesignSystemsView's stylesheet documents exactly that about `#fff`, and the checker reported the explanation as a violation. A checker that flags the documentation of a rule teaches people to stop documenting rules. Also narrowed `--fs-weight-medium`'s supersedes to the keywords. `600` and `700` are real violations of the two-weight rule, but a bare number matches too much to find by literal scan — `z-index: 600` is not a font weight. That needs a property-aware check, which is a different tool. Verified end to end: the generator's output parses back through the checker's reader, so the two halves cannot drift into disagreeing about the format.	2026-07-31 14:52:06 -04:00