198 Commits

Include renames

Author	SHA1	Message	Date
bvandeusen	46d88f9e7e	feat(design-systems): check the components against the sheet they claim to use ... "The snippets use the tags from the sheet" was a relation nobody could verify. Now it is three checks, and all three currently fail SILENTLY in this codebase: unknown `var(--x)` where the system declares no `--x`. Renders as nothing at all — no error, no failing test, no visual clue beyond the element quietly not being styled. superseded literals a value the sheet said to stop writing, paired with the token to write instead. Only possible because `supersedes` is declared rather than inferred. local definitions custom properties a snippet mints for itself instead of reusing the sheet's — the bloat a shared sheet exists to prevent, where a value stops being reused and starts being restated per component. The first is not hypothetical. Writing DesignSystemsView.vue earlier in this same session I used `--color-accent` throughout; it does not exist, and nothing in the toolchain noticed. This check is the thing that would have. A token that is both defined and read locally is reported ONCE, as an unknown reference — "--btn-bg does not exist in the sheet" is the more precise statement of the same problem, and reporting both would double-count one fact. Literal matching is boundary-aware and case-insensitive: `#fff` must not fire inside `#ffffff` (different colours, and a finding on the wrong one sends someone to change correct code), while `#FFFFFF` in a rulebook has to match `#ffffff` in a stylesheet — the same trap `normalize_hex` exists for. Snippets with nothing to report are omitted entirely. A list of everything that is fine is a list nobody reads twice — the same principle the auto-inject menu and the drift panel are both built on. Two integration mistakes fixed while wiring it: `list_snippets` returns `(rows, total)` and caps its limit at 100, and `get_snippet` returns a Note model rather than a dict. The list rows carry a preview, not the code, so the check reads each full body — checking the preview would have reported on a truncation.	2026-07-30 21:47:47 -04:00
bvandeusen	b0a7d9e89b	feat(design-systems): the master sheet — purpose tokens, not per-element values ... Operator's new requirement (#2299, architecture in #2296): a design system does not just hold tokens, it generates and manages a master CSS sheet. That settles the milestone's open "authority mechanism" question — the record is authoritative because the stylesheet comes out of it. **The sheet is shaped by purpose and styles no elements.** It declares custom properties, grouped by what they mean, and contains no `.btn-primary`, no `table`, no `input`. That is the design, not a shortcut: a sheet that styled elements would restate the same handful of values once per element and grow with the UI, where purpose-named values are stated once and reused. Components live as SNIPPETS that reference these names — a surface that already exists and already carries prose, locations, drift checks, merge and write-path recall. A token named after an element (`--fs-button-bg`) is the smell that the two have been mixed; a purpose name (`--fs-action-primary`) is reused across all of them. Alongside the CSS the endpoint returns what the text cannot say for itself: which tokens are still valueless, and which VALUES are declared under more than one name. The second is the operator's "reuse consistent values" constraint made checkable — and it reports rather than refuses, because a design system legitimately aligns colours on purpose ("Success = Moss, by design") and only a human knows which case it is. Mode maps to selector the way the codebase already does it: base on the root selector, every other mode layered on `[data-theme="…"]`. The root selector is a PARAMETER — #251 recorded that a container-scoped preview cannot use `:root`, so hardcoding it would have made the generator useless to the preview surface. A token the rulebook names but states no value for is emitted as a commented-out declaration IN ITS GROUP rather than dropped. Its absence is the finding, and a comment puts that finding where the reader already is. Values are validated, not escaped, and this is a real boundary rather than tidiness: design systems are shareable records (rule #47), so `red; } body { display: none` in a system shared with you would otherwise inject CSS into your page. A value containing `{ } ; @ < >`, a comment delimiter or a newline is REFUSED and rendered as a comment saying so — rejecting beats stripping, since a partially-sanitised value is one the operator never wrote and the sheet's whole claim is that it is the record. Not in scope, and deliberately: serving this as the app's actual stylesheet. Generating and exposing a sheet is reversible; swapping theme.css for a generated one is not, and it should be an explicit call rather than a side effect.	2026-07-30 21:42:08 -04:00
bvandeusen	4dc57f8ab2	feat(design-systems): import a design system out of a rulebook's prose ... Milestone #254 step 3 (#2288). Reuses #251's prose extractor as the reader and adds the part that makes it an import rather than a list of claims. **The join is the whole trick.** A rulebook states a design system in two places and neither half is a token: one rule names the colours ("Obsidian #14171A (page bg, deepest surface)"), another names the custom properties (`--fs-obsidian/iron/slate`). The import pairs them on the word — `--fs-obsidian` ends with `obsidian` — which is the only reason it produces something usable instead of seventy empty names. The parenthetical becomes the token's purpose, which is the field a bare hex could never carry. **Prohibitions arrive as replacements, per the operator's reframe.** Rule 52 declares Parchment and forbids pure white in one breath, so the import emits "write --fs-parchment instead of #ffffff" — the same fact stated forwards. It attaches to the FIRST token that rule supplied a value for, not to every token of that rule, because claiming Vellum is also the replacement for white would be putting words in the rulebook's mouth. **A token the rulebook names but states no readable value for is still proposed, with an empty value.** Radius steps and type sizes are prose ("Small 4px") and nothing here parses them; inventing a parse per shape would be guessing. The name is real and the value needs a human, so the proposal says exactly that — and the UI leads with the COUNT of those, because an import that hid them would look more complete than it is. Preview is the default on both surfaces and in the UI. An import is a proposal: rulebooks are written aspirationally and some of what they describe was never built, so every entry carries the rule id and the sentence it came from and a reviewer can check the claim rather than trust it. Existing token names are never overwritten. A value already in the record was put there deliberately — most likely correcting this importer — so a re-run fills gaps and lists the rest as skipped, which also makes it safe to repeat. Colours the rulebook names but never exposes as a custom property produce no token: it never asked for one, and inventing a name would put something in the record no rule sanctions.	2026-07-30 21:23:32 -04:00
bvandeusen	3da40abcb8	feat(design-systems): declare what to write instead, rather than what not to ... Milestone #254 step 6, first half (#2295) — and this reframes the task rather than answering it. The operator's call: "in this case we should declare what should be used in place of pure white, it's not a prohibition it's what should be used in its place." None of the three options on the table (a constraints record / the panel reads both sources / negative token rows) was right, because all three kept the prohibition as a KIND OF THING. It isn't one. "Pure white is never text" is the shadow cast by a positive fact — text is Parchment — and a design system that stores what things ARE has no row for a ban because it never needed one. So `design_tokens` gains `supersedes`: the literal values this token should be written instead of. `--color-text-on-action` supersedes `#fff` / `#ffffff`. Same fact as the rule, stated forwards, and now actionable — a finding can say what to write rather than only objecting. It has to be DECLARED, not derived, and that is the crux: `#fff` and Parchment `#E8E4D8` are different colours, so no value-matching check could ever have connected them. That mismatch is precisely why the prohibition looked unrepresentable until it was turned around. `supersedes` cascades on EMPTINESS rather than on None. A child overriding a colour says nothing about which literals it replaces, and blanking the family's declaration there would silently disarm the check for every app that customises the token — while a child that states its own list replaces it wholesale. Two things this deliberately does NOT do: - It does not feed the drift panel. Superseded literals live in component CSS, which `designDrift.ts` cannot see and already documents as a blind spot. This is input for the source lint (#2277). Declaring it with nothing consuming it yet is honest; wiring it to a panel that cannot check it would not be. - It does not remove the panel's `prohibited_color` arm yet — that happens when the panel is repointed at a resolved system, which needs #2288 first. The declaration also exposes a missing token: most of the 67 hardcoded `color: #fff` (#2275) are text on a coloured action button, and the system has no token for that role at all. Every view hardcodes it. Declaring the token that was never there is the first real output of the operator's framing.	2026-07-30 21:16:45 -04:00
bvandeusen	143b968c5d	feat(design-systems): REST + MCP surfaces, at parity ... Milestone #254 step 4 (#2290). Eleven capabilities, both surfaces, one service. Design systems are owner-scoped top-level records rather than project-scoped ones, so these do not nest under /api/projects/ the way systems do — routes/rulebooks.py was the closer shape. The one exception is the project pointer, which is genuinely about a project: PUT /api/projects/<id>/design-system, PUT rather than PATCH because clearing it is a first-class outcome and not an omission. `/resolved` and `/tokens` are deliberately separate endpoints. One answers "what does this system CHANGE", the other "what does it end up BEING", and a system that overrides nothing has an empty token list and a full resolved set. Shipping only one would have made the other a client-side computation of exactly the kind the record model exists to remove. ResolvedToken.to_dict carries the SHADOWED contributions, not just the winner. Dropping them at the serialisation boundary would have discarded the one thing step 2 was built to preserve, and it would have been invisible — the payload still looks complete. Three sentinel translations on the MCP side, each tested, because an agent cannot omit an argument and a wrong mapping here is silent: - parent_id: 0 = unchanged, -1 = clear (become a family system), positive = set. Renaming a system must not silently re-root it. - order_index: -1 = unchanged, since 0 is a valid position. - value_by_mode: guarded on `is not None`, not truthiness, so `{}` can strip every mode from a token instead of being unreachable. DesignSystemCycle maps to 400 on REST and to a ValueError carrying the message on MCP — kept apart from 404 throughout. An agent told "not found" retries the same call; one told what the loop is can fix it. Two structural guards beyond the parity list: every endpoint must be reachable on the app (catching a decorator copied without its path, where the second handler silently never runs), and every public coroutine in the tools module must be registered (a tool written but never registered is invisible to an agent, and nothing else would notice).	2026-07-30 17:09:25 -04:00
bvandeusen	839d6902ad	feat(design-systems): resolve the chain, and keep the argument not the verdict ... Milestone #254 step 2 (#2287). `resolve_tokens` flattens a system's inheritance chain into its effective token set — walk to the root, deepest wins by token name. Pure and duck-typed, so a test states a whole hierarchy in literals and the service hands the same function ORM rows. **Provenance is stored as the contest, not the winner.** A ResolvedToken carries every system that offered a value, per mode, deepest first — `[0]` won and `[1:]` are what it shadowed. "Which system supplied this?" and "what did it override?" are then two reads of one list and cannot disagree, where a winner plus a separate provenance field would be two things to keep in step. **Merging is per (name, MODE), and that is the storage decision paying off.** A system that deepens one accent for light backgrounds while leaving dark alone owns `base` and still inherits `dark`. A token-level "overridden here" flag would have to lie about one of them, and the two-column shape could not have represented it at all. Metadata cascades separately by the same deepest-wins rule, with one exception: `order_index` treats 0 as UNSTATED rather than "first", because 0 is the column default. Reading it as a real value would let a colour-only override drag its token to the top of its group — a visible reshuffle in return for a change that touched nothing structural. One fix to step 1 while wiring this up: `_parent_map` is now scoped to the SYSTEM'S OWNER rather than the caller. A caller reading through a shared project owns no link in the chain, so the caller-scoped version would have handed them an empty forest and truncated the cascade to a single system — a page rendering with plausible wrong values and no error anywhere. The ACL already grants read along the whole chain; this is the loading side keeping that promise, and it now has a test naming the shared-project case. `BASE_MODE` moves from the model to the cascade module, where it belongs: it is a resolution rule, not a storage fact, and design_cascade.py deliberately imports nothing so both access.py and the service can depend on it.	2026-07-30 17:02:31 -04:00
bvandeusen	03b3998585	feat(design-systems): the model, the parent chain, and the guard on it ... Milestone #254 step 1 (#2286). A design system becomes a record Scribe holds rather than prose in a rulebook: a named set of tokens with an OPTIONAL parent, so a family system carries the house style and an app system carries only what it changes. Answering "what does this app alter?" is then `list its tokens` — nothing to compute. `parent_id` is the whole model. It replaces both an `always_on` flag (a family system is one with no parent) and a subscription join table (a project points at ONE system; the chain supplies the rest) — less schema than the rulebook shape it mirrors. Two decisions the task left open, settled here: - **Token values are JSONB keyed by mode**, not `value_light`/`value_dark` columns. The deciding argument was not flexibility, it was ambiguity: in a child system an unset mode means "inherit", in a root it means "not mode-dependent", and as columns both are NULL and the resolver cannot tell them apart. As a map, resolution is `{**parent, **child}` at every level with no special case for roots. Against it: queryability — but nothing filters tokens by value in SQL, so that buys a query no caller makes. - **`group_name` is free text, no CHECK enum.** Groupings are each design system's own vocabulary; a whitelist would bake one install's kit into the schema. No CHECK is introduced anywhere, so rule #36 does not fire. The cascade lives in `services/design_cascade.py` as pure functions over a `{id: parent_id}` map, importing nothing — which is what lets both the service and `access.py` use it without a cycle, and lets a test state a whole hierarchy in one literal. Cycles are refused on WRITE by walking up from the proposed parent (the cheap direction), and survived on READ by a visited-set, because a loop from a direct DB edit must truncate rather than hang. ACL (rule #78) is deliberately asymmetric: owning a system grants write, reaching one through a project you can see grants READ ONLY. An editor on a shared project must not be able to rewrite the family system every other project in that family resolves through. Also renames `services/design_system.py` -> `design_rulebook_import.py`. It is the #251 prose extractor, whose role is already scheduled to become a one-shot importer (#2288), and leaving it one character away from the new `design_systems.py` was a trap for every later session. Rule #115 throughout: nothing seeds a system or implies a default. An install with zero design systems is ordinary, not degraded.	2026-07-30 16:54:41 -04:00
bvandeusenandClaude Opus 5	d3ee24f239	feat(design-explorer): rulebook binding + prose→claims extraction ... Milestone #251 step 2 (#2259). The half of the drift panel that needed to be testable, which is why it is Python: the frontend has no test runner, so the fiddly extraction lives server-side and the browser only does set arithmetic over live token values. BINDING. A per-user setting `design_rulebook_id` names the rulebook that describes this install's design system. A setting rather than a column: no migration, discoverable in the Settings UI (rule #25), and honest about being a per-install choice rather than a property of the rulebook. No rulebook designated returns an empty set with rulebook_id: null — the NORMAL case for any install but the one that set it up (rule #115), which the client renders as an explanatory empty state rather than an error. The id comes back alongside the list so "not designated" and "designated but empty" stay distinguishable. EXTRACTION. No NLP. Rule statements are prose written for humans and should stay that way, so this takes only what is unambiguous in any prose — the hex colours and custom-property names a rule mentions. Anything subtler needs a rule author to opt into a structured form, deliberately left for when someone wants it. Three things earn their complexity: - SENTENCE-SCOPED NEGATION. A rule routinely states what the palette requires and what it forbids in consecutive sentences ("Parchment #E8E4D8 …, Vellum #C2BFB4 …. Pure white #FFFFFF is NEVER used."). Detecting negation across the whole statement would mark the required colours as forbidden — inverting the finding rather than missing it, which is worse. Per sentence, all four come out right. - HEX NORMALISATION is load-bearing, not tidiness. The rulebook writes #FFFFFF and components write #fff; if those don't compare equal the largest drift finding in the codebase — 67 hardcoded white text colours (#2275) — reads as zero. Alpha forms keep their alpha, since #fff and #ffff are different colours and collapsing them would manufacture equality. - SLASH SHORTHAND. Rulebooks write token families as --fs-radius-sm/md/lg/xl and --fs-obsidian/iron/slate/pewter. Both expand under one rule — prefix is everything up to and including the LAST hyphen of the first segment — which also handles --fs-dur-fast/base/slow. Verified against the real rule text: 18 tokens from three different shorthand shapes. how_to_apply is read alongside statement, because rulebooks routinely keep the statement declarative and put the concrete values in how_to_apply; ignoring it would miss the checkable half. Claims dedupe on (kind, value), first source winning, so a colour named by several rules is one expectation attributed to the rule that introduced it. Prose with nothing checkable yields nothing — most rules are judgement, not specification, and a panel that reported unparseable rules as problems would be unusable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 15:25:36 -04:00
bvandeusenandClaude Opus 5	293a14361a	fix(db): bind datetimes, not strings, when filtering timestamptz columns ... Closes #1727 and #2257 — the same bug in two services written months apart. AppLog.created_at is `timestamp with time zone`. asyncpg binds a Python str as VARCHAR and Postgres has no `timestamptz >= text` operator, so both of these raised when Postgres planned the query: notifications.check_due_tasks AppLog.created_at >= today.isoformat() logging.get_logs AppLog.created_at >= <raw request.args str> #1727 was the worse of the two because a per-user `except Exception` swallowed it: reminder emails silently never sent, and the only outward trace was an hourly traceback in the Postgres log. It has been open since 2026-07-19 with the diagnosis written and the fix never applied. #2257 has no swallowing handler, so it merely breaks the admin log viewer's date filters outright. notifications: `utc_day_start(day)` returns midnight UTC as an AWARE datetime. Deliberately not the bare `date` the original diagnosis suggested — comparing timestamptz to date does work via an implicit cast, but Postgres resolves that cast in the SESSION's TimeZone, so the dedup window would drift with a server setting nobody remembers is load-bearing. logging: `parse_filter_datetime()` converts the query-string value to an aware UTC datetime; unparseable input returns None so the filter is skipped rather than 500ing the viewer. It also fixes a bug the naive fix would have introduced — `date_to=2026-07-30` parses to midnight, so `<=` would exclude the entire day the user asked for. Date-only upper bounds now run to 23:59:59.999999, while a value carrying an explicit time is left as given. The guard is the point. This class is invisible to ordinary testing: the failure happens when Postgres plans the query, not when Python builds it, so no unit test that doesn't execute SQL can see it. tests/test_timestamp_filters.py fails CI on two shapes — 1. a local bound to .isoformat() compared against a *_at column (#1727) 2. a str-ANNOTATED PARAMETER compared against a *_at column (#2257) Shape 2 is the one that matters. Nothing in logging.py looks date-ish, so a guard built only from #1727's shape finds nothing there — which is exactly how the second instance survived. Verified by replaying both checks against the pre-fix files out of git: shape 1 catches `today_str`, shape 2 catches `date_from`/`date_to`, and the current tree is clean. Found by grepping for siblings after fixing #1727 — the third instance today of "the second place nobody checked", after #2245. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 12:53:57 -04:00
bvandeusenandClaude Opus 5	6ca215d2b6	fix(telemetry): record a pull on get_task, closing the surfaced→pulled loop ... Closes #2245. note_usage_events recorded `surfaced` for every auto-inject menu line regardless of kind, but `pulled` only from get_note, get_snippet and the REST snippet route. get_task recorded nothing. Auto-inject ranks kind-blind over a corpus that is overwhelmingly tasks and issues, so tasks are most of what it surfaces. Measured live, "write a function to debounce a callback in the frontend" returned three tasks and zero snippets — all three written as surfaced, none able to record a pull. surfaced and pulled only mean anything as a PAIR; the rate between them is what #1038 and #2085 gate on. So the gap sat exactly where the volume is, and the metric would have said "auto-inject surfaces things nobody opens" for its own dominant kind — an artifact of the instrumentation, not a fact about the feature, and one that pointed at a plausible-sounding wrong conclusion. get_note already carried a comment stating this was meant to cover ANY note kind precisely so tasks wouldn't look like dead weight. get_task is a separate tool in a separate module and never got the call — sibling drift, invisible because a missing side effect changes no return value. Guarded by a rule-#33 contract test that asserts, by source inspection, that every getter reachable from an auto-inject menu calls record_pulled. Source inspection because no behavioural test can see a call that isn't there. Not fixed here: the REST note/task detail routes still record nothing while the REST snippet route records `rest_snippet`. That asymmetry is real, but a human reading a note in a browser is arguably not the same event as an agent recalling one, and collapsing them could skew the signal the other way. Raised as a question for the retrieval survey instead of decided in passing. Pre-fix rows under-count task pulls, one-sidedly by kind — treat them as unknown rather than zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 10:35:23 -04:00
bvandeusenandClaude Opus 5	390846a3d5	fix(write-path): disclose cross-language prior art instead of hiding it ... Closes #2244. Retrieval matches on concept, and concepts are language-agnostic: asking about a TypeScript union-find scores 0.72-0.73 against a PYTHON snippet, comfortably over the 0.68 bar. That is useful — a different-language solution gives you the shape even when the code isn't reusable — but the menu line said nothing about it, so the reader either dismissed a good structural reference or pasted Python into a .ts file. Worth noting this predates the concept-query change: raw TS code already matched the Python snippet at 0.73, because the embedder reads identifiers and structure semantically rather than syntactically. The fail state has been shipping quietly; #2242 only made it an intended use rather than an accident. - knowledge._note_to_item projects `language` from the data mirror, same shape as the existing verification projection — a plain column read, no body parsing. - The semantic arm carries language through on the item it builds; it is the arm where these arise, since a snippet recorded AT the path you're editing is almost never in another language. - _prior_art_line folds it into the marker: [similar 0.72 · python]. Together with the score rather than after the title, because the two jointly are the judgement being offered. - One explanatory line is added to the menu, and only when something on it is actually tagged. Two deliberate calls: LABEL, DON'T FILTER. A stricter threshold for foreign-language hits would suppress exactly the shape-borrowing this exists for. They were never the problem; their being undisclosed was. ONLY CLAIM A MISMATCH YOU CAN ESTABLISH. _foreign_language returns "" when either side is unknown — unrecognised extension, or a snippet with no recorded language. A wrong "· python" is worse than no tag. Same-language hits stay unlabelled, so the common case keeps a clean line and the preamble stays off the menu entirely. Operator-typed language names fold through an alias table first (py/python3 → python, tsx → typescript, c++ → cpp); unrecognised names pass through lowercased, which still makes an unknown-but-equal pair compare equal. Trap found while building: _note() in the tests is a MagicMock, so `note.data` auto-created a truthy mock that would have rendered its repr into a menu line. Both test helpers now set data = None explicitly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 10:31:56 -04:00
bvandeusenandClaude Opus 5	57781770c3	feat(write-path): query snippets by concept, not by raw code ... Implements #2242, from the operator's question: would it make more sense to search by the concept of the snippet than by the code itself? It would, measurably. A snippet's embedded text is f"{title}\n{body}", and a snippet's body is composed markdown — When to use / Signature / Location, then the fenced code — so `when_to_use` appears TWICE in the vector and the document is prose-forward. The semantic arm was interrogating it with raw code carrying no prose at all. Measured on the deployed instance against snippet #2222: query built from score best unrelated separation raw code body 0.743 0.630 0.11 name + docstring 0.823 0.602 0.22 hand-written concept prose 0.835 0.583 0.25 A 12-word description beats a near-verbatim reimplementation of the function, and code-as-query RAISES the noise floor. It's also the cleanest explanation for the fragment miss recorded on #2223: a short excerpt carries almost no prose to match a document that is mostly prose. So build the query from what the code says it's FOR — declarations plus the first docstring / JSDoc / leading comment block — shaped as "name(params) — what it does", mirroring a snippet's own title, which is the form that measured 0.823. Server-side rather than in the hook: no manifest bump, so installed 0.1.20 plugins get this immediately; multi-language parsing in bash would be miserable; and it's unit-testable here. Two rules worth calling out, both measured rather than chosen: - NO DOC, NO REWRITE. A bare identifier is not a concept and scored 0.671 vs the code body's 0.743. Separation from noise is identical either way (0.113), but the absolute drops under the 0.68 bar, so preferring a bare name would convert a comfortable hit into a miss. Undocumented code keeps the raw payload. - The 48-char floor still judges the RAW payload, before the rewrite. A concept query is allowed to be shorter than the floor — that is the point, the best queries are short — but a sub-floor edit stays silent even with a docstring. Applying the floor after extraction would discard the best queries. Regex, not a parser: this is on a PreToolUse critical path and an Edit's new_string is rarely a valid module, so a miss must cost only a fallback. Every unrecognised language (Vue SFC, config files) degrades to exactly the previous behaviour. Telemetry now logs the concept query rather than the code, since retrieval_logs is what the threshold gets tuned from and the two aren't comparable. 0.68 is left alone: signal rises to 0.82 while noise FALLS to 0.58, so the bar sits mid-gap instead of near the edge. To be re-measured against the deployed instance rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-30 09:58:06 -04:00
bvandeusenandClaude Opus 5	1fb883b72f	fix(write-path): give the semantic arm its own threshold + a payload floor ... Closes #2223. The write-path prior-art trigger's semantic arm shared auto-inject's 0.55 threshold, which was tuned on prose. Code embeddings sit on a much higher similarity floor — any two Python-shaped payloads share keywords, indentation and structure — so 0.55 landed INSIDE the noise band. Measured against the live instance: near-duplicate of a recorded helper 0.73-0.74 true positive unrelated colour math / Vue SFC / CSS 0.55-0.63 false positive `x = 1` 0.58 false positive 6 of 8 probe payloads produced a nudge; 4 were noise. The margin gate couldn't help — _AUTOINJECT_BAND is relative to the top hit, so with a single hit it never engages. Two gates are now the write-path arm's own: - kb_writepath_threshold, default 0.68 — above every measured false positive, still 0.05 below both true positives. Auto-inject keeps 0.55; it was tuned on prose and is not implicated. The comment this replaces explicitly reserved the split for when telemetry showed the surfaces wanted different values, so this is the change it described, not a reversal of it. - WRITEPATH_MIN_CODE_CHARS = 48 non-whitespace chars, below which the semantic arm doesn't run at all. Whitespace is excluded so a deeply indented one-liner can't pass on padding. 48 sits under the smallest plausible reusable helper (~60) and well over a degenerate edit, so it errs toward keeping recall — precision is the threshold's job. This is the cheap half of the operator's #89 idea; the full length<->threshold curve stays open there, since they asked to brainstorm it rather than have a scale invented for them. top_k stays shared — "how many titles at once" means the same thing on both surfaces. The existing tests were passing `code="x"` / `code="def f(): ..."` into the semantic arm, i.e. exactly the payloads the floor now drops, so the gate tests were never exercising a realistic payload. They now use a REAL_CODE fixture, plus new coverage for the floor (trivial payload, padding, place-arm unaffected, real helper passes) and a guard on the constant itself. Settings UI carries the new knob with the reasoning in its hint, and the write-path checkbox no longer claims it shares the threshold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-29 23:26:59 -04:00
bvandeusenandClaude Opus 5	fe63f3985b	feat(snippets): un-merge — reverse one source out of a merged survivor ... Closes the last of milestone #232. The task said to settle the design before coding; here is what was settled and why. THE HAZARD. Restoring a merged-in source from the trash brought the record back but never stripped its locations off the survivor, so both claimed the same call sites and the reverse lookup read the duplicate claims as real. Subtracting blindly is not a fix: a location can arrive from a source AND genuinely be the survivor's own, and _normalize_locations dedups them into one, so blind subtraction would strip a call site the survivor owns. Same problem defeated partial un-merge — `merged_from` recorded ids, not which locations came from which source. THE ANSWER. Record per-source attribution AT MERGE TIME, where it is known exactly: each entry keeps only what that source ADDED, computed incrementally as sources fold in. Anything the survivor already had, or an earlier source already brought, is attributed to nobody. Both open questions fall out of that one change — partial un-merge is exact, and a survivor-owned location can never be stripped, because it was never attributed in the first place. The shape moved from [id] to [{id, locations, tags}]. Free to do: the corpus holds one snippet and zero merges, so there is no legacy data (rule #22). A bare int still normalizes to {"id": n} — not legacy tolerance, but because snippet_fields falls back to PARSING THE BODY when a row has no `data`, and the body's provenance line can only carry ids. Such an entry shows history and refuses un-merge with a reason rather than guessing. WHICH SURFACE. Neither option in the task, quite. Making trash-restore notice the merge would teach the generic trash path snippet semantics for one record type. Instead un-merge OWNS the restore: one operation, one authorization check, trash stays ignorant. Restoring by hand is still allowed and still leaves both records claiming the same places — so un-merge treats an already-alive source as the normal case and goes straight to the subtraction that repairs it. That is the state that motivated the feature, not an error. Adds trash.restore_entity(user_id, type, id) — the missing inverse of delete(), which returns a batch id callers don't keep. Restores the whole batch, since the batch is the entity plus its cascade. Refs #2165 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-28 18:42:29 -04:00
bvandeusenandClaude Opus 5	6db791965f	feat(snippets): near-duplicate finder — surface the sets worth merging ... #231's premise was unifying reusable things already scattered as one-offs. The create gate PREVENTS a new duplicate and merge_snippets CURES one you point it at, but nothing FOUND the duplicates already in the record — someone had to notice them by hand, which is the exact failure the Drafter exists to remove. One indexed self-join over note_embeddings, not an N² Python scan: pgvector's cosine distance is the same operator semantic search uses, so a similarity floor is a distance ceiling and the work stays in Postgres. `left.note_id < right.note_id` yields each unordered pair once and drops the self-pair that would otherwise dominate the ranking. Pairs are collapsed into merge SETS by connected components. Transitive on purpose: A~B plus B~C puts all three together even when A and C don't directly clear the bar, which is what merge actually does (it folds every source into one survivor). The cost is that a chain of mild resemblances can rope in a member that isn't really alike — so the UI presents a set as a proposal, shows the members, and never merges without a confirm. Two scope decisions worth naming: - OWN snippets only. merge_snippets requires one owner across the set, so surfacing someone else's would propose a merge that cannot be performed. The report is bounded by what the operator can act on, not what they can see. - Threshold defaults to 0.82, LOOSER than the write gate's 0.90, and is a setting rather than a constant (rule #25). The gate blocks a create and has to be unforgiving of noise; this only suggests a merge under review, so it must reach further or it would never surface the pairs the gate already let through — which are precisely the ones that accumulated. Fixes a real bug in the merge flow while wiring the UI: selectedList filtered the selection against the CURRENT PAGE, and doMerge derives its source ids from that list. A corpus-wide suggested group with off-page members would have rendered incomplete and silently merged only the visible subset. A group under review is now the authority for that list. Refs #2088 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-28 18:32:40 -04:00
bvandeusenandClaude Opus 5	84c5c0dc81	fix(snippets): satisfy the parity guards the drift check tripped ... Two CI failures, both the guards working as intended rather than defects in them: - The MCP tool-name manifest and the REST route/service parity lists are explicit, so a new capability has to be added to both surfaces or the test fails. verify_snippet / verify_snippet_route / record_verification added, plus an assertion that the `verification` filter reaches both callers — a verdict an agent records must be visible to the human looking at the same corpus, or the two surfaces disagree about what's rotten (rule #33). - vue-tsc rejected two object-literal lookups keyed by the status union: `status` includes "ok" and "unverified", and an object literal has to enumerate every member even just to say "nothing to show for these". Typed as Record<string, string>, which is what the ?? "" fallback already assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-28 18:24:48 -04:00
bvandeusenandClaude Opus 5	35f3f09d12	feat(snippets): drift check — verify a snippet still matches its source ... A recorded snippet points at a repo · path · symbol that WILL rot: files move, symbols get renamed, implementations diverge from the copy stored here. Nothing detected any of it, so a record degraded silently from "canonical reference" to "confidently wrong" — worse than no record, since it is surfaced with the same authority either way. WHERE THE CHECK RUNS. Agent-side, which the task flagged as the design question to settle first. Scribe has no checkout of the operator's repos and must not acquire one: giving the server repo access would make every install a credential problem and break instance-agnosticism (rule #115). The agent already has the working tree, so it does the comparing; the server remembers the verdict, makes it queryable, and knows when it has expired. New MCP tool verify_snippet teaches the four-step procedure and records the result; a REST endpoint mirrors it so the UI can clear a marker after a manual fix. WHY THE VERDICT CARRIES A CODE HASH. A verdict describes the code it was checked against. Invalidating it on edit means deciding which edits count — a when_to_use tweak shouldn't void a code check, a rewrite must — which is fiddly and easy to get subtly wrong, and easy for a new write path to forget entirely. Stamping the verdict with a hash sidesteps all of it: one whose code_sha no longer matches is self-evidently expired, computed at read time, no invalidation branch to maintain. That makes "expired" the interesting filter case. It is not `drifted` (nothing was found wrong) and not `unverified` (a check did happen), yet it plainly needs looking at — so `verification=attention` covers both. To keep that one index-served predicate rather than a post-filter that would make the pagination total a lie, data now also mirrors the CURRENT code's fingerprint as data.code_sha, and a jsonpath compares the two fields within the row. The filter is implemented in both dialects, SQL and Python, for the same reason the location filter is: the semantic arm's candidates arrive already fetched. A merge deliberately carries no verdict forward — the survivor's code is a union of several sources, so no prior check describes it, and unverified is the honest answer. UI: a danger-toned drift badge on each card (an actively misleading record outranks a merely unused one), and a "Needs attention" filter. Its empty state says plainly that never-verified snippets don't appear there — otherwise `attention` would mean "everything" on day one and be useless as a worklist. Refs #2086 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-28 18:20:33 -04:00
bvandeusenandClaude Opus 5	2b85443dd1	feat(snippets): usage signal — was a surfaced record ever actually pulled? ... retrieval_logs answers "what did the ranker return, at what scores" — the right substrate for tuning a threshold. It cannot answer the question the snippet corpus actually needs: did anyone open this? A snippet nobody opens is not neutral. It takes a slot in every future auto-inject menu and crowds out something useful. Adds note_usage_events (migration 0071): one row per note per event, either 'surfaced' (we put its title in front of an agent) or 'pulled' (someone opened it in full), tagged with which surface produced it. Closes the gap #2082 recorded against this work. The write-path PLACE arm carries no score, so it has no home in retrieval_logs — folding it in would corrupt the score distribution that table exists to capture. The result was that the arm firing on the STRONGEST claim ("there is already a canonical helper in this exact file") was the one arm nobody could measure. Both arms now emit usage events under distinct sources, so their pull-through rates are finally comparable. Deliberate departures from the task as written: - Not in-session correlation. The original framing was "correlate result_ids against a later get_note in the same session." There is no session identity server-side — the MCP endpoint is stateless and the hooks send no session id — and adding one would mean threading an opaque client-supplied token through every read path. Two independent counters answer the question without it: surfaced 40×, pulled 0 is dead weight regardless of how those events distribute across sessions. - Pulls record at the ENTRY POINTS (MCP tools, REST detail route), not in snippets_svc.get_snippet, which update and merge also reach. Counting those would inflate precisely the number meant to say "someone chose to look at this." - get_note records for every note kind, not just snippets. The auto-inject menu surfaces tasks and processes too; scoping this to snippets would pin those at zero pulls forever and make them read as dead weight next to snippets that merely had a counter. Surfaced in the Snippets list as an "N/M used" badge, warning-toned once a record has been offered 3+ times and never opened, with the tooltip saying what to do about it (usually: its "when to reach for it" doesn't say when). No badge at all below one surfacing — "0/0" reads as a verdict when it's an absence of evidence. Also returned from MCP list_snippets so the agent can see dead weight without opening the UI. Telemetry keeps the retrieval_telemetry contract throughout: writes are fire-and-forget, reads degrade to zeroes, and no path can raise into the surface it observes. Refs #2085 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs	2026-07-28 18:09:17 -04:00
bvandeusen	e0328f2b1c	feat(plugin): write-path trigger — offer prior art before code is rewritten ... The milestone headline. Auto-inject fires on the operator's prompt; the moment reuse is actually lost is later, when the agent decides mid-task to write a helper. A PreToolUse hook on Write|Edit now fires there. Channel: `additionalContext` with NO permissionDecision, so the note reaches Claude beside the tool result and the write is never blocked — a recall aid must not be able to stop the operator's work. Plain stdout would have been invisible to the model, and deny/ask would have made a nudge into a gate. Two arms, different in kind: - BY PLACE — a snippet recorded at this path (or its directory) is prior art by definition, not resemblance, so it is neither scored nor thresholded. This is what #2083's reverse lookup was built to answer. - BY MEANING — semantic search restricted to snippets (new `note_type` filter on semantic_search_notes) over the code about to be written. Place ranks first; the top-k cap spans both arms. Gates carried over from milestone 93 verbatim: threshold, margin, session dedup, titles-never-bodies. Own `source='write_path'` in retrieval_logs so precision is tunable separately — the docstring records that the place arm is unlogged and hands that to #2085. Its own on/off in Settings but the SAME threshold/top-k: one "how loud may Scribe be" knob is easier to reason about than two that drift, and splitting them later is then a data-backed change rather than a guess. Details worth keeping: the hook sends a REPO-RELATIVE path because that is how locations are recorded; the git remote resolves to a project and is never used as the location `repo` filter (different namespaces, would silently match nothing); the endpoint stays a GET because a read-scoped API key cannot POST and every other hook depends on that. plugin.json 0.1.17 -> 0.1.18. Refs #2082, milestone #232.	2026-07-28 09:08:17 -04:00
bvandeusen	083944f0fd	fix(snippets): backfill must also catch JSON null, not just SQL NULL ... Integration lane caught it (run 2984): the backfill reported 0 rows to fill and left `data` unset. `IS NULL` was the whole predicate, but a JSONB column has two empty states. SQLAlchemy's JSON types default to `none_as_null=False`, so assigning Python `None` persists the JSON encoding of null — `IS NULL` walks straight past it. Migration 0070 left genuine SQL NULLs, so the product path was right; the test was constructing the wrong shape with `data=None`. Fixed both ways, because both states mean "no usable mirror": - predicate is now `data IS NULL OR jsonb_typeof(data) = 'null'`; - the legacy-row test OMITS `data` (a real SQL NULL, 0070's actual shape), and a second test covers the JSON-null shape and asserts the premise with `jsonb_typeof` rather than assuming it. Refs #2083.	2026-07-27 23:12:48 -04:00
bvandeusen	dd1b5e5ddb	feat(snippets): reverse lookup — find snippets by repo/path/symbol ... "What canonical helpers already live in this file?" was unanswerable: location lived only in the body markdown. It is now a jsonpath containment query over the `notes.data` mirror added by migration 0070. - One predicate in two dialects in services/knowledge.py: SQL (`data @?`, applied in the browse arm and the keyword arm before count/pagination, so totals stay honest) and Python (`location_matches`, for the semantic arm which post-filters candidates it already holds). Both must change together. - Parts are ANDed within a SINGLE locations entry — repo A in one entry and path B in another is not "recorded at A/B". `path` also matches as a directory prefix, via jsonpath `starts with` rather than `@>`, which the same GIN index serves. - `repo`/`path`/`symbol` reach the service, the REST list and the MCP tool under one name with one default (rule #33); the MCP docstring teaches the place form, and so does the reusing-code skill (plugin.json bumped). - UI: a Location disclosure beside the snippet search, with its own empty state — "nothing kept there, so what you're about to write is new." Settles #2083's open question (pre-0070 NULL `data`) by backfilling after all: `backfill_snippet_data` runs at startup, deriving the mirror from the body with the same parser the read path trusts. 0070's caution was about mangling a hand-edited body; this never touches the body. The alternative was a permanent second body-regex arm, or a query that silently answers "nothing here" for an old snippet and gets the helper written twice. Refs #2083, milestone #232.	2026-07-27 23:09:37 -04:00
bvandeusenandClaude Opus 5	0d396de215	feat(snippets): record merge provenance on the survivor ... Merge kept the target's fields, unioned locations and tags, and trashed the sources — recording nothing about what it absorbed. If a variant handled an edge case the survivor doesn't, that difference left the visible record entirely; recovering it meant knowing to go digging in the trash. The survivor now carries `merged_from`: a "**Merged from:** #2, #3" line in the body for humans, and the same list in the `data` mirror for queries, written from one value like every other field (#2087). It accumulates rather than replaces — a target merged twice keeps both histories — and skipped sources (cross-owner, per #231) are excluded, so the record never claims to contain something it never absorbed. Ordinary edits carry it forward. update_snippet recomposes body and mirror from scratch, so an omission there would silently erase the history on the next unrelated edit; that path is pinned by its own test, including the pre-0070 case where the body line is the only copy. Surfaced in the snippet detail view as a "Merged from" row — the view renders parsed fields, not the raw body, so the body line alone would have been invisible to the operator (rule #27). Un-merge, the other half of #2087, stays open: restoring a source from trash still doesn't strip its locations off the survivor, and what partial un-merge should mean is a design question, not a coding one. `merged_from` is the record that makes it tractable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-27 15:23:41 -04:00
bvandeusenandClaude Opus 5	9fa474b3c4	fix(mcp): make get_note / get_task share-aware ... The injected menu tells the agent to open any hit with get_note(id), and that menu can list a collaborator's record reached through a shared project. The fetch was still owner-only, so those lines answered "not found" — for a record the same user opens fine in the browser. The agent path was strictly narrower than the web path for the same id. Same boundary miss as #2093: the list side was widened for sharing, the fetch side wasn't. Both tools now resolve through get_note_for_user, apply the trash filter themselves (permission resolution says nothing about liveness), and attach describe_provenance so a shared record arrives marked as someone else's rather than passing as the caller's. routes/tasks.py's parent-title lookup had the same narrowness: a shared subtask rendered as an orphan when its parent was equally shared. Four fetch tests across three files were patching notes_svc.get_note and had to be retargeted — note 2109's third sub-case, caught by grepping tests/ for the old name before pushing rather than by CI (#2159). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-27 15:18:03 -04:00
bvandeusenandClaude Opus 5	8977bed28d	feat(inject): label the record kind on each auto-inject menu line ... Every injected hit rendered identically, so a recorded snippet was indistinguishable from a stray dev-log in the one place prior art most needs to stand out. Each line now carries its kind — [snippet], [process], [task], [issue], [note] — and the header says "records" rather than "notes", which it can no longer claim. Task-ness wins over note_type in the marker: "there's an open issue about this" is the more useful thing to know at a glance. Still title-first: the marker is metadata already on the ORM object, so no extra query and no bodies (#2084). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-27 15:18:03 -04:00
bvandeusenandClaude Opus 5	fc9c8119a2	test(snippets): fix the round-trip test's shared kwargs spread ... Run 2923: 1 failed, 408 passed. My test bug, not a code one — I spread one `fields` dict into both compose_body and compose_data, but compose_body takes no `name` (the name lives in the title). Each serializer now gets its own argument list. The migration itself was fine: the integration lane ran 0001→0070 on pgvector/pg17 green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-26 18:28:21 -04:00
bvandeusenandClaude Opus 5	cca40affe4	feat(snippets): add notes.data JSONB — the indexed mirror of snippet fields ... Milestone #232 step 1 (task #2081). Takes the enabler first rather than the write-path trigger: reverse lookup, drift checks and the duplicate finder all need to QUERY structured fields, and building them on body-regex first means writing them twice. #227 deferred this bag "unless body-convention ergonomics prove insufficient" — answering "which snippets live in this file?" by scanning every snippet and regexing its body is that condition being met. Migration 0070 adds `notes.data` (nullable JSONB) + a GIN index. The body is UNCHANGED and still what gets embedded and read by humans; `data` mirrors the same facts in a shape Postgres can index. Code is deliberately not copied into it — the body holds it, and duplicating a blob into the column we index around would be waste. - compose_data() builds the mirror, omitting empties so the column stays sparse - snippet_fields() prefers `data`, falling back to parsing the body. Rows written before 0070 have no `data` and are never backfilled, so a hand-edited body stays authoritative for them with no conversion deadline - create / update / merge all write body and mirror from the same merged field set, so the two can't drift; merge in particular has to grow the mirror with the survivor's location set or a merged snippet would be unfindable at the very call sites the merge just recorded Named `data`, not `metadata`, because that collides with SQLAlchemy's declarative Base.metadata — which is why the pre-0069 model had to map an awkward `entity_metadata` attribute. Not a revival of the column 0069 dropped: different name, different purpose, nothing reads the old shape. Two test fakes needed an explicit `data = None`: snippet_fields prefers `data` when truthy and an auto-MagicMock attribute is truthy, so every parsed field would have come back a MagicMock. Checked every fake reaching snippet code this time rather than waiting for CI (note 2109). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-26 18:25:55 -04:00
bvandeusenandClaude Opus 5	4b5d9005fd	test(processes): retarget the update_process patch at get_note_for_user ... Run 2899: 1 failed, 402 passed. test_update_process_rejects_non_process_note still patched notes.get_note, which update_process no longer calls now that it resolves shares — so the real get_note_for_user ran and reached for a database. Retargeted at get_note_for_user (which returns (note, permission)), and added the companion case the new behaviour deserves: a viewer grant is refused with the read-only reason and never reaches update_note. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-26 00:07:08 -04:00
bvandeusenandClaude Opus 5	3ffdbbc521	fix(acl): align MCP writes with the share model — editor edits, viewer doesn't ... Option B, per the operator. Closes the last inconsistency from the ACL work. The agent path had drifted into an indefensible position: delete_snippet honoured editor shares (I made it share-aware so widening the read wouldn't let a VIEWER trash things) while update_snippet still resolved through the owner-only notes.get_note. So through an agent you could destroy a colleague's snippet but not improve it — and the refusal claimed "not found" for a record you could plainly open. Now update_snippet, merge_snippets and update_process all resolve the read scope and then require can_write_note, matching the REST routes and the sharing UI's own promise that viewer / editor / admin are distinct grants. A viewer grant is refused with the actual reason ("shared with you read-only — ask its owner for edit access, or record your own version"), because not-found would send an agent hunting for a missing id instead of recording its own copy. Authorised writes are performed as the OWNER, since the underlying note update is owner-scoped and a shared editor's own id would match nothing. Merge additionally requires each source to share the TARGET'S owner and to be writable by the caller — merging trashes the source, so read access isn't enough, and cross-owner merge stays out of scope (#231). Sources failing either test are skipped rather than half-merged. A record the caller cannot read at all still returns not-found rather than forbidden, so the error can't be used to confirm that an id exists. Also fixed _fake_snippet's missing user_id proactively — the same auto-MagicMock-reads-as-foreign trap that broke CI twice (see note 2109). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-26 00:04:48 -04:00
bvandeusenandClaude Opus 5	e9cd3435ad	test(acl): give search/inject fixtures real owner ids; fail soft on name lookup ... Run 2892: 3 pre-existing tests broke, 392 passed. Same shape as the last breakage — a DB-touching call landed in a path unit tests exercise, and their fixtures had auto-MagicMock user_id attributes that compare as "someone else's", sending the code off to look up a username. Fixed the fixtures rather than the assertion: _fake_note in the search tests and _note in the plugin-context tests now take a real user_id defaulting to the caller those tests bind. That makes "is this shared?" meaningful in both files instead of accidental, which is what the new provenance behaviour actually needs from them. Separately, and not as cover for the above: owner_names_for now fails soft. A lookup error yields no names and callers render "another user". The part that matters — that the record is NOT the caller's — comes from comparing owner ids, not from this query, so losing an attribution is cosmetic where failing the whole search would not be. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 23:10:12 -04:00
bvandeusenandClaude Opus 5	ef1dbdfc86	feat(acl): unify the retrieval scopes; mark shared rows in Knowledge browse ... Closes #2092 and the Knowledge-browse provenance gap. The two halves of a hybrid search disagreed: the keyword half honoured shares while the semantic half was pinned to NoteEmbedding.user_id, so a shared record was findable by wording and invisible by meaning — the case a semantic search exists to serve. semantic_search_notes now scopes on Note via a `scope` parameter, and each of its five callers declares which kind of act it is: mcp/tools/search.py read the agent asked routes/search.py read the user typed it knowledge.py (semantic) read matches the keyword half beside it plugin_context.py browse nobody asked; never a one-to-one share dedup.py own a verdict that blocks a write must not hinge on another person's notes That last one is the reason this isn't a single global widening: the dedup gate returns "update the existing one instead", so matching a stranger's record would refuse a legitimate create and point at something the caller can't edit. Scope defaults to "own" so a caller that forgets is wrong in the safe direction, and an unknown scope raises rather than falling back — a typo there would be a data-exposure bug. Auto-inject keeps the browse scope, which still admits a collaborator's note via a shared project. Its menu line is the only provenance an agent sees, so a foreign hit now reads: #12 "Title" (0.71) - shared by alex, treat as a suggestion. MCP and REST search results carry shared/owner too. Knowledge browse: the feed hydrates cards from /api/knowledge/batch rather than the list route, so both paths label rows now, and KnowledgeView shows "by <owner>" on records the viewer doesn't own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 23:06:52 -04:00
bvandeusenandClaude Opus 5	8b069cc93f	fix(acl): make the visibility predicates pure builders; repair CI ... Run 2888 failed with 7 tests down, both causes mine. The real problem was a design flaw, not the tests: readable_notes_clause and browsable_notes_clause each opened their own DB session to fetch the caller's group ids. That made them unmockable at the call site, so every unrelated service test suddenly had to know they existed and stub them — four modules broke the moment a service started calling one, and one of my own stubs patched the wrong name (readable_* where the code had moved to browsable_*). Fixed at the root: group membership is now a SUBQUERY rather than a fetched list, so both clauses are synchronous pure functions with no session. One fewer round-trip per query, membership folded into the statement the caller was already running, and nothing for callers' tests to mock. The "no groups means no group arm" special case disappears too — an empty subquery simply matches nothing. Also: _fake_note in the process tool tests had no real user_id, so its auto-MagicMock attribute reached session.get(User, ...) through the new provenance check and SQLAlchemy rejected it. The fixture now takes a real user_id defaulting to the bound caller, which makes "is this shared?" meaningful, and gains a case asserting another user's process comes back flagged. Test assertions on compiled SQL are deliberately loose about formatting: the local env has no SQLAlchemy (rule #10), so they check that the arms exist rather than guessing at exact rendering. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 22:48:01 -04:00
bvandeusenandClaude Opus 5	04b58ce01e	feat(acl): shared records are search-only and always labelled as someone else's ... Narrows the b7d6fc7 widening per the operator's call, and fixes a regression it introduced. Decision recorded as note 2094. Two scopes now, deliberately different: readable_notes_clause — everything the ACL permits, including records reached only via a direct/group note share. For EXPLICIT acts: a search the caller typed, a fetch by id. browsable_notes_clause — the caller's own records plus anything in a project they can reach. For PASSIVE surfaces: browse lists, facet counts, the process->skill manifest. The split is a trust boundary. Anything appearing unasked — in your own list, your own counts, or as a skill installed on your machine — reads as material you endorsed. A one-off someone shared with you hasn't earned that standing, so it waits until you go looking. This also dissolves the shared-Process problem by construction rather than by special case: the manifest is a passive surface, so a directly-shared Process is never installed as an auto-surfacing skill. Regression fix (#2093): b7d6fc7 widened the list queries but left the fetch path owner-only, so on the MCP path a record could be listed and then not opened — get_snippet raised not-found, get_process couldn't resolve, and the manifest emitted stubs whose get_process call would fail. snippets.get_snippet and notes.resolve_process now resolve the read scope. delete_snippet gained an explicit can_write_note guard, since being able to SEE a shared snippet must not imply being able to bin it. Provenance, so nothing arrives looking like the operator's own work: - access.describe_provenance / label_shared_items add shared/owner/permission; labelling costs no query when everything is the caller's own. - MCP: get_snippet, list_snippets, get_process and list_processes carry it, and get_process now says outright NOT to follow a shared process verbatim — its follow-as-written contract was the sharpest instance of the problem. - The skill stub for a shared Process names its author and asks for a go-ahead, instead of describing it as "the operator's saved Scribe process". - Policy stated once in the MCP _INSTRUCTIONS and the reusing-code skill: a shared record is that person's suggestion, weigh it, attribute it, ask before adopting it. - UI: shared snippets show "by <owner>" in the list and a notice above the code in the detail view, reusing SharedWithMeView's vocabulary. Plugin 0.1.15 -> 0.1.16 (skill text changed). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 22:43:00 -04:00
bvandeusenandClaude Opus 5	b7d6fc7e5d	fix(acl): make shared records findable, not just openable (#2079) ... Option A of the #2079 fork, per the operator's call: widen the scope in query_knowledge for every caller rather than special-casing the snippet path. Until now the list/search queries filtered on Note.user_id alone, while get_note_permission resolved shares properly. The result: a record shared with you could be OPENED by id but never FOUND — invisible in Knowledge browse, in snippet and process lists, and in the facet counts beside them. - services/access.py gains readable_notes_clause(user_id): the same resolution get_note_permission does per row (ownership, direct share, group share, inherited project share), expressed as set membership so a list query can use it in one statement instead of O(n) permission round-trips. - services/knowledge.py routes every query through it — query_knowledge, the keyword half of the hybrid search, query_knowledge_ids, get_knowledge_by_ids, get_knowledge_tags and get_knowledge_counts. The facets follow the list, or a tag visible in the list would filter it down to nothing. - list items now carry user_id, since these lists can be mixed-ownership and the client has no other way to mark what isn't yours. Reaches four surfaces: the Snippets list (the original report), Knowledge browse, list_processes, and the plugin's process manifest — so a Process shared with you now also syncs as a local skill stub, which is the point of sharing one. NOT widened: semantic_search_notes, which scopes by NoteEmbedding.user_id and also backs auto-inject and the search MCP tool. Widening it would put another user's content into your agent context automatically — a product decision, not a bug fix. Consequence until that call is made, marked at the call site: a shared record is findable by wording but not by meaning. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 19:19:18 -04:00
bvandeusenandClaude Opus 5	b33e2a79c6	fix(snippets): close the recall-surface gaps found reviewing the Drafter ... Four defects from the 2026-07-25 review of the recall (#227) and merge (#231) milestones. The theme: a snippet could be recorded but not fully corrected, and the agent and web surfaces had drifted apart. - #2076 language was mis-derived from the first caller tag, so a snippet created with tags and no language read that tag back as its language — corrupting the tag set and the code fence on the next update. Only the FIRST tag can carry the language, since compose_tags emits [language, "snippet", *caller]. - #2077 MCP update_snippet mapped "" to "unchanged", so no field could ever be cleared and no snippet detached from its project. Now an omitted field is left alone, an empty string clears, and project_id follows the -1 = detach convention. A service-level UNSET sentinel keeps None available as the clear. - #2078 surface parity: adds delete_snippet (MCP had none, so a wrong snippet could not be retired by the agent that recorded it), locations on MCP create and update, system_ids through the REST routes and the editor, and the near-duplicate gate on REST create with a "record it anyway" escape. - #2079 project scoping: list_snippets takes project_id through the service, the MCP tool and the REST route, defaulting to every project — reaching across projects is the point when the helper you need was written elsewhere. Sharing the list across owners is deliberately NOT in here: query_knowledge is shared with the Knowledge browse surface, so widening it changes behaviour well beyond snippets. Left open on #2079 for a scope decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt	2026-07-25 19:00:05 -04:00
bvandeusenandClaude Opus 4.8	85625de394	feat(scribe): merge_snippets — unify found one-off snippets into one canonical ... Step 2 of the snippet-merge milestone (#231). The dedup gate only PREVENTS new near-duplicates; merge is the CURE for the ones already scattered. - services/snippets.py: merge_snippets(user_id, target_id, source_ids) — keep the target's scalar fields (name/when_to_use/signature/language/ code), union the sources' locations + extra tags onto it (so the survivor carries every call site as a location), trash the sources (recoverable), re-embed the survivor. Pure merge_snippet_fields() factored out for unit testing. Returns (survivor_note, merged_ids). - mcp/tools/snippets.py: merge_snippets(target_id, source_ids) tool (5th), and a create_snippet dedup-path nudge toward merge over a forced copy. - routes/snippets.py: POST /api/snippets/<id>/merge {source_ids} — share- aware (can_write target + every source, rule #78) with a same-owner guard (cross-owner merge is out of scope). - plugin reusing-code skill + MCP _INSTRUCTIONS: point found-duplicates at merge as the cure (rule #119 surfaces, not a Scribe rule). plugin.json 0.1.13 -> 0.1.14 in the same change (the #1040 marketplace-ship lesson). - Tests: pure merge-helper union/dedup; MCP tool (requires a source, survivor+merged_ids, not-found); route handler + 5-tool registration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t	2026-07-25 17:11:47 -04:00
bvandeusenandClaude Opus 4.8	eb400a521b	feat(scribe): multi-location snippet body convention (backward compatible) ... Step 1 of the snippet-merge milestone (#231). A snippet that unifies N found one-offs carries N locations (one per call site), so `location` becomes a list. Ships on the body-convention — no migration, swappable to the deferred `data` JSONB later (as #227 decision 5 anticipated). - compose_body: renders `**Location:**` for a single location, a `**Locations:**` bullet list for several. Accepts a `locations` list; the single repo/path/symbol params remain as a one-location shorthand (create path + existing callers/tests unchanged). - parse_snippet_fields: reads BOTH the new `**Locations:**` list block AND the legacy single `**Location:**` line (tolerant, never raises); returns a `locations` list and mirrors the first into repo/path/symbol for back-compat (rule #33). - update_snippet: gains a `locations` param — replaces the whole set; else a legacy single triple overlays onto the first location; else kept. - Tests: multi-location round-trip, singular-vs-plural label, legacy single-line parse, normalize dedup/empty. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t	2026-07-25 17:07:09 -04:00
bvandeusenandClaude Opus 4.8	d257c0fd67	feat(scribe): snippet management UI + REST routes; embed snippets on create ... Step 6 of the Drafter recall milestone (#227): a human-facing surface for the reusable-code snippets that agents record via MCP, plus the REST API behind it. Backend embeds snippets inline on create/update so they're recallable immediately, not only after a restart. Backend: - routes/snippets.py: GET/POST /api/snippets, GET/PATCH/DELETE /api/snippets/<id>. Share-aware per rule #78 (get_note_for_user + can_write_note), writes performed as the owner; list owner-scoped — mirrors routes/notes.py. Registered in app.py. - services/snippets.py: embed on create/update via a _embed_snippet fire-and-forget helper, covering BOTH the MCP tool and the REST route by construction. A snippet's value is immediate recall, so it can't wait for the startup-only backfill (see issue: MCP create path doesn't embed inline for notes/tasks generally). - tests/test_routes_snippets.py: structural registration + handler/service contract + PATCH-field ↔ update_snippet-kwarg parity (rule #33). Frontend (Vue 3 + TS): - api/snippets.ts: typed client, modeled on api/systems.ts. - views: SnippetListView (search, skeleton/empty/error states), SnippetDetailView (read + copy-to-clipboard, ConfirmDialog delete), SnippetEditorView (create/edit all fields, Ctrl/Cmd+S, Esc, autofocus, validation). v1 quality per rules #24/#27. - router: /snippets, /snippets/new, /snippets/:id, /snippets/:id/edit. - NoteType union widened to include 'snippet'; Snippets nav link added to AppHeader (desktop pill bar + mobile menu). - Design system: Moss --color-action-primary for action buttons, accent --color-primary reserved for tags/brand (Hybrid rule); focus rings; JetBrains Mono for code/name/signature/location. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t	2026-07-25 14:16:02 -04:00
bvandeusenandClaude Opus 4.8	1942913366	feat(scribe): add snippet recall — note_type='snippet' service + MCP tools ... Record reusable functions/components once so they surface via the existing semantic search + title-first auto-inject, instead of re-solving as one-offs. A snippet is a Note with note_type='snippet' (no schema change): note_type is free-text, and semantic_search_notes never filters by type, so snippets join the recall/auto-inject pool the moment they're embedded. Structured fields (name/language/signature/location/when_to_use/code) are stored via a body convention — title = "name — when to use" (what auto-inject surfaces), language + "snippet" as tags, templated markdown body — keeping storage swappable later without changing the tool/UI contract. - services/snippets.py: compose/parse helpers + create/get/list/update wrappers over notes_svc (dedup + System association reused). - mcp/tools/snippets.py: list_snippets / create_snippet / get_snippet / update_snippet, registered in tools/__init__.py. - unit tests for the serialize/parse round-trip and the MCP tool surface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pa2EsuB54BuWQ8GfJq9c7t	2026-07-25 11:00:33 -04:00
bvandeusenandClaude Opus 4.8	dd60244429	test(trash): update model-count assertions after Event removal ... restore() / purge_expired() now iterate 6 soft-deletable models (Note, Project, Milestone, Rulebook, RulebookTopic, Rule) — Event was removed with the calendar surface. Adjust the two count-coupled assertions (execute.await_count 7→6; the rowcount list drops a zero so the sum stays 4). Caught by CI run #2580. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPtbSzA4JLMAKgFZ8VTg7Q	2026-07-19 13:32:30 -04:00
bvandeusenandClaude Opus 4.8	b49efdcb11	refactor(scribe): retire calendar/events + person/place/list entities (backend) ... Narrow Scribe to a Claude-Code work system-of-record (milestone #194, decision note #1759). Wholesale removal per rule #22 — backend + schema half. Calendar/events + CalDAV: delete models/event, services/{events,caldav, caldav_sync}, routes/events, mcp/tools/events; strip event branches from backup (bump v3->v4), dashboard (upcoming_events), trash, recent, and the mcp server read-only allowlist + instructions. Typed entities (person/place/list): delete mcp/tools/entities; drop the notes.metadata (entity_meta) column from model/service/routes and the knowledge browse service. note_type STAYS — it also marks 'process' notes. Scheduler: event_scheduler -> recurrence_scheduler, keeping only the recurring-task spawn job (drops event reminders + CalDAV sync). Schema: migration 0069 drops the events table + notes.metadata column + orphan caldav settings rows (faithful downgrade recreates them). KEEP: recurrence.py (task recurrence), notifications task reminders, graph view, and every work surface. Frontend + plugin/docs true-up follow next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BPtbSzA4JLMAKgFZ8VTg7Q	2026-07-19 13:29:14 -04:00
bvandeusenandClaude Opus 4.8	8126db3203	feat(plugin): knowledge auto-inject (Path A) — title-first per-turn awareness ... New UserPromptSubmit hook (scribe_autoinject.sh) + GET /api/plugin/retrieve that surface the TITLES (never bodies) of the few notes clearing four anti-bloat gates: a per-user confidence threshold (stricter than pull search), a margin gate, per-session dedup (exclude_ids), and a top-k ceiling. Each retrieval is logged to retrieval_logs as source=auto_inject so the threshold can be tuned from data. Per-user config (enable / threshold / top-k) is DB-backed via /api/settings with a Settings UI card; defaults enabled, threshold 0.55, top-k 3 (conservative — tune once auto_inject telemetry accrues). Scribe: project 2, milestone 93, task 1033. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E	2026-06-22 20:31:07 -04:00
bvandeusenandClaude Opus 4.8	807f478cac	feat(search): retrieval telemetry — log every semantic retrieval ... Add retrieval_logs (migration 0068) + services/retrieval_telemetry with a fire-and-forget record_retrieval(), wired into the MCP search tool (source=mcp_search) and the REST search route (source=rest_search). Captures query, effective params, and the per-result score distribution so KB-injection thresholds can be tuned from data rather than guessed. Scribe: project 2, milestone 93, task 1032. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E	2026-06-22 20:10:15 -04:00
bvandeusenandClaude Opus 4.8	513019786e	feat(search): pgvector substrate — vector(384) + HNSW for semantic search ... Move semantic_search_notes off the full-table Python cosine scan onto a native pgvector column: indexed ORDER BY embedding <=> :q LIMIT k (HNSW, cosine). Migration 0067 enables the extension, converts the JSONB embedding column to vector(384) (stale-dim rows dropped and regenerated by the startup backfill), and builds the HNSW cosine index. Postgres image moves postgres:16-alpine -> pgvector/pgvector:pg17 across prod, quickstart, and CI. Scribe: project 2, milestone 93, task 1031. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E	2026-06-22 20:10:15 -04:00
bvandeusenandClaude Opus 4.8	4f31890bde	test(ci): dispose engine between integration tests (per-loop pool) ... First integration run proved the lane works (run_maintenance test passed against real Postgres), but the health test failed with 'Future attached to a different loop': pytest-asyncio uses a fresh loop per test while the app's module-level engine pools a connection from the prior test's loop. Dispose the engine in each test's teardown so the next test starts with an empty pool on its own loop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 19:24:09 -04:00
bvandeusenandClaude Opus 4.8	2ad2e943f3	test(ci): add Postgres integration lane + real run_maintenance guard ... The unit suite can't catch sync/async API mismatches against SQLAlchemy (an un-awaited execution_options passed green CI but failed at runtime: VACUUM 0/6). Add a real-Postgres integration lane modelled on the family pattern (rules 6/79-82): a new CI 'integration' job with a postgres:16 service, bridge-IP discovery, busybox-safe readiness wait, and 'alembic upgrade head', running pytest -m integration. Non-gating, like the unit lane. - tests/test_integration_db_maintenance.py: runs run_maintenance() and get_table_health() against real Postgres; asserts all allowlisted tables vacuum OK (the await regression makes this fail) and health reports real stats. - pyproject: register the 'integration' marker. - conftest: integration-marked tests use the real DATABASE_URL, not the stub. - ci.yml: unit 'test' job now runs -m 'not integration'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 19:19:18 -04:00
bvandeusenandClaude Opus 4.8	e6c89f6b88	fix(db): await AsyncConnection.execution_options in run_maintenance ... execution_options() is a coroutine on AsyncConnection and must be awaited; the un-awaited call returned a coroutine, so exec_driver_sql() blew up with AttributeError and every table's VACUUM was skipped (Run-now reported 0/6). A prior change had wrongly dropped the await. Fix it and make the test mock execution_options async so this call shape is actually exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 18:31:30 -04:00
bvandeusenandClaude Opus 4.8	96079d5b77	feat(db): table-health readout — per-table bloat metrics in admin card ... You can't decide what to maintain without seeing what's bloating. Adds a read-only health panel driven by Postgres' own statistics views. - services/db_maintenance.py: get_table_health() queries pg_stat_user_tables + pg_total_relation_size + pg_database_size — per-table size, live/dead tuples, dead-tuple ratio (the bloat signal), and last (auto)vacuum/(auto)analyze. - routes/admin.py: admin-only GET /api/admin/db-maintenance/health. - SettingsView.vue: 'Table health' table in the maintenance card, all tables sorted by dead tuples, rows >=20% dead-ratio flagged; total DB size shown; refreshes after a Run-now so the dead-tuple drop is visible. - Tests: health row/size shaping + null-timestamp passthrough; route + service surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 17:53:24 -04:00
bvandeusenandClaude Opus 4.8	c4553d937c	feat(db): scheduled DB maintenance — daily targeted VACUUM (ANALYZE) ... Adds a daily off-hours VACUUM (ANALYZE) over the high-churn tables the retention/purge sweeps churn (app_logs, notifications, token tables, notes, note_versions), on top of Postgres autovacuum, to reclaim bloat left by the nightly bulk DELETEs and keep planner stats fresh. - services/db_maintenance.py: run_maintenance() over a closed table allowlist via an AUTOCOMMIT connection (VACUUM can't run in a txn); per-table summary persisted as the db_maintenance_last_run admin setting. - services/db_maintenance_scheduler.py: BackgroundScheduler cron (default 04:00 UTC, after the 03:30 trash purge); enabled-gate checked at fire time; live reschedule on hour change. Wired into app.py start/stop. - routes/admin.py: admin-only GET/PUT /api/admin/db-maintenance + POST /run. - settings.py: set_admin_setting() (write-side of get_admin_setting) for out-of-request writes. - SettingsView.vue: admin 'Database maintenance' card — enable toggle, run-hour (UTC), Run-now, last-run summary. - Tests: allowlist is closed, VACUUM issued per table, one failure doesn't abort the rest, summary persisted; route/scheduler/service surface. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-14 16:42:04 -04:00
bvandeusenandClaude Opus 4.8	dd1fc2d506	feat(mcp): extend dedup gate to create_rule / create_project_rule ... Completes the Phase 5 follow-up: rules now get the same update-over-create gate. Title-based only (rules aren't a semantic-retrieval/RAG surface), scoped to the same topic (rulebook rule) or same project (project rule). force=true overrides; fail-open like the note/task gate. Deferred-item decisions (operator): REST/web gating SKIPPED (kept MCP-only — humans rarely double-create and a hard block needs UI affordance); orphan scope kept orphan↔orphan (no change). So this rule gate is the only remaining build. - services/dedup.py: find_duplicate_rule(title, topic_id|project_id). - create_rule + create_project_rule: force param + gate. - tests: rule title match, scope-required guard, tool gate (block + force). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 13:43:17 -04:00
bvandeusenandClaude Opus 4.8	322cbc3b5e	feat(mcp): Phase 5 — write-time near-duplicate gate (update-over-create) ... #755 Phase 5. create_note / create_task now BLOCK a near-duplicate instead of silently inserting: they return {"duplicate": true, "existing_id", message} pointing at the record to UPDATE. Fights store bloat and stale competing copies that semantic search (RAG) would otherwise resurface for reconciliation. A force=true override creates anyway for genuinely-distinct records. - services/dedup.py: find_duplicate_note — two signals, scoped to owner + same project + same kind: (1) normalized-title exact match (cheap, always); (2) semantic cosine ≥ 0.90 but ONLY when body ≥ 200 chars (short/title-only embeddings false-positive — the pre-pivot lesson). Project-less (orphan) records compare only to other orphans on BOTH signals (orphan_only on the semantic call) — they're not matched across every project. - Gate wired into the MCP create_note/create_task tools (the LLM write path) with force override; _INSTRUCTIONS documents the duplicate response + force. - Opt-in by design: the service helper is only called from the interactive create tools. Internal/programmatic creates (recurrence spawn, imports) go straight through services.create_note and are NOT gated — a recurring task spawning its next same-titled instance must not be blocked. - Scope v1: MCP tools only. REST/web (human CRUD, needs a UI affordance) and create_rule (not a RAG surface; _INSTRUCTIONS already steer it) are follow-ups. - tests: dedup service (title/semantic/body-gate/type-filter) + tool gate (blocks, force bypasses) for notes and tasks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-14 13:21:35 -04:00