192 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	44e0b0541f	feat(rules): rules retrieve against the operator's message (#3852) ... The third rule arm, and the one the other two cannot reach. `write_path_rule` is keyed on code, `pre_tool_rule` on a command — both things the session is about to DO. A rule that governs what to SAY has no such trigger: extract intent from loose phrasing, raise a conflict before acting, hand off an action with its reason, end a finding with an offer all bind on a RESPONSE, and no tool call precedes one. The operator's message is the only query that exists before a response is composed. That hook searched notes alone, so no rule had ever been retrieved against a thing the operator actually said — and residency was the only surface those rules had, which is what milestone 394 removes. A SEPARATE FUNCTION, not a branch in build_autoinject_hint, because of its early returns. That arm bails when auto-inject is disabled, when the query is blank, when nothing clears the note bar — every one a statement about NOTES. Folded in, an operator who turned the awareness menu off would silently lose their rules, a coupling with no symptom since both look like a quiet hook. Two functions, two sets of gates, composed in the route. Guarded as "the rule arm never asks the notes arm's config", which is the structural fact. Joins _ARMS rather than getting its own test file. #3497's history is that the pre-tool arm inherited a defect from its sibling by being MODELLED on it instead of sharing with it, and a third arm modelled on two is two chances to repeat that. Repeat rendering, fresh-only counting, log-before-bailout, the kind register and the two-recorders identity are properties of every arm or of none. The bar is INHERITED and says so. 0.72 was tuned against code and commands; prose is a different query shape against the same documents, and triggers are written in the vocabulary of the moment — which for most rules is act vocabulary. Starting at the only number with evidence behind it and logging every call from the first deploy is what makes it settleable; guessing lower would put an unmeasured bar in front of a corpus that binds. k=3, anchored on this hook's own budget rather than the act arms'. RULEHINT_LIMIT is 1 because that arm fires before every Bash call; this one fires once per turn, beside a notes menu already spending three slots. And a prompt genuinely contains more than one act — "merge to main and then start on X" is two — where a command is one thing. `prompt_rule` added to RANKED_SOURCES: a ranker picked it, and a ranked source missing from that tuple is silently counted as bulk delivery and drops out of the pull-through denominator. The hook reads and writes the SHARED rule ledger under scribe-priorart, not a private one — one session keeps one list, aged (#3751), so a rule named here is not re-announced before the next Bash call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 07:56:19 -04:00
bvandeusenandClaude Opus 5	26e0dff706	feat(rules): a preference does not speak in a rule's voice (#3849 step 3) ... Two independent axes on one hint line. `kind` decides the head, `seen` decides the tail, and neither reads the other — which is what let a second kind arrive without reopening #3750's repeat question. Whether a record is already on the exclusion ledger has nothing to do with how much force it carries, so the seen branch is shared verbatim. The noun carries the whole visual difference, deliberately. A reader skimming an injected block gets one word to place the register, so the word that moves is the one naming force: "Standing rule" / "Preference". Everything structural after it is identical, so the kinds read as one set rather than two formats. Force is asserted in exactly one other place, and that moves too. A rule's line says to read it BEFORE DECIDING IT DOES NOT APPLY, because dismissing a rule unread is how the thing it prevents happens. A preference makes no such claim: it says where to find HOW THIS HAS BEEN DONE BEFORE, and following it buys consistency rather than correctness. Guarded on both places at once. Pinning the noun alone would pass a line reading "Preference … before deciding it does not apply" — label swapped, instruction kept — which is worse than not distinguishing them, because it looks handled. And a guard on the independence claim itself, exercising all four combinations: the way this breaks silently is a seen branch that grows a kind test, leaving one combination rendered by nobody's intention. Noted, not fixed: plugin/skills/using-scribe/SKILL.md still says "Standing rules are binding" with no room for a kind that does not. That surface is step 6's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-10 22:04:14 -04:00
bvandeusenandClaude Opus 5	c63172272d	feat(rules): a preference is a rule that does not bind (#3849 step 1) ... Adds `kind` to rules — `rule` binds, `preference` is how the operator wants work done. One column, because the two differ in exactly one dimension and everything else a preference needs already lives on `rules`: a trigger column, a trigger-dominated embedding document, ownership-scoped search, three retrieval arms with telemetry, typed relations, and versioning. Defaults to `rule`, so nothing changes force on upgrade — 0088's argument for `tier`, unchanged. `rule_versions` gets the column too, and that half is not bookkeeping. `record_if_changed` decides whether an edit deserves a snapshot by comparing the fields a version carries, so a field absent from SNAPSHOT_FIELDS is a field whose change records no history at all. Without it, turning a rule into a preference — the moment something stops binding, and the single most consequential edit either kind can undergo — would leave the history silent. Backup carries it through all four seams. A missed one would have restored every preference as a rule, quietly. Guarded on real Postgres in three halves: a preference writes, a typo is refused (without which every other assertion would pass against a table whose CHECK had been dropped), and a row written with no kind reads back as `rule` — the migration's whole safety claim, asserted rather than assumed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-10 21:14:29 -04:00
bvandeusenandClaude Opus 5	c4908f093f	feat(rules): a repeat is referenced, not withheld (#3750, #3752) ... Both arms used to drop a hit already on the session's exclusion ledger and emit nothing. That is correct only while the session still HOLDS what it was told, and a compaction breaks exactly that: the earlier injection is summarized away while the id stays on the ledger, leaving the rule absent from context AND unreachable for the rest of the session. #3749 closed the compaction half by clearing the ledger; this closes the ordinary half, where a session simply stops holding a line it read an hour ago. Only ONE CLAUSE of the existing line is false on a repeat — the claim that the rule is not in the session's loaded set — so only that clause changes. The fresh line is byte-identical to what it was. Both tails now come from one `_rule_hint_line`. The arms phrase their heads differently on purpose; everything after must not differ, and #3497's history is that the pre-tool arm inherited a defect by being modelled on its sibling rather than sharing with it. THE BUDGET DECISION, recorded at RULEHINT_LIMIT. A repeat competes for the single slot on rank alone: nothing is fetched behind it, and it never rides alongside as a second line. Promoting a fresh rule past a better-ranked repeat would reinstate the withholding one rank deeper, and a second line is the one thing the limit exists to forbid. The consequence is deliberate — a rule that keeps ranking first for a recurring situation keeps being referenced, and its decay belongs to exclusion ageing (#3751), not to a first-place rule being demoted for having won before. THE TELEMETRY, decided before shipping rather than after a number moved (#3752): nothing changes. A reference is a RENDERING decision, not a retrieval outcome. `results` stays `fresh`, `suppressed` stays len(hits) - len(fresh), and `record_rule_surfaced` still counts only what the arm freshly chose. This matters more than it reads: the naive implementation drops the `fresh` filter and takes suppressed_count to zero everywhere — and #3739's near-miss fix identifies repeat-caused zeros by `suppressed_count > 0`, so the contamination corrected on 2026-09-08 would return by a different route, in the same field, with the fix still in the code and no longer working. A test asserts the counters as unmoved, because "nothing changed" is only worth something if it is checkable. Also corrects two comments that outlived #3702 — both arms still claimed CONDITIONAL ONLY while the module-level note above RULEHINT_LIMIT said the opposite, in the exact code this change edits. GUARDS - an already-held hit produces a rule line at all (the regression), asserted on `get_rule(<id>)` rather than a truthy context: the write-path arm fills its context from four other sources, so truthiness passes under the OLD behaviour and pins nothing there. - the two tails are distinguishable, each excludes the other, and neither injects the rule statement — the budget claim, both branches. - the counters are unmoved, per #3752. - test_a_rule_the_session_already_holds_is_not_re_offered asserted the old contract (`"161" not in context`). Repinned rather than deleted: the telemetry half of what it protected still holds. Repinned the #3497 log-placement guard on structure (rule 167). It read `body.index("if not fresh:")` — a local variable NAME, not the property. This change renames that guard to `if not hits:`, so the old assertion would have raised ValueError and reported #3497 as back while the arm was entirely correct. Now walks the AST for the first early return after the search and asserts the call row is written before it. Falsified both ways: it fails on the #3497 mutation, and it refuses to pass when no early return exists at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 21:48:50 -04:00
bvandeusenandClaude Opus 5	d5ac8408f6	feat(telemetry): record WHAT the bar turned away, not only how close it came (#3807) ... #3670 added `best_available_score` so a threshold could be judged from its rejections. It records how CLOSE the bar came to firing and not WHAT it refused, and that is the half a decision actually needs. Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 — about 117 declines a day within 0.013 of firing. Dropping to 0.707 would take that arm from 22 hits a day to roughly 139: six-fold, on a surface that runs before every Bash call. The percentile says the mass is there. Nothing said whether it was worth showing. NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected rule line already carries title and trigger, so a session can comply without ever calling get_rule, and rule pull-through understates usefulness by construction. Reading the rejected records can — and `result_ids` holds only what was RETURNED, so on a zero-result call the near-missed record had no name at all. So the id, from the SAME ranked candidate as the score. Both searches unpack `best` once and read both fields off it, because splitting that into two expressions is exactly how a later edit pairs a score with its neighbour's id — and a score attached to the wrong record is worse than no id, since it invites judging the wrong one and concluding the bar is fine. write_path withholds the id on the same condition it withholds the score (#3739): a surviving id beside a null score names a record without saying what it scored, the pair disagreeing in the other direction. THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled, and a reader tuning a bar needs to go and read the records. Opt-in via `near_miss_samples` (0-20, default 0) so the ordinary readout keeps its size, and deliberately NOT a window function: this module's one production outage was a grouped query Postgres rejected, swallowed by the broad except, every counter reading zero while the mocked tests passed (#2663). One flat ordered query, overfetched, bucketed in Python — the shape that lesson prescribes. Migration 0097, nullable and unbackfilled. Not a foreign key: the table spans record types and `source` says which, exactly as result_ids works. The integration guard pins the listing as PER SOURCE. A global LIMIT would let a noisy source eat the whole quota and leave the surface being tuned showing nothing — which reads as "nothing was close", the misreading this milestone has spent itself correcting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 21:24:32 -04:00
bvandeusenandClaude Opus 5	623464323e	fix(telemetry): a search that never ran is not a decline (#3765) ... `best_available_score` was added by #3670 so a bar could be judged from what it rejected, and it arrived null on four unrelated causes: the corpus offered nothing, the query was empty, the embedder was down, or the DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the #2663 shape — a swallowed failure rendering as a clean zero — inside the field added to fix an instance of the #2663 shape. Found while trying to explain why reuse_slot returned nothing on 45 of 45 calls, and auto_inject on 153 of 161. That investigation is still open; what it established first is that the readout could not answer it. THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so every remaining null means one thing: searched, and nothing came close. That is the convention the pre-tool arm already follows for a blank command — "a row here would report a call that never happened and drag the clear-rate down with phantom declines" — extended from the case a caller can see in advance to the ones only the search knows about. Both searches stamp `report["searched"]` FALSE before anything can return and True only where a real result set exists, so every early return leaves it false. It has to be the first thing done to the dict: a return added above that line would leave the key absent. ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no report cannot know, and the safe reading there is the old behaviour. Only a real search can report False, so absent means "nobody asked" and never "it failed" — which is also why 66 existing mocked searches across twelve test files keep working unchanged rather than being rewritten to simulate a flag they do not care about. A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a WARNING on a query failure, which is where a broken search belongs: a counter cannot say "I am broken" without a reader already trusting it. Three tests, and the middle one is what makes them discriminate — a blanket `return` passes the first and fails the second, because a call that searched and came back empty is the only evidence a threshold is too high (#3497). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 12:47:46 -04:00
bvandeusenandClaude Opus 5	277f5df515	fix(telemetry): write_path reported records it withheld itself as near misses (#3739) ... Found by verifying the previous fix on live data — the check that fix was meant to make possible. write_path near_misses.max 0.822 p90 0.7521 top_score.min 0.6857 so the bar is at or below this A "rejection" that outscored every acceptance, and not one outlier: the p90 is above the bar too. #3739's fix keyed on `suppressed_count`, and I justified its NULL branch as "null means the caller passed its exclusions INTO the search, so the score is already post-exclusion". That holds for auto_inject and reuse_slot, both of which log the RAW search output and do their Python filtering after. It does not hold for write_path, the one note arm that filters TWICE: `exclude_ids` takes `seen - pulled_seen` into the search, but the pulled-and-seen ids stay in the query on purpose — the arm's query doubles as the resemblance test — and are dropped afterwards in Python. So the row carries a POST-filter count beside a PRE-filter score. The suppression column cannot rescue it the way it does for the rule arms. This arm's count would be PARTIAL — covering the drops made here and not the ones `exclude_ids` made inside the search — and a partial number under a name that reads as complete is the substitution this milestone exists to stop. So it reports null whenever its own filter removed anything: not measured on this call, because the bar was not the only thing that turned something away. Calls that withheld nothing keep reporting, which is most of them. Both directions are asserted. Without the second test, setting the field to null unconditionally would pass the first while deleting the measurement #3670 was built for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 00:34:11 -04:00
bvandeusenandClaude Opus 5	a165483b92	fix(telemetry): a repeat is not a rejection, and near_misses counted it as one (#3739) ... Caught on the first live read after deploying #3670. The readout contradicted itself: pre_tool_rule top_score.min 0.7204 the lowest score ever RETURNED near_misses.max 0.7457 "rejected", but scored higher `best_available_score` is measured pre-threshold, which is right, but for the rule arms it is also PRE-EXCLUSION, which is not. The note arms pass `exclude_ids` into semantic_search_notes so their score is already post-exclusion and clean; `semantic_search_rules` takes no such parameter, so the rule arms filter in Python after the search and a rule that cleared the bar and was dropped as a repeat still reported its score on a zero-result row. That is #3497's distinction — a ranker decline versus a reader already ahead of it — reintroduced one level up, inside the field built to replace a tautology. The population now also requires `suppressed_count IS NULL OR = 0`. The NULL arm is principled rather than permissive: null means the caller filtered INSIDE the search, which is exactly the case where the reported score cannot be contaminated. Deliberately conservative — a call carrying both a repeat and a lower genuine miss is dropped whole, losing that point. It undercounts; it cannot corrupt, which is the right way round for a number read against a bar. It also makes `near_misses.max < threshold` true BY CONSTRUCTION rather than by fixture: an above-bar candidate nobody excluded would have been returned, so its call is not in the population at all. THE TEST DID NOT CATCH THIS, and that is the part worth keeping. The assertion `nm["max"] < 0.72` was already there, with exactly the right intent. It passed because the fixture contained no suppressed call — the guard held because the breaking shape was absent, not because the code was right. Rule 167's stated failure mode, in a test written while citing rule 167. The fixture now builds that shape: a 0.9 hit dropped as a repeat, which lands in the population and drags `max` above the threshold unless the predicate excludes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 16:42:51 -04:00
bvandeusenandClaude Opus 5	e7c1af32a0	fix(telemetry): the bar can only be judged from what it rejected (#3670) ... `cleared_threshold` was documented as the number to read first. It was a tautology. The search applies the threshold before returning, so every returned result cleared it by construction and a call with no results has no top_score to compare — the condition was true exactly when `result_count > 0`. It was `calls - zero_result_calls` under a name that promised a second opinion, and `zero + cleared == calls` held on all nineteen source/window readings ever taken, today's live seven included. The reading procedure built on it asked the reader to compare a number with itself, and a threshold change was unobservable through it: raise the bar and both numbers move together, so the field could never show a bar set too high. REPLACED, NOT JUST REMOVED. The question the table exists to answer is whether the bar is in the right place, and that is only answerable from the calls that returned NOTHING: how close did the best rejected candidate come? A 0.72 bar turning away a stream of 0.71s is set too high by a hair; the same bar turning away 0.30s is working. Both render as a zero-result call today and nothing separates them, because the losing score is discarded inside the search. So both searches now rank WITHOUT the bar and apply it in Python. The qualifying set is provably identical — rows arrive ordered by distance, so every above-bar row sorts ahead of every below-bar one, and an over-fetch that returned N above-bar rows returns the same N plus some losers. What changes is that the losers are visible instead of dropped in the query. `report` carries the score out without changing what a search RETURNS: eight of eleven call sites want hits and nothing else. New column (migration 0096), nullable and unbackfilled. A row written before this genuinely does not know, and a 0.0 would read as "the corpus held nothing remotely relevant" — a claim invented out of a caller's silence, which is the substitution this whole milestone corrects. The new aggregate is a percentile_cont WITHIN GROUP over a CASE, one step from the shape that produced #2663, where a rejected query was swallowed by the broad except and every counter read zero. It carries an integration guard for that reason: only real Postgres can say it parses, and the symptom of failure is silence. Also adds a guard that no int field in a bucket equals `calls - zero_result_calls`. That identity is what `cleared_threshold` satisfied for its whole life, and it survived because it had its own name and nobody added the two numbers beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 13:50:05 -04:00
bvandeusenandClaude Opus 5	277aea58e4	test(telemetry): pin the identity that falsified this milestone (#3668) ... Reverts the deliberate break from 5e19a1b. `plugin_context.py` is now byte-identical to before it; the only change against that baseline is the guard itself. FALSIFIED, not argued (rule 167). CI run 6046 with `results=hits` in both arms failed exactly the predicted four cases — [one-already-held-write_path] FAILED [one-already-held-pre_tool] FAILED [all-already-held-write_path] FAILED [all-already-held-pre_tool] FAILED [nothing-held-*] passed — and the nothing-held cases passing is the point, not a gap: with no exclusions both recorders see the same list however wrongly they are wired, so that case can never discriminate and a guard built only from it would read as coverage while catching nothing. WHAT IS PINNED. Both arms build one `fresh` list and hand it to two recorders in one function, so the call log and the surfacing log cannot disagree about what a single call showed. Ids, not counts: equal counts drawn from different lists is a real way for this to break, and a count comparison would call it agreement. Three hits, where production returns at most one. The identity holds at any limit because both recorders read the same list, and stating it that way survives RULEHINT_LIMIT moving again — it has moved once already (2 → 1, 2385100), and that move is half of why the original reconstruction misread its own numbers. NOT DONE, deliberately: the readout-level self-check the task also proposed. `cleared_threshold` counts CALLS that beat the bar while `surfaced` counts RULES, so that identity holds only while the limit is 1 — it would fire on a healthy system the moment the limit rises. The arm-level form has no such coupling, which is why the guard lives here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 13:36:03 -04:00
bvandeusenandClaude Opus 5	5e19a1b028	test(telemetry): FALSIFICATION — prove the identity guard can fail (#3668) ... Deliberately broken, reverted in the next commit. Rule 167 requires every guard be falsified against the regression it names before it is trusted, and rule 10 puts CI as the only place that can run it — so the failure has to be made to happen here rather than argued for. The guard: both rule arms feed one `fresh` list to two recorders, so the call log and the surfacing log cannot disagree about what one call showed. The regression: `results=fresh` becomes `results=hits` in both arms, so the call log counts what the ranker found while the surfacing log counts what was shown. That is not a hypothetical shape. It is exactly the divergence that would make a correct system report a lost write when the two tables are later compared in aggregate — the reading that scoped this milestone at five steps against a defect that did not exist. Expected red: the one-already-held and all-already-held cases on both arms. The nothing-held case must still PASS — with no exclusions both recorders see the same list however wrongly they are wired, which is why it could never have been the discriminating case. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 13:31:49 -04:00
bvandeusenandClaude Opus 5	7a2aff7bc1	fix(telemetry): a surface that stopped recording is not one that never ran (#3720) ... `out["sources"]` was built only from the windowed aggregate, so a source with rows in `retrieval_logs` but none inside the window got no bucket at all. Absent is exactly how a source that never existed renders, so a surface that WAS recording and went silent became unreadable — #2663 one level up, the failure that looks like the correct answer. Two queries at different scopes, and only one shaped the output. `_complete_from` reads all-time and knows every source the table has ever held; the windowed loop dropped whatever it did not return. Every such source now gets a zero bucket. Zero is a real measurement here rather than a manufactured one: the all-time query proves the source was recording, and it made no calls across a window it fully covers. No `covers_window` special case is needed either — a source whose first row fell after `since` would have that row IN the window and already hold a bucket, so anything reaching this branch began before it. The counts are 0 and everything else is null. A sampled distribution is not the same claim as a call count, and rendering p50 as 0.0 for a source nobody sampled would assert a measurement — #3311's mistake, in the readout built to prevent it. Found while fixing #3712's fixture, which failed with KeyError for this exact reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 11:07:21 -04:00
bvandeusenandClaude Opus 5	950c93c5d4	fix(telemetry): the coverage helpers were defined inside the function they serve (#3712) ... `_complete_from` and `_coverage` landed between `retrieval_summary`'s docstring and its body. Python does not object to that the way it looks like it should: blank lines do not close a block, so the whole remaining body — indented four spaces, sitting after `_coverage`'s `return` — became unreachable code INSIDE `_coverage`, and `retrieval_summary` became a function that is nothing but a docstring. The error surfaced three ways at once, none of which named the cause: fourteen F821s for `days` and `user_id` (real: those are `retrieval_summary`'s parameters, and the body no longer lived there), a SyntaxError on `async with` (real: `_coverage` is sync), and every test module that imports this file failing to collect. Moved both helpers above `retrieval_summary`, beside `_bucket` and `_round`, where the file's other helpers already are. No behaviour change — this is the code that was meant to be there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 10:34:57 -04:00
bvandeusenandClaude Opus 5	21a5831479	feat(telemetry): every counter says when it started being recorded (#3712) ... A window that opens before a counter existed reports that counter as though it had been measured throughout. The reader cannot tell "zero because nothing happened" from "zero because nobody was counting yet", and — worse — cannot tell a partial count from a complete one. That middle case yields a plausible FRACTION rather than an obvious zero, which is what makes it dangerous. It is not hypothetical. A 7-day window opened while the ranked rule surfacing recorders were four days old produced an apparent 64% write loss, which survived a code review, four ruled-out alternative causes and a five-step milestone before an identity check falsified it in one read. Every counter block now carries `complete_from` and `covers_window`. THE GRAIN IS THE SOURCE. retrieval_logs accumulates for months, so a per-table earliest row says months for every source it holds — including an arm added days ago whose counter means something else entirely. The old source would vouch for the young one, which is the exact reading this prevents. A SECTION TAKES ITS LATEST CONTRIBUTOR, NOT ITS EARLIEST. A figure summing several sources is complete only once every one of them was being written, so "*" is a max. Using min would reproduce the original error in miniature. `covers_window` is null, never false, when nothing was ever recorded: "no measurement" is not "partial measurement" — the null convention #3497 established for `suppression`, one level up. Also corrects a stale claim in the tool docstring: it still taught readers that write_path_rule "has never once declined to fire" (#3311). That was the arm writing its retrieval_logs row only on calls that found something; #3497 fixed it, and the arm declines the large majority of its calls. _complete_from takes the caller's session rather than opening its own, departing from the services canon (#2860) because it runs inside an existing block; to be recorded against the ledger once it ingests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 10:29:02 -04:00
bvandeusenandClaude Opus 5	dd1e6e2645	feat(rules): the rule arms stop filtering the corpus to one tier (#3702) ... Both arms passed tier="conditional", on the reasoning that an always-on rule is already in the session so re-surfacing it is pure noise. That conflates two different things: PRESENT IN CONTEXT the rule was delivered at session start SALIENT AT THE MOMENT the rule is in front of the reader when the action it governs is about to be taken A rule handed over in a list at turn zero is present while a session writes a config value three hundred turns later. It is not surfaced. So the filter did not skip a redundant hint — it made a whole class of rules permanently ineligible for the only mechanism that puts a rule in front of an agent AT the moment, and the more important a rule is, the likelier it sat in that class. Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs in a hint is a relevance question and a similarity bar is the control for relevance. A categorical exclusion standing in for a relevance judgment cannot be tuned, cannot be measured, and cannot be wrong in a way anybody notices. MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on every write and says obvious things teaches the reader to skip the block. It had simply never been checked, and retrieval_logs already records the scores to check it with. Rules clearing often and high means the fear was justified and the BAR is the work; rules clearing rarely in a thin band means relevance was always sufficient. Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the resulting distribution has one cause — and k=1 bounds the blast radius: a wider pool can change which rule surfaces and how often, never how long a single hint gets. The threshold rationale's first premise ("the eligible corpus is TINY — tier=conditional only") is updated rather than deleted: a larger pool makes clearing the bar mean MORE, so that argument weakened, and the bar was left alone anyway rather than move two variables at once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 00:24:02 -04:00
bvandeusenandClaude Opus 5	8be555d6dd	feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497) ... Making the rule arms log every call exposed a second ambiguity in the same row. `result_count == 0` is two unrelated events wearing one number: - the ranker found nothing above the bar — the only evidence a threshold is set too high; and - the ranker found only what this session had already been shown — which says nothing whatever about the bar. A long session excludes its way into the second, so the arm reads worse the longer it runs correctly. Rows written now carry the ambiguity permanently, which is why this lands before any watch period rather than after. `retrieval_logs.suppressed_count` (0095, nullable) holds what the caller dropped as already-shown. Both rule arms report it; they filter in Python and always know. The note arms pass exclusions INTO semantic_search_notes and never see what was dropped, so they store NULL. THE NULL IS LOAD-BEARING. It means "not measured here", and the readout renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0 would let an unmeasured surface read as a perfectly clean one — the same substitution of an artifact for a measurement that #3311 made. No backfill, for the same reason: existing rows genuinely do not know. `retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`, `calls_with_suppression` and `zero_because_already_shown`; subtract the last from `zero_result_calls` for the true ranker declines. The MCP tool docstring says to read the two together and warns against reading the null as a zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-03 21:12:38 -04:00
bvandeusenandClaude Opus 5	154a5de13e	fix(telemetry): both rule arms logged only their hits, so the clear-rate could only read 100% (#3497) ... `write_path_rule` reported `zero_result_calls: 0` and `cleared_threshold: 133/133` — a perfect record no other surface comes near (`write_path` 421 zeroes of 613, `reuse_slot` 124/199, `auto_inject` 114/326). #3311 read that as a measurement and milestone 333 was scoped on it. It was an artifact. Both arms called `record_retrieval` inside a guard on having results — the write-path arm behind `if fresh:`, the pre-tool arm below `if not fresh: return out` — so a call that found nothing wrote no row. The statistic was a fact about the shape of the code, true at any threshold whatsoever. The call log moves out of the guard in both arms. The surfacing log stays in it: nothing was shown, so no surfacing occurred. `results=fresh` is kept deliberately — the note arms pass exclusions into `semantic_search_notes`, so what they log is already post-exclusion, and logging `hits` here would make this row mean something other than every other row in the same readout. The defect bites hardest on the pre-tool arm, which fires on every Bash call: with no rows at all, a ranker that declined is indistinguishable from a hook that never fired — the silent failure the arm exists to stop. Tests cover both arms behaviourally (found nothing; found only what the session already held; searched nothing at all, which must stay silent) plus a structural guard, because this was one level of indentation and it appeared independently in two places. #3311 and the `rule_usage` docstring corrected rather than quietly rewritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-03 06:57:13 -04:00
bvandeusenandClaude Opus 5	2ee24b9d2b	feat(rules): rules before tools — a PreToolUse arm keyed on the action (#3476) ... The only just-in-time rule surface was registered on `Write|Edit` and queried with `code or path`, so a rule could be retrieved at the moment of a code write and nowhere else. Every rule about which tool to reach for — don't curl the forge, don't stand up a stack, don't run the suite locally, don't branch — was unreachable exactly when it mattered, and residency in the always-on preload was the only surface it had. That is the pressure that grew the resident set to 31 against #3089's ceiling of ~23; it was never a judgment anybody made. A reflex generates no query, so an instruction to check the rules cannot catch one. A mechanical trigger can: the tool call IS the query, and a reflex has to become a tool call before it can do anything. `build_tool_rule_hint` is deliberately tool-agnostic — a name and a string — so widening the matcher later is a hooks.json edit with no server change. The hook starts on Bash, which is where the action reflexes live. The two pre-tool arms share ONE session ledger of already-named rules (`<state>/<sid>.rules.ids`). Two ledgers would mean a rule named by one arm gets re-offered by the other, and the hint that fires most often is exactly the one that must not repeat itself. A test asserts both scripts build the same path, and another checks the shell hook and the Python route agree on every query-arg name (rule 33) — a rename there fails silently, looking like a surface that never finds anything rather than a broken one. Deliberately silent on outage, unlike the prior-art hook: a write is occasional, a Bash call is not, and an outage line before every command is what gets a channel muted. `tier="conditional"` matches the write arm and is the transition point — an always-on rule is already resident, so re-tier one and it starts arriving here instead of in every session's preamble. `pre_tool_rule` joins RANKED_SOURCES: this arm chose what it showed, so a pull can settle whether the choice landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-02 23:31:22 -04:00
bvandeusenandClaude Opus 5	8b9b3a1d9b	feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473) ... The ranked rule arm became measurable in M333. The preload did not — and that is the surface whose value is actually in question. `list_always_on_rules`, the SessionStart block and every `rules_payload` caller handed rules over wholesale and emitted nothing, so the resident set's token cost was certain and its usefulness could not be tested even in principle. Bulk deliveries now record as AMBIENT, beside the ranked count and never inside pull-through. Folding them in would mean growing the always-on set depressed the arm's measured precision and trimming it flattered the arm, neither for any reason to do with the arm. `RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there is one ranked rule source and this change adds seven bulk ones, so naming the rare half makes a forgotten surface default to ambient — under-counting it — rather than padding the denominator with surfacings nobody chose. Two lookalike call sites are deliberately left silent, with a test to keep them that way: the write-path etag arm and `rules_etag_for` read the rules to build or compare a MARKER and show nobody anything. No migration — `event` and `source` are plain Text with no CHECK (rule 36 does not apply). Snippet #2858 updated to the new `rules_payload` contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-02 23:06:40 -04:00
bvandeusenandClaude Opus 5	238510080e	feat(retrieval): the standing-rule arm gets its own bar, and asks for one rule not two (#3318) ... Milestone 333 step 4 — the split #2223 made one surface down, now made for the third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT. THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two facts hold on any install, including one with six rules and no telemetry: - The eligible corpus is tiny — conditional rules only, a handful to a few dozen against thousands of notes. A top-k over forty candidates always returns something, so "the best match cleared the bar" stops meaning "a good match exists". A bar calibrated for best-of-thousands is cleared by best-of-forty as arithmetic, not relevance. - Rules are short imperative technical English, far more homogeneous than note prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it; a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely inherited, it sits below where this corpus's noise lives. 0.72 errs deliberately toward silence on an asymmetry that is also structural: this hint fires on EVERY write. A missed rule is recoverable — it is still in Scribe and the agent can search it. A hint that cries wolf is not: it teaches the reader to skip the whole block, and the true positives go with it. The arm's own comment already said "noise on a hint that fires on every write is how a hint gets ignored". Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting the relationship — which would silently reinstate #3311 — does not. RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2 means the second line is almost always the second-best noise wearing the same confident framing as the first; halving k halves that regardless of the bar. It stays a constant because it is a decision about how loud one hint may be, not a per-install tuning question — and a knob nobody turns only adds a way to misconfigure the surface. Reachable from Settings, no restart (rule 25), with copy that says which way to move it and points at retrieval_telemetry's rule pull-through — which step 3 made readable — to tell "arriving unread" from "never arrived". Every config stand-in in the suite gained the key, not just the one that noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so a missing key raises inside its fail-open except and turns the arm into a silent no-op — indistinguishable from it running and finding nothing. That is the same vacuous-pass shape that bit step 2, one layer down (rule 33). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 18:05:00 -04:00
bvandeusenandClaude Opus 5	8901c904a9	feat(telemetry): retrieval_telemetry reports rule pull-through where it reported nothing (#3317) ... Milestone 333 step 3, the read half. Steps 1 and 2 built the table and filled it; until now nothing read it, and `usage` — sourced entirely from note_usage_events — described notes only while `sources` happily listed a write_path_rule row above it. A reader takes the aggregate as covering everything named above it. It did not. A SEPARATE `rule_usage` BLOCK, not folded into `usage`. Two reasons, and the second is the one that bites: the corpora differ by orders of magnitude, so a blended ratio would be the note ratio with noise on it and the rule arm would stay invisible inside it; and `usage` is what existing callers already read and compare across windows, so silently changing what it counts would move a number nobody was told had changed meaning. There is a test asserting rule events stay out of the note block. No `ambient` key, unlike the twin. Nothing surfaces a rule un-ranked — list_always_on_rules and enter_project hand rules over wholesale but emit no event — so there is no ambient class to subtract. The absence is a fact about the data, not an oversight, and it returns when a bulk loader starts emitting. Guarded separately, like `by_source`. This table did not exist a commit ago, and an instance running upgraded code against un-migrated schema would otherwise take down two readouts that work perfectly in order to report a third that cannot. On failure the FLAG is added and the SHAPE is kept — a caller must not have to choose between crashing on a missing key and quietly rendering zeros it has no right to. `pull_through` is None rather than 0.0 on an empty window, matching the note block. A ratio of zero asserts "rules were shown and none opened"; with an empty numerator and denominator that is a claim the data does not support, and it is the reading that would make a brand-new install look like a broken one. Also fixed, from #3311: the rule arm never timed its search, so it was the one source in the readout reporting a null p90_duration_ms — a gap that reads as "this surface is somehow not measurable" rather than "nobody passed the number". Both docstrings updated in the same change. The tool's is the agent-facing contract (rule 119) and it explicitly said rule surfacings were absent and had "no usage counter at all". Leaving that would have had a reader conclude the arm has zero pull-through rather than a separate one. Tests are integration for the reason the block above them is: real GROUP BYs and count(distinct) against a table a commit old, in a module whose one production outage was a SQL shape the database rejected inside a broad except. A mock would agree with whatever the code does, including nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 17:25:11 -04:00
bvandeusenandClaude Opus 5	8f7f447fda	feat(telemetry): the rule arm records what it showed, and get_rule records the read (#3316) ... Milestone 333 step 2. Step 1 built the table; a counter nobody calls reads zero and looks exactly like a surface nobody uses, which is #2663's shape. SURFACED — the standing-rule arm in build_write_path_hint, beside the record_retrieval it already made. Two tables, and the split is not arbitrary: retrieval_logs is one row per CALL keyed on the score distribution a threshold is tuned from; rule_usage_events is one row per RULE per event, the grain "was this hint ever acted on" needs and the grain a JSONB result_ids array cannot be indexed at. The comment there said rule ids had nowhere to go — that note_usage_events remaps ids on restore, so a rule id would return attached to whatever note took that number. Still true of the NOTE table, and precisely why step 1 built its own. Rewritten to say the gap is closed rather than leaving a stale rationale that would have someone re-derive the same dead end. Records `fresh`, i.e. AFTER exclude_rule_ids. A rule the session already holds was considered and not shown; counting it would inflate the denominator with claims the agent never saw, and the ratio would then fall for a reason that has nothing to do with whether hints land. PULLED — two doors, both after their access check so a refused read is not a pull. mcp_get_rule is the one that matters: the arm's own message ends "Read it with get_rule(N)", so that call is the exact action a landed hint produces. rest_rule carries the other prefix, and the prefix is load-bearing — "is this rule dead weight?" is served by any pull, "did that injected hint land?" by agent pulls only. NOT a pull: rule_history. It loads the rule for its title and its own output says "The current wording is on the rule itself — get_rule(N)", so counting it would credit a read of the history as a read of the rule and double-count anyone who then follows that pointer. list_always_on_rules and enter_project are likewise bulk resident loads, not somebody choosing to open one record. tests/test_rule_usage_wiring.py is cross-cutting on purpose: the surfaced end is in plugin_context, the pull end in two other modules, and "both ends meet" is a property no module-shaped file asserts. It covers the exclusion boundary, that a failing recorder cannot break the write, that a refused read records nothing, and two completeness guards — every door records, and the bulk loaders still do not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 17:15:03 -04:00
bvandeusenandClaude Opus 5	111eef7e30	fix(telemetry): the user-scoped rule_usage export read _rule_ids before it existed (#3315) ... Ruff F821, twice, on the same two lines. The query was placed next to its note-usage counterpart — which reads `note_ids`, defined much earlier — while `_rule_ids` is not built until forty lines further down, beside the rules themselves. Moved to sit directly after the `rule_versions` query, which is the other consumer of that variable and the block whose scoping argument this one restates. Worth noting what did NOT catch this. The integration round-trip passed on the same commit: it drives `restore_full_backup` against a hand-built payload, so it exercises the import side and the full export, and never calls `export_user_backup` at all. A per-user export of any account owning a rule would have raised NameError at runtime. The lint lane found it because a static check does not need the path to be reachable by a test. The comment moved with it and got sharper, since the hazard is that the plausible column is the wrong one: `user_id` on a usage row is whoever the arm fired FOR, not who owns the rule, so scoping a per-user export by it would carry this user's surfacings of someone else's rule and drop the ones fired for someone else on theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 16:56:54 -04:00
bvandeusenandClaude Opus 5	8826be7a91	feat(telemetry): rule_usage_events — the table, the service, and a restore that maps rule ids through the rule map (#3315) ... Milestone 333 step 1. The write-path standing-rule arm is the only retrieval surface in Scribe whose usefulness cannot be observed — and, not coincidentally, the only one that has never declined to fire. 296 calls, zero zero-result, 100% clearing its threshold, while every other surface declines most of the time (#3311, and re-measured in note #3430). `retrieval_logs` gives it scores; scores say what the ranker thought, never whether the hint landed. WHY A SIBLING TABLE AND NOT A COLUMN ON note_usage_events. The row carries no note-specific field and the readout is the same shape, which is the strongest case for sharing that note #3163 admits. What decides against it is identity at RESTORE: the note importer maps note_id through note_id_map, so a rule id parked in that column comes back attached to whatever note holds that number in the target database. Not dropped — reattached. The restore reports success, the counters are populated, and every one is about the wrong record, with no other field to disagree with. rule_versions made the same call for the same reason; this is the third rule-side sibling and it reads like the first two. FK-free on rule_id and user_id, matching note_usage_events / retrieval_logs / app_logs, and deliberately unlike rule_versions. A version belongs to a rule's history and dies with it; telemetry outlives what it describes. Deleting a rule must not erase the evidence that it was surfaced forty times and opened never, because that evidence is the case for having deleted it. The service uses `background.spawn` rather than a third copy of the strong-reference dance — that module's own docstring says new callers should, and a fourth copy is how one of them drifts. The AppLog canary #2663 demands is kept, and since `rule_usage` needed exactly `note_usage`'s semantics, that canary moved into `background.report_telemetry_failure` and note_usage now calls it. `retrieval_telemetry` deliberately keeps its own: its canary is a different shape (one process-wide flag, no AppLog row), so repointing it would change behaviour rather than consolidate it. No ambient bucket, and that is a decision. The note twin splits ranked from ambient surfacings because enter_project and the skill sync deliver records without choosing them (#2477). Rules have the same problem waiting — list_always_on_rules loads them wholesale — but nothing emits here yet, so an empty AMBIENT_SOURCES would be machinery pretending to a distinction the data does not contain. `source` stays granular, so the split stays a readout-level change needing no migration. Backup carries it (v14). The round-trip test seeds a NOTE alongside the rule so the target database has a note id to collide with — without that decoy, a restore running rule ids through the wrong map would merely drop them and the test would pass by absence, rather than failing on the populated-and-wrong result that is the actual hazard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 16:55:22 -04:00
bvandeusen	0d4b155699	feat(telemetry): pull-through per surface, not just per corpus (#3311) ... The readout already grouped usage by source — `group_by(event, source)` — and the loop directly below it threw the source away, collapsing every surface into one corpus-wide ratio. So the question a threshold is actually tuned against, "is THIS surface worth its noise", could not be asked of any surface, while the data to answer it sat in the table. `usage.by_source` reports notes_surfaced / notes_pulled / pull_through per surface. The grain is the note, not the call: a pull records the door it came through, not the surface that led there, so grouping the pulled rows by source would answer a different question. Joining surfaced rows to pulled rows on note_id answers this one without the session identity #2085 declined to invent — at the cost of being an upper bound per surface, which the docstring says where it is read. Ambient surfaces report counts and a null ratio: nothing chose those records, so "surfaced often, opened never" is not a judgment about them. A surface that genuinely produced nothing reports 0.0, which must not look like the null. The join is guarded separately from the two reads above it. #2663 was a novel SQL shape the database rejected inside a broad except; this is the novel shape here, and it must not take down two readouts that work. Tests are integration for that same reason — a mock passes on a query Postgres refuses. They pin the distinct-first property (three surfacings of one note are one note), the ambient null, and the LIKE escape, since an unescaped `mcp_%` also matches `mcpXget_note` and nothing else in the payload would show the difference.	2026-08-31 15:52:17 -04:00
bvandeusen	9d8104f7a5	fix(embeddings): key_share alone is FOR NO KEY UPDATE, not FOR KEY SHARE (#3262) ... SQLAlchemy spells Postgres's four row locks as a read/key_share pair, so `with_for_update(key_share=True)` renders FOR NO KEY UPDATE — an exclusive lock that two refreshes of the same record would fight over, and that an ordinary concurrent edit would block. The claim needs `read=True` as well to be the FOR KEY SHARE the docstring describes. Caught by the unit test that compiles the statement, which is the whole reason it asserts on the rendered lock mode rather than on behaviour that looks identical either way.	2026-08-31 08:13:29 -04:00
bvandeusen	7827b4ce63	fix(embeddings): the index refresh loses the race it used to deadlock (#3262) ... An embedding refresh replaces a record's vectors as delete-then-insert, which takes the chunk rows first and the parent row second (via the insert's foreign key). A cascading delete of the parent takes exactly those two locks in the other order. Postgres calls the cycle a deadlock and kills one side: sometimes the detached embedder, silently, and sometimes the user's delete, as a 500 on an operation that should have worked. Both upserts now claim the parent row with FOR KEY SHARE NOWAIT before touching any chunk row. That removes the cycle instead of narrowing it — either the embedder is first and the delete queues behind it, or the delete already holds the row and the embedder loses at once, which is the side designed to lose. FOR KEY SHARE is the lock the insert would take anyway, so an ordinary edit is unaffected. The note twin, recorded as unverified on the issue, has the same shape and the same fix; a trash purge is the hard delete that reaches it. Unit tests pin the ORDER and the lock mode by compiling the statement; the integration pair holds a real delete open in one transaction and proves the embedder returns having written nothing, with a deadline so a regression fails instead of hanging.	2026-08-31 08:09:09 -04:00
bvandeusenandClaude Opus 5	efabba58dd	fix(rules): the staleness signal must not wait for prior art to match (#3244) ... CI caught two things, and the first is the feature not working rather than a test being wrong. THE SIGNAL WAS GATED ON A COINCIDENCE. build_write_path_hint returns early when no prior art, stamp, divergence or derive matched, and that guard sat ABOVE the new arm — so a session whose rules had changed was told only if the file it happened to be editing also matched something else. A staleness signal that fires on that coincidence is not a staleness signal. The arm now runs above the guard, collecting into its own list that `lines` is seeded from, and the guard accounts for it. The standing-rule arm (milestone 307) is deliberately LEFT below that guard, and this is a finding rather than a fix: it has the same gating and probably should not, but it runs a SEMANTIC search, so lifting it would put an embedding query on every write in every session. That is a cost decision, not a bug fix, and not this task's to make. THE MARKER MUST NOT BREAK THE PAYLOAD IT DECORATES. rules_etag is computed on the SessionStart path, where `max()` raising costs the whole context payload — every rule title, the project, all of it — to save a hint. A row with no usable timestamp is now skipped and a set with none degrades to a count-only marker, which still catches a rule added or deleted and only loses edits. That is the right way round to lose information. CI found it because build_session_context's tests pass MagicMock rules and `max()` over those raises TypeError. Also: list_always_on_rules on an install with no always-on rulebooks returns `rules_etag: "empty|0"`. Its exact-dict test is updated rather than loosened — the key being present on an empty install is the behaviour, not noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-30 13:10:04 -04:00
bvandeusenandClaude Opus 5	5c9bb40777	feat(rules): a session is told when its rules move under it (#3244, milestone 323 step 5) ... The rules payload carries a marker; the write-path hook hands it back; the server says which rules moved. Nothing is said when nothing moved. THE COUNT IS NOT DECORATION. max(updated_at) alone cannot see a DELETED rule — it moves no timestamp — and that is the single change that takes an instruction OUT of force, which is the one a session most needs to hear about. The marker is `<max updated_at>|<count>`, and a deletion is reported through the count because there is no row left to name. THE HOOK IS THE CARRIER because it already fires before a write, which is the moment acting on a stale rule costs something. One comparison, no payload, no extra round trip. WHERE THE MARKER IS CAPTURED, and it could not be anywhere else: the SessionStart hook, from /api/plugin/context. The model also receives one from list_always_on_rules, but a hook cannot see an MCP tool's result — so the value the write path compares has to be stored where a shell script can reach it. Keyed by session id in the state dir the prior-art hook already uses, so "changed since" means since THIS session loaded its rules. NOT ON rules_payload, against the task's letter. Those are applicable_rules — a different, subscription-derived set. One key name over two sets is how a comparison starts reporting phantom changes, and the write path compares against the always-on set. WHAT IT CANNOT SEE is stated in both the service and the write-path arm as a table, because a reader who finds an etag will assume it covers staleness generally: another session edits a rule mid-flight | caught the session is misremembering a rule read hours ago | caught compaction summarised the rules out of context | NOT caught The third is the most common, and the marker is blind to it — the etag was in context too and went with the rules. The SessionStart nudge is that case's only mechanism and must not be softened because this shipped. A test asserts both modules still explain that. Instance-agnostic (rule 115): an install with no rules produces a stable marker rather than an error, and "no rules" reads as a state rather than as a change. An unreadable or absent marker reports nothing — a signal that cries wolf is worse than none, because it trains a reader to skip the line that will one day be true. The arm fails open like every other arm on this hook. The delivery is tested through the real build_write_path_hint rather than the helper alone: the feature IS a line arriving in a session, and the arithmetic being right proves nothing about that. Live acceptance is deploy-gated and not yet recorded on the task. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-30 13:02:00 -04:00
bvandeusenandClaude Opus 5	0704988528	feat(rules): the history is readable — service, REST and MCP (#3242, milestone 323 step 3) ... `list_rule_versions` / `get_rule_version` in the rulebooks service, a pair of REST routes beside the note-version ones, and an MCP `rule_history` tool. The ACL-scoped reads live in services/rulebooks.py rather than in services/rule_versions.py because rulebooks already imports rule_versions for the write path and the reverse would be a cycle. It is also the honest split: rule_versions owns what a version IS, rulebooks owns who may read one. Scoping is through the RULE, never the version's user_id, and both directions of that mistake are now pinned by tests. That column is the ACTOR — scoping by it would hand someone the snapshots they personally wrote on a rule that has since moved out of their reach, and would hide from the rule's owner every edit anyone else made. `get_rule_version` takes the rule id as well as the version id so the ownership check and the fetch agree about which rule is in play; the test for that uses a second rule the caller genuinely owns, because a nonexistent id would pass on the ownership check alone and prove nothing. An unreadable rule returns None, not an empty list. The two mean different things — "not your rule" versus "never reworded" — and the MCP tool keeps them apart: None raises, empty says so in band. THE DIFF QUESTION, ANSWERED — and the task's premise was half wrong. It says "notes have DiffView.vue and a diff endpoint already". The component exists and is reusable as-is: it takes `DiffLine[]` and nothing note-shaped, so step 4 can render a rule diff with it unchanged. The ENDPOINT does not exist — diffs are computed client-side by `computeDiff` in useAssist.ts. So no diff route is needed here, and none was written. For the MCP door the answer is different again: an agent has no client to compute a diff, but it also does not need one. Each entry holds the text the edit REPLACED, so "what did this say before the most recent change?" is the first entry, and the text that change produced is the rule as it stands. The docstring says so, and a test pins that sentence — read the other way round, every diff comes out backwards. No restore, per the task. Putting an old wording back goes through update_rule, which snapshots what it replaces, so the undo stays visible like any other edit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 23:46:15 -04:00
bvandeusenandClaude Opus 5	6fa66f202b	feat(rules): an edit leaves behind what it replaced (#3241, milestone 323 step 2) ... `update_rule` now snapshots the rule's text before it writes. Rescoping rule 79 meant hand-copying the superseded statement into a task log to keep it (#3237); the history is that, done by the write path instead of by somebody remembering. The snapshot is taken BEFORE the field loop, which is the one ordering that matters. `update_rule` drops `verified_at` when `verify_with` changes, and rewriting a check is exactly the edit whose history is worth most — a snapshot taken afterwards would file the NEW check against the OLD wording. Taking it up front also covers `clear`, which is a separate argument from the field loop and is how a rule that stops being a constraint loses its check entirely. Session-bound rather than opening its own like note_versions.create_version: the version and the edit that caused it commit together, so a failed update cannot leave a history entry for an edit that never happened. Two guards from the sibling are deliberately absent, and both are now pinned by tests rather than only by comments — "make it consistent with note_versions" is a plausible-sounding change that would silently start dropping history: - No MIN_VERSION_INTERVAL_SECONDS. That 300-second gate exists because note autosave fires every 60. Every version here comes from a deliberate update_rule, so three edits in one second are three edits. - No MAX_VERSIONS and no pruning. A rule is edited a handful of times in its life; a cap could only ever discard the one edit somebody went looking for. Kept from the sibling: the identical-content skip. Both doors resend every field, so without it a form saved twice would file an identical snapshot. `order_index` is excluded from the snapshot fields for the same reason — reordering a rulebook is not an edit to what any rule says. No delete-time snapshot, against the task's original scope and on the operator's call. A delete goes through trash_svc and is SOFT: the rule row keeps its full text and restores untouched, so there is nothing for a snapshot to preserve. Anything that survived a purge would be data the operator explicitly asked to be gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 23:01:16 -04:00
bvandeusenandClaude Opus 5	9006affda8	feat(rules): a rule keeps what it used to say — rule_versions (#3240, milestone 323 step 1) ... The sibling notes already had. `note_versions` snapshots a note's every meaningful edit; a RULE, which binds behaviour on every session that loads it, had nothing — an edit destroyed the previous wording with no record anywhere. Rescoping rule 79 meant hand-copying the superseded statement into a task log to keep it (#3237). The more consequential record had the weaker protection. Schema and transport only. Nothing writes a version yet — that is step 2. Three guards are deliberately NOT copied from note_versions, each defending against autosave, which rules do not have: no pruning or MAX_VERSIONS, no pin columns, no minimum interval. A rule is edited a handful of times in its life, and capping invites losing the one edit somebody needed. `user_id` is the ACTOR rather than the owner, and SET NULL rather than CASCADE: deleting a user must not erase the history of the rules they edited. The restore diverges from its NoteVersion sibling accordingly — an unmappable user leaves the row with a null actor instead of dropping it, which is the whole point of choosing SET NULL. The integration round trip pins that, because nothing in the code says which of the two shapes is intended and "make it match the sibling" would silently delete the record. Backup goes to v13. Both export paths carry the table; the per-user one scopes through the rule rather than the version's user_id, or it would carry the versions this user wrote on someone else's rule and drop the ones someone else wrote on theirs. The restore remaps rule_id through rule_id_map — #3182's arose_from_id trap on a new table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-29 18:38:16 -04:00
bvandeusen	8489206224	feat(notes): the sweep — which notes assert a fact nobody has confirmed (#3166, milestone 317 step 3) ... The read half. `notes_due_for_verification` + `mark_note_verified` + the MCP pair, ordered `verified_at ASC NULLS FIRST`: never-checked outranks checked-long-ago, because a note nobody has ever confirmed is a claim with no evidence behind it at all. Postgres sorts NULLs LAST on ASC by default, so getting this wrong would not error — it would silently invert the one signal the sweep exists to carry, which is why it has a test of its own. A SIBLING of rules_due_for_verification, not a shared implementation, and this milestone is a deliberate self-application of note 3163: the row could have been shared, the QUERY could not. That sweep scopes by rulebook ownership XOR project ownership because rules have no sharing ACL at all; a note scopes by the note ACL — browse, not read, so a record shared one-to-one never arrives in a passive surface unasked (decision 2094). What genuinely IS common moved to services/verification.py: how a stamp reads, how old it is, and the three states `last_verified` distinguishes — None ("a decision, the question does not apply"), "never" ("a fact nobody has confirmed"), a date. Rulebooks now imports it rather than defining it, so this is a consolidation and not a third copy. A failed check writes NOTHING, carried over from 312: there is no "verified false" state, because a note whose check failed is not in a special condition — it is WRONG, and recording the failure as a flag would let it sit there being false with the sweep quietly satisfied that somebody had looked. Two decisions worth naming: The sweep does NOT filter to non-task, non-snippet records even though the write path permits a check on nothing else. Such a row would be in an ILLEGAL state and this is the one surface that could say so; hiding it to match the invariant would make the sweep agree with a database it had stopped describing. A negative `older_than_days` raises instead of meaning "everything" — silently answering a different question is the failure shape this guards. `notes_due_for_verification` is classified read-only in server.py, spelled out because its name matches none of the prefixes the completeness test derives from. `rules_due_for_verification` is in the same position and is NOT listed, so it fails closed for read keys today — filed as #3191 rather than fixed here, since widening an auth boundary on a tool I did not write is the operator's call.	2026-08-28 16:39:50 -04:00
bvandeusen	700ef20eb0	feat(notes): a note's check is writable through both doors, and empty means empty (#3164, milestone 317 step 2) ... The rules path's three lessons (#3096), inherited: EMPTY MEANS NULL. The sweep's whole signal is `verify_with IS NULL` = "this is a decision, there is nothing to go and check". A "" that is not NULL makes a norm look like a constraint nobody has verified — and never-checked sorts FIRST, so it would sit at the top of the sweep forever. CLEARING IS EXPLICIT. At the MCP door "" means "leave this alone", so an agent updating a body does not wipe a check it was never asked about — which leaves no value meaning "remove it". `clear` names the field, and naming it cannot happen by accident. The REST door, where a cleared form input arrives as "", reaches the same place through normalisation: two idioms, one outcome. THE STAMP CERTIFIES A CHECK, NOT A RECORD. Rewrite or clear `verify_with` and `verified_at` is dropped, so the note re-enters the sweep. A note wrongly listed as due costs one look; a note wrongly vouched for costs exactly what the sweep exists to catch. `verified_at` is also no longer settable through an ordinary edit — a stamp says somebody performed THIS check, and minting one from a write that ran no check is the one thing that would make the sweep lie. And one this path adds: not every record may carry a check. A task's decay is its status — a done issue records what happened rather than asserting something that can go false — and a snippet already has verify_snippet, which compares its recorded location and code against the repo and expires its own verdict. Both are refused with a message naming the alternative, never dropped silently (minted_kind's reasoning, #3129), and the gate lives at the service so the two doors cannot come to disagree. Written as an INVARIANT over the resulting record, not a filter on which fields were passed. That is what catches the sideways route — a checked note being turned into a task, a write that names no check at all and would sail past any per-field gate. The MCP docstrings carry the norm-vs-constraint test, because at that door the docstring IS the contract and a field described only as "how to verify this" gets filled in on every note. Step 5 does this properly across the instruction surfaces; this is the minimum that stops the field being misused on arrival. tests/helpers gains `drive_update_note` — the patch stack for driving update_note, written twice before this and now once. The note-shaped fakes gain the trio explicitly, for fake_note's own stated reason: an unset attribute is a truthy MagicMock, and a truthy verify_with reads as a check that is there.	2026-08-28 15:42:12 -04:00
bvandeusen	a6ef3a6a5a	fix(backup): a restore stops flattening the record vocabulary, and a column guard stops the next one (#3182) ... `_note_rows` carried 16 of the `notes` table's 27 columns. A backup -> restore cycle reported success and handed back a corpus with every snippet and process flattened into a plain note, every issue and spike into `work`, every provenance edge gone, and recurring tasks no longer recurring. The record-type and kind vocabulary is what #3128 and milestone 312 were about, and a restore erased it. Two more found by auditing every row helper rather than only the one being edited: `_milestone_rows` dropped `body` — a milestone IS the plan (0066), so every plan restored as a title with no reasoning behind it — and `_repo_binding_rows` dropped `ref`, the branch a ledger follows (#2873), so a restored binding silently accounts for a different tree. `arose_from_id` is deferred to a second pass beside `parent_id`, never written in the constructor: it is an id in the SOURCE database, so copying it through lands the edge on whatever record happens to hold that number here. An edge whose target did not survive stays NULL rather than being guessed at. This is the trap that kept the fix out of milestone 317 step 1. THE STRUCTURAL HALF. The coverage guard from #2293 checks TABLES against Base.metadata; nothing checked COLUMNS, which is how nine went missing from a table that had been "covered" for years — added to the model and the migration, both of which fail loudly, and never to the serialiser, which fails silently. `_COLUMN_EXCLUSIONS` now declares, per table, every column deliberately not exported and why, and a parameterised guard walks all 23 helpers and asserts the two agree. Forgetting is no longer expressible. Reconciling all 23 turned up one more deliberate exclusion worth naming: the `code_shapes` proposal columns are the machine's standing suggestion, cleared by judgment and recomputed by every refresh, so carrying them would restore stale guesses over a tree the proposer has not seen. Tests: the round trip drives the REAL restore_full_backup against Postgres, not a reimplementation of its loop — a test that re-derives the remap it is checking would agree with whatever the product does, including nothing. Backup v12.	2026-08-28 15:22:50 -04:00
bvandeusen	2263fd04a4	fix(tests): the notes table has 27 columns — deleted_batch_id is the second deliberate exclusion (#3165) ... The pin test caught its own inaccuracy on the first run, which is what it is for. `deleted_batch_id` comes from SoftDeleteMixin alongside `deleted_at` and is excluded for the same reason: trashed rows are not exported, so neither is the batch id that groups them for restore(). The nine-field gap #3182 tracks is unchanged.	2026-08-28 14:55:44 -04:00
bvandeusen	2065781302	feat(notes): a note can carry its own check — verify_with, expires_when, verified_at (#3165, milestone 317 step 1) ... The sibling of migration 0090, one table over. Same distinction: a NORM is a decision with no truth value; a CONSTRAINT asserts a fact about someone else's software and goes false with nobody watching. Notes hold far more constraints than rules do and hold them longer — a cross-project reference asserting what a signing service does on a duplicate upload is believed by every project that reads it, and nothing in the record says when anyone last looked. note_supersessions only fires once a human has already believed it. Three nullable columns, no backfill, no index. The index margin is thinner than 0090's — thousands of note rows against hundreds of rules — so the comment says to decide it in step 3 against a real query plan rather than guessing here. The columns land on every row in `notes`, but only non-task, non-snippet records will be OFFERED them (gated at the service in step 2): a task's decay is its status, and a snippet already carries a richer location-aware verdict in data.verification. A schema-level gate would have meant a CHECK across three columns to say what the write path says in two lines. Backup carries the trio (v11), with `verified_at` restored through _dt_or_none — _dt substitutes now(), which would restore every never-checked note as checked at the moment of the restore, inverting the one signal the sweep reads. Found while doing that, NOT fixed here, and now pinned by a test: `_note_rows` carries 16 of the `notes` table's 26 columns. note_type, task_kind, arose_from_id, the recurrence pair, the lifecycle stamps, description and data have all been missing for a long time, so a restore flattens every snippet and process into a plain note and every issue and spike into `work`. The coverage guard cannot see it — it checks TABLES, not columns, which is #2293's failure mode one level down. #3182 tracks it; arose_from_id needs the second id-remapping pass parent_id gets, which is why it is not a drive-by fix.	2026-08-28 14:54:07 -04:00
bvandeusen	f80401d58e	fix(knowledge): the browse vocabulary catches up three kinds, and a snippet's mirror survives the generic door (#3128 recs 2-6) ... Spike #3128 found the storage sound and the retrieval vocabulary frozen before `issue` shipped (0065). Five things, in the order they had to land. **The mirror (rec 5, the data-integrity one).** `notes.data` is DERIVED from a snippet's body, but only `update_snippet` knew that. `update_note` is a hasattr loop with no snippet awareness, and both doors reach it — so PATCH /api/notes/<snippet_id> {body} rewrote the body and left the mirror behind. `snippet_fields` PREFERS the mirror, so the row went on reporting its old repo/path/symbol to the location reverse lookup and to prior-art recall while displaying its new body: surfaced with full authority, and wrong. `snippets.recompose_data` rebuilds it from the body, carrying `verification` and `provenance` (neither is in the body to parse). An explicit `data` still wins, so every snippet-service write is untouched. **One facet table (rec 3), before adding any facet.** The type predicate was written three times — SQL, Python over semantic candidates, and a ternary computing the `is_task` pre-filter — and agreed only by luck. Adding `issue` to the SQL arm alone would have set the pre-filter to is_task=False, handed the Python arm a candidate set with no tasks in it, and returned an empty semantic half for the Issues facet forever with nothing red. `_FACETS` now generates all three. The Python arm also regains the `status IS NULL` half its SQL twin always had. **Issue and spike become facets (rec 2).** 435 issues — 17% of every task — were filterable nowhere on the human surface, while retired `plan` (90 rows) had a chip of its own. `_VALID_TYPES` was a hand-kept copy and is now derived. `plan` stays a valid facet for its legacy rows; it loses its chip. **Snippets stop being half-present in the feed (rec 4).** All 90 were in the All list, in no count, wearing an empty badge, and opening in the note editor. Counts now group by task_kind — every kind for the same two round-trips, which is why `issue` had no number — and total includes snippets, so the All chip matches the list it labels. Snippet cards route to /snippets/:id. **The prose that excused it (rec 6).** `snippet_fields` and the `data` column both still said pre-0070 rows were "never backfilled". True when 0070 landed, false since `backfill_snippet_data` shipped, and it read as licence for a stale mirror. Tests: the pre-filter can never exclude a row its own facet accepts (the regression, parameterised over every facet); both dialects select exactly their own rows; an unknown facet matches nothing; the mirror follows a body or title write, carries the verdict, and yields to an explicit `data`. `compiled_sql` moves to tests/helpers rather than becoming a third copy. Write-up: note #3161.	2026-08-28 12:06:43 -04:00
bvandeusenandClaude Opus 5	a0b54ff6a3	feat(ui): task rows show their kind — a badge for issue and spike (#3124) ... task_kind was only visible inside the task editor's Kind select, so every list surface rendered work, issue and spike identically and a list of tasks hid the fact that three different things were in it. ONE component, not a fifth spelling. The badge layer had already drifted — StatusBadge.vue is the recorded canon (#2960) but WorkspaceTaskPanel, ProjectView and KnowledgeView each carry their own scoped `.status-badge`. KindBadge is modelled on PriorityBadge, its closest sibling, which already does the thing that matters here: the DEFAULT value renders nothing. `work` is most tasks, so badging it would put a chip on nearly every row and say nothing — the same reason RuleListPane marks only `conditional`. COLOUR BY TEMPERATURE, measured rather than eyeballed. Issue and spike are opposite in character — corrective vs exploratory — so they split warm (warning) against cool (info), which survives being small and stays distinguishable without reading the word. Neither uses the accent; kind is not one of the places it is allowed. The raw semantic colour FAILS the contrast floor on the dark palette: warning on its own 12% tint measures 2.97:1 against AA's 4.5. So the text is the hue mixed toward --fs-text-primary, which passes and, because that token inverts by mode, follows light/dark for free. Measured both ways — issue 5.23:1 dark / 6.68:1 light, spike 5.33:1 / 9.26:1. `plan` renders hue-free and italic: retired since 0066, so a legacy row should read as archival rather than as a fourth kind competing for attention. In KnowledgeView it is passed as null instead, because the type badge beside it already says "Plan" and two chips reading the same word would look like two facts. Weight is 500, not the 600 the two older badges use — the house style allows 400 and 500 only, and copying 600 would spread it. SERVER FIX, without which this was decorative: dashboard's `_task_row` omitted task_kind entirely. The badge would have rendered nothing there while working everywhere else, which reads as "this list has no issues" rather than as a missing field. The guard is on the payload, where the omission was. Surfaces: ProjectView's three status columns, WorkspaceTaskPanel's two task lists, DashboardView's milestone and no-milestone rows, KnowledgeView's result rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 18:43:45 -04:00
bvandeusenandClaude Opus 5	69d93898d9	fix(tasks): a task's kind is correctable — the Kind select stops lying (#3129) ... `kind` was accepted at CREATE on both doors and dropped at UPDATE on both: update_task had no such parameter, and the REST PATCH allow-list never read the field. So a task filed under the wrong kind could never be corrected. The frontend made it worse by looking like it worked. TaskEditorView binds a Kind select, marks the form dirty, and HAS ALWAYS SENT `kind` in the update payload — the store even types it. The route ignored it, returned 200, the view optimistically updated, the toast said "Task saved", and the old value came back on reload. Silent success, same class as #2709. Found by trying to re-file #3126 as a spike after deploying 0091. It could not be done; the task had to be recreated as #3128 and the original cancelled. One seam, not two doors. `minted_kind()` lives in services/notes.py because the REST route cannot import an MCP tool module and a second spelling of the list is how the doors would come to disagree. Both create and update route through it, so a bogus kind is now a readable error rather than a CheckViolationError surfacing as a 500. TaskKind joins TaskStatus and TaskPriority as a real enum, and update_note validates task_kind exactly as it already validated those two — the field had been reaching setattr through the hasattr guard with no validation at all, unnoticed only because no door ever offered it. The `-> plan` question #3129 raised is answered in code rather than left implicit: MINTABLE_KINDS is work/issue/spike, deliberately NARROWER than the column's CHECK. `plan` stays a valid stored value because historical plan-tasks carry it and must stay writable; it is simply not a value any door hands out, and the refusal names start_planning because a caller reaching for it wants a plan. The whitelist and the policy answer different questions and are not the same list. Every new test reads the value BACK. One that only asserted the call succeeded would have passed against the broken code — the route returned 200 while dropping the field, which is how this survived long enough to be found by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 15:55:04 -04:00
bvandeusenandClaude Opus 5	410d616c22	feat(rules): the staleness sweep — which standing rules assert a fact nobody has confirmed (#3097, milestone 312 step 3) ... The query the last two steps were storage for. `rules_due_for_verification` returns every rule carrying a `verify_with`, ordered by `verified_at` ASC NULLS FIRST, each row carrying the check IN FULL — the opposite call from rule_brief, because the reader is about to go and run it. NULLS FIRST is the ordering this turns on. Postgres sorts NULLs last on an ASC ordering, which would put the rules nobody has ever confirmed BEHIND every rule someone once looked at. Exactly backwards: a claim with no evidence at all outranks an old one. Rules with no check never appear, and that is the property that keeps the list worth reading. Most rules are decisions — no truth value, nothing to go and check. If they appeared here the sweep would be the rulebook. `mark_rule_verified(rule_id, still_true)` closes the loop, asymmetrically: passing writes a stamp, FAILING WRITES NOTHING. There is no "verified false" state because a rule whose check failed is not in a special condition, it is wrong — and recording the failure as a flag would let it sit there being false with the sweep satisfied that someone had looked. So it stays at the top until someone corrects or retires it, and the response says so. An unrecognised `tier` filter raises rather than falling back. _valid_tier's silent always_on default is right for a WRITE — a typo should leave a rule binding — and wrong for a FILTER, where the same fallback quietly answers a different question and returns a short list that reads as good news. Deliberately NOT filterable by project: a project reaches rules through project scope, subscriptions, always-on rulebooks and exclusions, and a filter missing one of those paths would UNDER-report — the exact failure this surface exists to prevent. Said so in the docstring rather than shipping a half-correct filter. Ownership-scoped like every other rule read (owned rulebook, or owned project), in ONE statement with an OR across the XOR rather than two queries merged in Python, so the ordering is the database's and cannot disagree with itself. Note that rules have no sharing ACL in this schema — no rule_shares, no rulebook_shares — so there is no wider set for access.py to consult here. Also fixes a test title that had been lying for ten tools: "all sixteen tools" asserted 26. The number now lives only in the assertion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 10:49:47 -04:00
bvandeusenandClaude Opus 5	c61925be76	feat(rules): the write path carries a rule's check, and empty finally means empty (#3096, milestone 312 step 2) ... verify_with / expires_when now reach a rule through both doors and come back on every read. The open question this step existed to settle was how to UNSET a nullable field, and the answer is one convention per door: - MCP: "" still means "leave unchanged" — an agent filling three fields must not wipe the other five — so clearing is explicit, clear_fields=["..."]. Naming the field is the one form that cannot happen by accident. - REST: a cleared form input arrives as "", and the service normalises "" to NULL for every nullable rule column, so an emptied input does what it looks like it does. Two idioms, one outcome, and the normalisation is what makes the step-3 sweep correct: `verify_with IS NOT NULL` would otherwise be true for every rule ever touched through the UI, and the sweep would list the whole rulebook and mean nothing. to_dict renders "" and NULL identically, so this is only visible against a real column — hence the integration module rather than a mock. Editing verify_with drops verified_at. A stamp certifies A CHECK, not a rule; reword the check and the old stamp vouches for something that no longer exists. Safe direction, same asymmetry as _valid_tier: a rule wrongly listed as due costs one look, a rule wrongly vouched for costs the thing the sweep exists to catch. Editing anything else leaves the stamp alone, or a rulebook tidy-up would reset every constraint and the ordering would carry nothing. Reads: rule_brief attaches `last_verified` ONLY to a rule that carries a check — its presence is the signal, and it says both "this asserts a fact that can go false" and "here is how long ago anyone confirmed it". "never" rather than null, per #2483. The check text itself stays in get_rule; a listing needs to know which rules can rot, not how to test them. Search hits carry the full trio, since a hit is exactly the moment someone is about to act on a rule. Also folds in the #3078 finding, which had been sitting as a note: create_rule now teaches that when_to_apply is the retrieval surface and must carry the SYMPTOM — the words you would type while stuck — not just the situation. fake_rule gains the three fields as None for the reason the helper already documents one line up: unnamed, verify_with is a truthy MagicMock and every stand-in rule would claim a check it does not have. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 09:28:03 -04:00
bvandeusenandClaude Opus 5	e08e999406	feat(rules): a rule can carry its own check — verify_with, expires_when, verified_at (#3095, milestone 312 step 1) ... A rulebook holds two kinds of row in one table. A NORM is a decision: no truth value, changes only when its author changes it, and they know they did. A CONSTRAINT asserts a fact about someone else's software, and goes false with nobody present. Milestone 307's audit found nine stale sites; every one was a constraint, and not one norm had rotted. Three nullable columns so a rule can say how to check itself. expires_when is a STATE, not a date — constraints expire when the ground moves, not on a schedule. verified_at NULL means never checked and sorts FIRST in the sweep to come: unexamined outranks examined-long-ago. Most rules set none of the three; a null verify_with is the marker for "this is a decision, there is nothing to go and check," and it only reads that way while it stays honest. Nothing is backfilled and nothing is indexed. A migration cannot invent a check any more than 0088 could invent a trigger, and the sweep reads a whole rulebook — hundreds of rows, on operator demand, never on a request path. Also, in the backup service the fields had to pass through: - Restore now remaps arose_from_id through note_id_map. It has been exported since 0088 and silently dropped on the way back in ever since, so every restore lost every rule's provenance link. - _dt_or_none, because _dt substitutes now() for an absent value. That is right for created_at/updated_at and wrong here: a rule nobody ever checked would restore looking freshly checked and fall to the bottom of the sweep it should top. Column additions do not move BACKUP_VERSION; only new sections do, as when 0088 added when_to_apply/tier/arose_from_id to the same helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 07:44:13 -04:00
bvandeusenandClaude Opus 5	02c1e37620	feat(rules): the write path can notice a standing rule it was never given (#3031, milestone 307 step 5, hook arm) ... A conditional rule is not resident, so a session can be about to violate one it was never handed. This arm notices: when what is being written resembles a rule's trigger, the hint names it and says to read it before deciding it does not apply. A SUGGESTION, and the plan was wrong about why it could be more. It claimed the hook "already resolves a path to an area" — it does not, and nothing in Scribe maps a path to a System or a canonical area (build_write_path_hint resolves paths against snippet LOCATIONS, a different index; the learned-alias idea belongs to another project). Correction logged on the task. Rather than invent path→area inference to make a stale claim true, the arm does what D7 already decided and what this surface already IS: tags bind at enter_project, meaning suggests here. The header of the hook says NEVER BLOCKS; dressing a hint up as binding would have been the actual mistake. CONDITIONAL RULES ONLY. An always-on rule is already in the session, so re-offering it is noise — and noise on a hint that fires on every write is how a hint gets ignored. Telemetry goes to retrieval_logs, NOT note_usage_events, and that is a correctness call rather than a preference: note_usage ids are REMAPPED on a backup restore, so a rule id written there would come back attached to whatever note took that number — silently corrupting the evidence the next true-up is supposed to read. retrieval_logs is never restored and `source` already separates surfaces. record_retrieval's `results` type widened to match what it actually needs (an `.id`), instead of passing a Rule to something annotated Note. The rule dedup gets its OWN state file and query parameter, like the three channels before it — #2708's lesson was that one shared channel lets a hint of one class silence a different class that had never been shown. Plugin version bumped: a hook change clients cannot see did not ship (#1040). The stub is autouse in conftest rather than added to forty-odd call sites: the arm loads an embedding model, and every existing test that stubs the NOTES search would otherwise pull a real model in through the one arm it had no way to know about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 16:35:36 -04:00
bvandeusenandClaude Opus 5	4585cda3ff	fix(rules): import or_ in rulebooks — the tier gate used it unimported (#3031) ... NameError at runtime: the tier/area clause and co_surfaced_partners both build an or_(), which rulebooks.py never imported. py_compile passes on this (it is a name error, not a syntax error) and so did the whole unit suite. Where it surfaced is the useful part. The unit tests mock the session, so the project-areas query returns empty, `reachable` stays None, and the or_ branch is never taken — they exercised the path that avoids the bug. Only the integration lane, with a real project and real rows, went down the branch that needed the name. Six integration tests caught it, including the inception one that merely calls get_applicable_rules in passing. A reminder about which lane proves what: mocking the thing that selects the branch means the branch is untested by construction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:13:40 -04:00
bvandeusenandClaude Opus 5	cd9aa87aa4	feat(rules): tier 1 preloads, tier 2 arrives by area — and a split rule can no longer be read half-way (#3031, milestone 307 step 5) ... The payoff step: a rule stops having to be resident to be honoured. - list_always_on_rules returns the ALWAYS-ON tier only. It is the session-start call, made before any project is in scope, so there is no area vocabulary to match a conditional rule against yet. - get_applicable_rules carries a conditional rule when the project works in an area the rule is tagged to — resolved through systems.canonical_id, so the project's own NAME for the area is irrelevant, which is the entire reason the catalog exists. The gate is applied IN SQL, so `limit` counts rules that will actually surface rather than rules about to be dropped. - Bindingness is a deterministic TAG match, never a similarity score (D7). The vector channel stays a suggestion, in search. co_surfaced_partners is the fix that rule 144 never had. It was split off rule 46 and folded back the same day because "either rule could surface without the other and miss exposing a project to what the entire shape is intended to be" — correct, and merging was the only remedy available. Now a partner ARRIVES with its other half even when nothing else selected it, tagged `via: co_surfaces` so the payload says why. Two limits, both deliberate: only rules the caller owns, because an edge is not a back door into someone else's rulebook; and a project's suppressions are passed as exclusions, because an explicit mute is a decision and an edge does not outrank it. COMPATIBILITY, asserted first in the integration test rather than reasoned about: a rule with no tier, no areas and no edges binds exactly as it did before any of this existed. `tier` defaults to always_on, so an install upgrades and every rule it already had keeps arriving. Getting that backwards would silently stop enforcing rules people rely on, which is worse than any amount of payload bloat. Four unit tests were coupled to the ORDER of a mocked session's execute() calls, so a new query broke them. Rather than pad the sequence and deepen that coupling, the three post-query lookups are stubbed by name — they have their own coverage, and the real wiring is proven against Postgres. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:10:41 -04:00
bvandeusenandClaude Opus 5	6ada97bb0b	fix(rules): resolve the Rule forward ref for ruff and readers (#3030) ... F821: the quoted `"Rule"` in semantic_search_rules' return annotation evaluates fine at runtime — a string inside a subscript is a value, not a name lookup — but it points at nothing a reader or a type checker can follow, which is what ruff is objecting to and it is right to. A TYPE_CHECKING import resolves it at zero runtime cost, while the real import stays inside the function so this module still does not pull in the rulebook models. Placed after the import block, where isort wants the guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:03:40 -04:00
bvandeusenandClaude Opus 5	95a37318fc	feat(rules): rules become findable by meaning (#3030, milestone 307 step 4) ... Rules were the only major record type with no vector, so `search` could never return one and a rule could arrive only by being preloaded. That single fact is what made every rule compete for one always-on budget. THE DECISION THE TASK ASKED FOR, made explicitly: a sibling rule_embeddings table, not a polymorphic embedding row. The ROW could have been generalised; the SEARCH could not. semantic_search_notes is Note-specific scoping end to end — the visibility clause, the supersession penalty, note_type/task_kind/system filters — and a rule shares none of it, scoping instead by rulebook ownership or project. Generalising the row while still needing two searches is the worst of both: a key with referential integrity to neither table, on the path every session start runs, to share four columns. What is genuinely common is BEHAVIOUR — get_embedding, chunk_document, embedding_text, CHUNKER_VERSION — and those are reused as-is. Sharing them is the DRY win; sharing the table would have been the DRY costume. The document shape is measured, not chosen (note 2485). That pass found the snippet was the only discriminative record in the corpus — a 0.153 top-to-second gap against 0.010-0.023 — and that the cause was its SHAPE: purpose stated twice in a short single-topic document. rule_document reproduces it: the trigger in the title AND as the body's first line. And it excludes `why`, which matters more than any of it. `why` is dated incident narrative — rule 46's runs to 4,300 characters — and long multi-topic prose is exactly what made sixteen dev-logs mutually indistinguishable. Adding it would not give the vector more to work with; it would give every rule the SAME thing to work with. rule_document takes no `why` parameter at all, so a well-meaning caller cannot pass one. A rule with no trigger degrades to title + statement — findable, less sharp. That is an argument for backfilling triggers (step 6), not for padding the document with whatever text is nearby. search(content_type="rule") returns the rule WITH its why and how_to_apply: they are its operational half, the session payload never carries them, and a caller who went looking should not have to re-fetch. Writes re-index fire-and-forget like notes; startup backfills in its own try block so neither backfill can skip the other. rule_embeddings is derived, so it joins note_embeddings in the backup's explicitly-NOT-included list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 15:00:49 -04:00
bvandeusenandClaude Opus 5	ffb7a0fe38	feat(rules): both doors carry the trigger, the tier, the areas and the edges (#3029, milestone 307 step 3, surfaces) ... MCP and REST both gain when_to_apply / tier / system_ids / arose_from_id on create and update, plus relate_rules / unrelate_rules for the typed edges, and get_rule now returns a rule's areas and relations alongside it. rule_detail() is a SERVICE function, not one per door. It started as a copy in each — identical, and the prior-art hook flagged it immediately, which is the same lesson rules_payload (#2858) already recorded: a second copy drifts. Both doors call the one seam, so create, update and get cannot disagree about what a rule looks like coming back. The authoring guidance lands in create_rule's docstring rather than in a rule, per rule 119 as the operator described it: this is behaviour every instance should inherit, not one operator's preference. It states the test — ONE RULE = ONE THING YOU COULD VIOLATE. Rules that FAIL TOGETHER get linked with relate_rules(kind="co_surfaces"), never merged into one row. — and names why merging loses: a merged rule cannot be cited, surfaced or suppressed a clause at a time, and it grows without limit because adding to it is always cheaper than adding a rule. create_project_rule says the same about "overrides", which is what FabledCurator's 85/86 should have been instead of near-copies that drift from their parent. The tier arg carries the test itself: can you name the trigger WITHOUT naming a system, an artifact type or a moment? If the honest answer is "whenever you are working", it is always_on. Tests: the applicable-rules cases fabricated raw tuples matching the old column lists, so they move to the entity shape via fake_rule; new cases pin rule_brief (a DATE not a stamp, the depth left to get_rule, no null keys) and that an unknown tier falls back to BINDING. fake_rule gains when_to_apply / tier / arose_from_id for the note-2109 reason the helper exists: unnamed, they would be truthy MagicMocks. The tool tests stub the new rule_detail seam — they are about argument forwarding and have no database. The module header's "Sixteen tools" had been wrong for two milestones; the registration count test is what actually catches that, so the header now says so instead of carrying a number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 14:17:41 -04:00
bvandeusenandClaude Opus 5	6ddb8bf859	feat(rules): a rule can say when it applies, which area it is about, and what it belongs with (#3029, milestone 307 step 3, schema) ... A rule could not state its trigger, its area, or its siblings, so all three were being written as prose instead: a System's charter restating rule text, a `why` naming the note that caused it, and two halves of one shape merged into a single row because either could surface without the other. Migration 0088 adds the four fields those workarounds stood in for: - `when_to_apply` — the trigger. Nullable in the DB and required at the service layer: existing rules have none and a migration cannot invent one. - `tier` — always_on | conditional, defaulting to always_on. This migration therefore changes NOTHING about which rules bind; an install upgrades and every rule keeps arriving exactly as before. Getting that backwards is the one failure this milestone exists to prevent, so _valid_tier falls back to always_on rather than silently un-binding a rule with a typo'd tier. - `arose_from_id` — the record that caused the rule, the edge notes and tasks already have. SET NULL: trashing the source does not repeal the rule. - `rule_systems` / `rule_relations` — the canon tag and the typed edges (co_surfaces / overrides / elaborates), each earned from a workaround its absence forced. rule_brief() replaces the THREE hand-written trim dicts that had already diverged — two carried topic_id, one didn't, and none carried the timestamps the model has held all along. That omission is why a rule written before the capability it duplicates was indistinguishable at read time from one still doing work. It now carries updated_at as a DATE: the question is "how old is this", and a full stamp across the always-on set is ~2k characters for precision nobody reads. The two callers select the ENTITY rather than a column list, so rule_brief stays the single place deciding what a surfaced rule says. Backup: both new tables carried, area tags by canonical SLUG (ids are per-install). The rule-relation restore runs after ALL rules exist and after the catalog, because an edge names two rules and a tag names a global row — sections renumbered so the file reads in dependency order. A pre-0088 payload restores with tier=always_on, i.e. binding exactly as when it was taken. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 14:11:32 -04:00