17 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	9c5ab1d6ad	fix(394): subscription is the scope — areas narrow only where an author asked ... I got this wrong in 0e10f6b and the integration suite caught it. get_applicable_rules filtered `always_on OR area-reachable`. Removing the tier, I kept only the reachable arm and argued that a project with no canonical-tagged Systems should get no bulk rules and reach them by retrieval instead. Two things wrong with that. The query is ALREADY scoped to rulebooks the project SUBSCRIBED to, so the project had opted in and was then handed a subset of what it asked for — subscription is not bulk delivery, it is the opt-in. And milestone 394 is explicit that subscription-derived rules are not the always-on tier and are not what it removes; I narrowed something the milestone said to leave alone. The failure that surfaced it is a good one: a co_surfaces partner never arrived, because the rule it travels with had been filtered out before the edge could drag it in. A behaviour two steps from the change. What ships instead is narrower than "drop the clause" and wider than what I had: every rule in a subscribed rulebook applies, EXCEPT that a rule tagged to specific areas applies only to a project working in one of them. An untagged rule was never narrowed by anyone, so it is general to its rulebook by construction; a tagged one is an author saying "this is about CI" and meaning it. D7's deterministic narrowing is kept where it was asked for and not invented where it was not. Also: three tests covering the SessionStart preload, the always-on tool and the two marker paths — all surfaces that no longer exist — and the backup's declared-section list, which still named the section its table took with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 16:33:16 -04:00
bvandeusenandClaude Opus 5	0e10f6bb8a	feat(rules)!: retire the always-on tier — every rule arrives by retrieval (#394) ... Milestone 394, steps 5-8. Operator: "remove the always on rule functionality as the goal was to not have it at all since it didn't seem to work as expected." Unconditional preload had three failures the retrieval arms do not. It could not be MEASURED — a resident rule is in the context whether or not it mattered, so nothing distinguished "this governed the act" from "this was scenery", and it was the one surface structurally exempt from the scoreboard judging every other. It was SUMMARISED AWAY by compaction while the session went on believing it held the rules. And it CROWDED OUT the few rules that applied with the thirty that did not. WHAT GOES Schema (0100): rules.tier + ck_rules_tier, rule_versions.tier, rulebooks.always_on, and project_rulebook_exclusions — a table recording a project's opt-out of something that no longer binds it unasked. Tools: list_always_on_rules, exclude_always_on_rulebook, include_always_on_rulebook. Service: the same three plus rules_etag_for, _valid_tier and the whole etag family. The SessionStart preload and the write-path staleness arm go with them: nothing is resident, so nothing can have drifted since a session loaded it. THREE CALLS WORTH REVIEWING enter_project got NARROWER, not wider. Its filter was `always_on OR area-tagged`; dropping the tier arm leaves the deterministic half, so a project with no canonical-tagged Systems gets no bulk rules and reaches them by retrieval instead. Dropping the whole clause would have made that payload bigger than the preload this milestone deletes. Backups import tolerantly. A pre-394 archive carries tier, always_on and the retired inception choice; none is read, and the exclusion key is DROPPED rather than remapped, because restoring it would write data that validate_inception now rejects as unknown. The migration is irreversible in the way that matters and says so: downgrade recreates the columns at their defaults and cannot restore which rules were always-on. A value invented to fill a hole is not a measurement. THE INSTRUCTION SURFACES SAY THE HARDER THING Deleting "call list_always_on_rules()" is easy; replacing it is not, because the new model asks a session to trust something it cannot see. All three surfaces now say a session holds nothing, that rules arrive when work matches them, and — the half that got dangerous — that "no rule arrived" means "nothing matched", never "there is no rule". Under residency an empty session was rare and suspicious; it is now the ordinary state of most turns, so reading it as permission is wrong on nearly every turn rather than occasionally. That is #3720's defect at session scale. test_instruction_surfaces_agree is repointed rather than retired: its two halves collapsed into one instruction, and it gains a guard that every surface states what absence means. _INSTRUCTIONS is back at 1999/2000 — the inception clause paid for the longer HOW line. UI (rule 27, and the opportunity step 8 named) The tier selector is gone, and what replaces it is the point: `when_to_apply` is now the field that decides whether a rule is ever seen, so the editor marks it required, warns while it is empty, and both rule lists badge a trigger-less rule "never surfaces". A rule without one is not quiet, it is unreachable. TESTS Two files deleted outright — test_rules_etag.py and test_inception_rules.py tested subsystems that no longer exist. Elsewhere obsolete cases were removed and the rest repointed. One deserves naming: the wiring test asserted the act arms pass no `tier`, which had become an assertion that could not fail. It is repointed onto `kind`, which does still exist and where the same claim is live — a preference must reach a write exactly as a rule does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 16:22:17 -04:00
bvandeusenandClaude Opus 5	690ca0306e	feat(rules): the command arm gets its own bar, measured (#3853) ... One threshold served both act arms. The telemetry says they are not the same problem: write_path_rule 2,325 calls, speaks on 37%, near-miss p50 0.6989 pre_tool_rule 11,768 calls, speaks on 2%, near-miss p50 0.6794 The second is not quiet, it is mute — 11,530 of 11,768 calls said nothing, with near-miss p90 at 0.7097 against a 0.72 bar. Refused mass piled one hundredth under the line is what a bar set too high leaves behind, and the note arms are the control: auto_inject refuses at p90 0.5463, write_path at 0.6738, both far below theirs. The cause is query shape, not corpus. A write-path query is a code payload, long and rich — the case 0.72 was calibrated on. A pre-tool query is a shell command, often under a dozen words: less text, less signal, lower scores for the same relevance. MEASURED. Eight replayed queries against the post-#3855 corpus, consequential acts against innocuous ones: 0.7571 git push origin dev consequential 0.7245 cd ...; git fetch; git add -A consequential 0.7193 git pull --rebase origin dev consequential 0.6850 docker compose up -d consequential ------------------------------------- 0.68 0.6735 wc -l src/*.py && date innocuous 0.6544 grep -rn useState src/ innocuous 0.6099 sed -n '120,160p' package.json innocuous 0.6056 ls -la && cat README.md innocuous At 0.72 three of four consequential acts retrieved nothing, including `git pull --rebase origin dev`, where rules 153, 1 and 2 all ranked correctly between 0.7126 and 0.7193 and were all refused. The separation is 0.0115 wide. That is a direction, not a settled number, and the comment says so — near_miss_samples on a few days of post-#3855 traffic is what settles it. This also corrects an assumption the old comment stated: it argued 0.68 sat "below where this corpus's noise sits", inferring a higher floor from the corpus being homogeneous. Measured, the command arm's noise ceiling is 0.6735, so 0.68 clears it barely rather than sitting under it. Lowering is safer now than it would have been. Until #3851 this arm had one slot, so the bar was the only noise control; the band now filters downstream, so the bar's job shrank and the bar can. write_path_rule is unchanged — healthy at 0.72 on its own evidence. Guards: the two bars parse independently, garbage falls back to its OWN default rather than to the sibling's (which would silently re-merge them), the command default stays below the write-path default as a direction check, and each arm both SEARCHES and REPORTS at its own bar. That last one is a failure the single-bar code could not have had: retrieval_logs.threshold is what near-miss analysis is read against, so an arm searching at one number and logging another misreports the refusal and invites moving the bar that was already right. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 14:36:18 -04:00
bvandeusenandClaude Opus 5	40189147d2	fix(rules): a shortened rule line must not decide what it says about holding (#3851) ... CI run 6485 was red. Six failures, three causes, and only one of them was a stale test. THE REAL DEFECT. The compact branch dropped the `seen` TAIL along with the trigger, so a rule the session had already been told rendered exactly like one it had not. #3750's whole argument is that those are different claims — a repeat is rendered precisely because the session may no longer HOLD what it was told — and the tail is the entire difference a reader can act on. test_a_rule_the_session_already_holds_is_referenced_not_re_offered caught it within one commit, which is that guard working as intended. Fixed by keeping the tail and dropping only the trigger, which is both the cheaper and the safer cut: a trigger runs 300-400 characters after #3855, a tail about 100. Re-measured on the real renderer — top-full-plus-references is ~299 tokens against ~568 for five full lines, so about 2x the old single line rather than the 1.4x claimed before, for four more rules and no lost information. The comments carrying the old figure are corrected rather than left to read as a decision nobody made. THE FIXTURE THAT STRADDLED THE BAND. `_THREE_HITS` spanned 0.81-0.74 against a 0.05 band, so the act arms dropped its lowest hit and four cases of test_both_recorders_report_the_same_rules_for_one_call failed reporting a count mismatch — under a message blaming the exclusion filter. A guard pointing confidently at the wrong subsystem costs more than no guard, because it is believed. Scores retightened to 0.81/0.80/0.79 and the precondition is now asserted by a named test, so a future band change is told where the problem is instead of through four confusing failures. THE STALE CONSTANT GUARD. test_the_rule_arm_asks_for_one_rule_not_two pinned RULEHINT_LIMIT == 1 — a real decision, correctly guarded, for a world with a resident set. Rewritten to pin what replaced it, as relationships rather than values (rule 115): the arm can return several, and rules are narrowed HARDER than the notes menu because they measured flatter, not sharper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 14:07:32 -04:00
bvandeusenandClaude Opus 5	d5f96563fd	feat(rules): a slot a preference cannot lose (#3894) ... Milestone 399 step 4. A rule and a preference are not equally served by one ranking, because their losses are not equal: - a RULE crowded out at the prompt boundary still fires at an act arm. A push reaches pre_tool_rule, a write reaches write_path_rule. The prompt hit is a preview of a second chance. - a PREFERENCE about how to answer has no second chance. The response IS the act, so crowded out there it is never delivered at all. A straight ranking therefore favours the record whose loss is recoverable over the one whose loss is total, and does it INVISIBLY: the rule that won is a legitimate hit, the telemetry reads healthy, and the only symptom is a preference that quietly never arrives. reuse_slot exists for the same shape one corpus over (#2463). `semantic_search_rules` gains a `kind` filter, so the slot's query can only answer with what the slot is for. Verifying afterwards would be weaker — an unfiltered search that happened to return a rule would spend the slot on it, and that line would be indistinguishable from one that earned its place. THE SLOT BUYS POSITION, NOT A LOWER BAR, matching reuse_slot. A weak preference cannot buy it, so silence stays the default. The task asked for a separate threshold; I did not add one, and the reason is that the worry behind it — reading a miss rate as a fact about preferences — is answered by `preference_slot` being its own logged source, where best_available_id names which preference was refused. A knob added on a guess is a way to misconfigure the surface; a bar moved on evidence is an argument. The evidence arrives on its own now. IT EXTENDS, IT NEVER DISPLACES — and here it parts from reuse_slot, which evicts its menu's weakest hit. A displaced hit sits in prompt_rule's retrieval_logs row while never being surfaced, so that source's two tables stop agreeing and #3668's identity breaks for a reason nothing in the data explains. Milestone #379 is what losing that identity costs: five steps planned against two counters disagreeing, not a write path dropping rows. One extra line in a rare case is the cheaper price. It also runs BEFORE the bail-out. An empty general result is not proof no preference qualifies: that search overfetches by distance then collapses, so a preference ranked below the window is invisible to it while a kind-filtered query finds it at once. Bailing first would make the slot dead in exactly the corpus it exists for. One existing assertion repinned from a bare call_count to a per-source filter: the slot logs its own query on the same call, and a count would pin the number of arms rather than the property — going red the next time one is added, which is rule 167's false alarm about the thing it protects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 08:44:24 -04:00
bvandeusenandClaude Opus 5	44e0b0541f	feat(rules): rules retrieve against the operator's message (#3852) ... The third rule arm, and the one the other two cannot reach. `write_path_rule` is keyed on code, `pre_tool_rule` on a command — both things the session is about to DO. A rule that governs what to SAY has no such trigger: extract intent from loose phrasing, raise a conflict before acting, hand off an action with its reason, end a finding with an offer all bind on a RESPONSE, and no tool call precedes one. The operator's message is the only query that exists before a response is composed. That hook searched notes alone, so no rule had ever been retrieved against a thing the operator actually said — and residency was the only surface those rules had, which is what milestone 394 removes. A SEPARATE FUNCTION, not a branch in build_autoinject_hint, because of its early returns. That arm bails when auto-inject is disabled, when the query is blank, when nothing clears the note bar — every one a statement about NOTES. Folded in, an operator who turned the awareness menu off would silently lose their rules, a coupling with no symptom since both look like a quiet hook. Two functions, two sets of gates, composed in the route. Guarded as "the rule arm never asks the notes arm's config", which is the structural fact. Joins _ARMS rather than getting its own test file. #3497's history is that the pre-tool arm inherited a defect from its sibling by being MODELLED on it instead of sharing with it, and a third arm modelled on two is two chances to repeat that. Repeat rendering, fresh-only counting, log-before-bailout, the kind register and the two-recorders identity are properties of every arm or of none. The bar is INHERITED and says so. 0.72 was tuned against code and commands; prose is a different query shape against the same documents, and triggers are written in the vocabulary of the moment — which for most rules is act vocabulary. Starting at the only number with evidence behind it and logging every call from the first deploy is what makes it settleable; guessing lower would put an unmeasured bar in front of a corpus that binds. k=3, anchored on this hook's own budget rather than the act arms'. RULEHINT_LIMIT is 1 because that arm fires before every Bash call; this one fires once per turn, beside a notes menu already spending three slots. And a prompt genuinely contains more than one act — "merge to main and then start on X" is two — where a command is one thing. `prompt_rule` added to RANKED_SOURCES: a ranker picked it, and a ranked source missing from that tuple is silently counted as bulk delivery and drops out of the pull-through denominator. The hook reads and writes the SHARED rule ledger under scribe-priorart, not a private one — one session keeps one list, aged (#3751), so a rule named here is not re-announced before the next Bash call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 07:56:19 -04:00
bvandeusenandClaude Opus 5	26e0dff706	feat(rules): a preference does not speak in a rule's voice (#3849 step 3) ... Two independent axes on one hint line. `kind` decides the head, `seen` decides the tail, and neither reads the other — which is what let a second kind arrive without reopening #3750's repeat question. Whether a record is already on the exclusion ledger has nothing to do with how much force it carries, so the seen branch is shared verbatim. The noun carries the whole visual difference, deliberately. A reader skimming an injected block gets one word to place the register, so the word that moves is the one naming force: "Standing rule" / "Preference". Everything structural after it is identical, so the kinds read as one set rather than two formats. Force is asserted in exactly one other place, and that moves too. A rule's line says to read it BEFORE DECIDING IT DOES NOT APPLY, because dismissing a rule unread is how the thing it prevents happens. A preference makes no such claim: it says where to find HOW THIS HAS BEEN DONE BEFORE, and following it buys consistency rather than correctness. Guarded on both places at once. Pinning the noun alone would pass a line reading "Preference … before deciding it does not apply" — label swapped, instruction kept — which is worse than not distinguishing them, because it looks handled. And a guard on the independence claim itself, exercising all four combinations: the way this breaks silently is a seen branch that grows a kind test, leaving one combination rendered by nobody's intention. Noted, not fixed: plugin/skills/using-scribe/SKILL.md still says "Standing rules are binding" with no room for a kind that does not. That surface is step 6's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-10 22:04:14 -04:00
bvandeusenandClaude Opus 5	c4908f093f	feat(rules): a repeat is referenced, not withheld (#3750, #3752) ... Both arms used to drop a hit already on the session's exclusion ledger and emit nothing. That is correct only while the session still HOLDS what it was told, and a compaction breaks exactly that: the earlier injection is summarized away while the id stays on the ledger, leaving the rule absent from context AND unreachable for the rest of the session. #3749 closed the compaction half by clearing the ledger; this closes the ordinary half, where a session simply stops holding a line it read an hour ago. Only ONE CLAUSE of the existing line is false on a repeat — the claim that the rule is not in the session's loaded set — so only that clause changes. The fresh line is byte-identical to what it was. Both tails now come from one `_rule_hint_line`. The arms phrase their heads differently on purpose; everything after must not differ, and #3497's history is that the pre-tool arm inherited a defect by being modelled on its sibling rather than sharing with it. THE BUDGET DECISION, recorded at RULEHINT_LIMIT. A repeat competes for the single slot on rank alone: nothing is fetched behind it, and it never rides alongside as a second line. Promoting a fresh rule past a better-ranked repeat would reinstate the withholding one rank deeper, and a second line is the one thing the limit exists to forbid. The consequence is deliberate — a rule that keeps ranking first for a recurring situation keeps being referenced, and its decay belongs to exclusion ageing (#3751), not to a first-place rule being demoted for having won before. THE TELEMETRY, decided before shipping rather than after a number moved (#3752): nothing changes. A reference is a RENDERING decision, not a retrieval outcome. `results` stays `fresh`, `suppressed` stays len(hits) - len(fresh), and `record_rule_surfaced` still counts only what the arm freshly chose. This matters more than it reads: the naive implementation drops the `fresh` filter and takes suppressed_count to zero everywhere — and #3739's near-miss fix identifies repeat-caused zeros by `suppressed_count > 0`, so the contamination corrected on 2026-09-08 would return by a different route, in the same field, with the fix still in the code and no longer working. A test asserts the counters as unmoved, because "nothing changed" is only worth something if it is checkable. Also corrects two comments that outlived #3702 — both arms still claimed CONDITIONAL ONLY while the module-level note above RULEHINT_LIMIT said the opposite, in the exact code this change edits. GUARDS - an already-held hit produces a rule line at all (the regression), asserted on `get_rule(<id>)` rather than a truthy context: the write-path arm fills its context from four other sources, so truthiness passes under the OLD behaviour and pins nothing there. - the two tails are distinguishable, each excludes the other, and neither injects the rule statement — the budget claim, both branches. - the counters are unmoved, per #3752. - test_a_rule_the_session_already_holds_is_not_re_offered asserted the old contract (`"161" not in context`). Repinned rather than deleted: the telemetry half of what it protected still holds. Repinned the #3497 log-placement guard on structure (rule 167). It read `body.index("if not fresh:")` — a local variable NAME, not the property. This change renames that guard to `if not hits:`, so the old assertion would have raised ValueError and reported #3497 as back while the arm was entirely correct. Now walks the AST for the first early return after the search and asserts the call row is written before it. Falsified both ways: it fails on the #3497 mutation, and it refuses to pass when no early return exists at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 21:48:50 -04:00
bvandeusenandClaude Opus 5	5e19a1b028	test(telemetry): FALSIFICATION — prove the identity guard can fail (#3668) ... Deliberately broken, reverted in the next commit. Rule 167 requires every guard be falsified against the regression it names before it is trusted, and rule 10 puts CI as the only place that can run it — so the failure has to be made to happen here rather than argued for. The guard: both rule arms feed one `fresh` list to two recorders, so the call log and the surfacing log cannot disagree about what one call showed. The regression: `results=fresh` becomes `results=hits` in both arms, so the call log counts what the ranker found while the surfacing log counts what was shown. That is not a hypothetical shape. It is exactly the divergence that would make a correct system report a lost write when the two tables are later compared in aggregate — the reading that scoped this milestone at five steps against a defect that did not exist. Expected red: the one-already-held and all-already-held cases on both arms. The nothing-held case must still PASS — with no exclusions both recorders see the same list however wrongly they are wired, which is why it could never have been the discriminating case. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 13:31:49 -04:00
bvandeusenandClaude Opus 5	dd1e6e2645	feat(rules): the rule arms stop filtering the corpus to one tier (#3702) ... Both arms passed tier="conditional", on the reasoning that an always-on rule is already in the session so re-surfacing it is pure noise. That conflates two different things: PRESENT IN CONTEXT the rule was delivered at session start SALIENT AT THE MOMENT the rule is in front of the reader when the action it governs is about to be taken A rule handed over in a list at turn zero is present while a session writes a config value three hundred turns later. It is not surfaced. So the filter did not skip a redundant hint — it made a whole class of rules permanently ineligible for the only mechanism that puts a rule in front of an agent AT the moment, and the more important a rule is, the likelier it sat in that class. Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs in a hint is a relevance question and a similarity bar is the control for relevance. A categorical exclusion standing in for a relevance judgment cannot be tuned, cannot be measured, and cannot be wrong in a way anybody notices. MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on every write and says obvious things teaches the reader to skip the block. It had simply never been checked, and retrieval_logs already records the scores to check it with. Rules clearing often and high means the fear was justified and the BAR is the work; rules clearing rarely in a thin band means relevance was always sufficient. Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the resulting distribution has one cause — and k=1 bounds the blast radius: a wider pool can change which rule surfaces and how often, never how long a single hint gets. The threshold rationale's first premise ("the eligible corpus is TINY — tier=conditional only") is updated rather than deleted: a larger pool makes clearing the bar mean MORE, so that argument weakened, and the bar was left alone anyway rather than move two variables at once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 00:24:02 -04:00
bvandeusenandClaude Opus 5	8be555d6dd	feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497) ... Making the rule arms log every call exposed a second ambiguity in the same row. `result_count == 0` is two unrelated events wearing one number: - the ranker found nothing above the bar — the only evidence a threshold is set too high; and - the ranker found only what this session had already been shown — which says nothing whatever about the bar. A long session excludes its way into the second, so the arm reads worse the longer it runs correctly. Rows written now carry the ambiguity permanently, which is why this lands before any watch period rather than after. `retrieval_logs.suppressed_count` (0095, nullable) holds what the caller dropped as already-shown. Both rule arms report it; they filter in Python and always know. The note arms pass exclusions INTO semantic_search_notes and never see what was dropped, so they store NULL. THE NULL IS LOAD-BEARING. It means "not measured here", and the readout renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0 would let an unmeasured surface read as a perfectly clean one — the same substitution of an artifact for a measurement that #3311 made. No backfill, for the same reason: existing rows genuinely do not know. `retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`, `calls_with_suppression` and `zero_because_already_shown`; subtract the last from `zero_result_calls` for the true ranker declines. The MCP tool docstring says to read the two together and warns against reading the null as a zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-03 21:12:38 -04:00
bvandeusenandClaude Opus 5	154a5de13e	fix(telemetry): both rule arms logged only their hits, so the clear-rate could only read 100% (#3497) ... `write_path_rule` reported `zero_result_calls: 0` and `cleared_threshold: 133/133` — a perfect record no other surface comes near (`write_path` 421 zeroes of 613, `reuse_slot` 124/199, `auto_inject` 114/326). #3311 read that as a measurement and milestone 333 was scoped on it. It was an artifact. Both arms called `record_retrieval` inside a guard on having results — the write-path arm behind `if fresh:`, the pre-tool arm below `if not fresh: return out` — so a call that found nothing wrote no row. The statistic was a fact about the shape of the code, true at any threshold whatsoever. The call log moves out of the guard in both arms. The surfacing log stays in it: nothing was shown, so no surfacing occurred. `results=fresh` is kept deliberately — the note arms pass exclusions into `semantic_search_notes`, so what they log is already post-exclusion, and logging `hits` here would make this row mean something other than every other row in the same readout. The defect bites hardest on the pre-tool arm, which fires on every Bash call: with no rows at all, a ranker that declined is indistinguishable from a hook that never fired — the silent failure the arm exists to stop. Tests cover both arms behaviourally (found nothing; found only what the session already held; searched nothing at all, which must stay silent) plus a structural guard, because this was one level of indentation and it appeared independently in two places. #3311 and the `rule_usage` docstring corrected rather than quietly rewritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-03 06:57:13 -04:00
bvandeusenandClaude Opus 5	2ee24b9d2b	feat(rules): rules before tools — a PreToolUse arm keyed on the action (#3476) ... The only just-in-time rule surface was registered on `Write|Edit` and queried with `code or path`, so a rule could be retrieved at the moment of a code write and nowhere else. Every rule about which tool to reach for — don't curl the forge, don't stand up a stack, don't run the suite locally, don't branch — was unreachable exactly when it mattered, and residency in the always-on preload was the only surface it had. That is the pressure that grew the resident set to 31 against #3089's ceiling of ~23; it was never a judgment anybody made. A reflex generates no query, so an instruction to check the rules cannot catch one. A mechanical trigger can: the tool call IS the query, and a reflex has to become a tool call before it can do anything. `build_tool_rule_hint` is deliberately tool-agnostic — a name and a string — so widening the matcher later is a hooks.json edit with no server change. The hook starts on Bash, which is where the action reflexes live. The two pre-tool arms share ONE session ledger of already-named rules (`<state>/<sid>.rules.ids`). Two ledgers would mean a rule named by one arm gets re-offered by the other, and the hint that fires most often is exactly the one that must not repeat itself. A test asserts both scripts build the same path, and another checks the shell hook and the Python route agree on every query-arg name (rule 33) — a rename there fails silently, looking like a surface that never finds anything rather than a broken one. Deliberately silent on outage, unlike the prior-art hook: a write is occasional, a Bash call is not, and an outage line before every command is what gets a channel muted. `tier="conditional"` matches the write arm and is the transition point — an always-on rule is already resident, so re-tier one and it starts arriving here instead of in every session's preamble. `pre_tool_rule` joins RANKED_SOURCES: this arm chose what it showed, so a pull can settle whether the choice landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-02 23:31:22 -04:00
bvandeusenandClaude Opus 5	8b9b3a1d9b	feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473) ... The ranked rule arm became measurable in M333. The preload did not — and that is the surface whose value is actually in question. `list_always_on_rules`, the SessionStart block and every `rules_payload` caller handed rules over wholesale and emitted nothing, so the resident set's token cost was certain and its usefulness could not be tested even in principle. Bulk deliveries now record as AMBIENT, beside the ranked count and never inside pull-through. Folding them in would mean growing the always-on set depressed the arm's measured precision and trimming it flattered the arm, neither for any reason to do with the arm. `RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there is one ranked rule source and this change adds seven bulk ones, so naming the rare half makes a forgotten surface default to ambient — under-counting it — rather than padding the denominator with surfacings nobody chose. Two lookalike call sites are deliberately left silent, with a test to keep them that way: the write-path etag arm and `rules_etag_for` read the rules to build or compare a MARKER and show nobody anything. No migration — `event` and `source` are plain Text with no CHECK (rule 36 does not apply). Snippet #2858 updated to the new `rules_payload` contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-02 23:06:40 -04:00
bvandeusenandClaude Opus 5	238510080e	feat(retrieval): the standing-rule arm gets its own bar, and asks for one rule not two (#3318) ... Milestone 333 step 4 — the split #2223 made one surface down, now made for the third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT. THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two facts hold on any install, including one with six rules and no telemetry: - The eligible corpus is tiny — conditional rules only, a handful to a few dozen against thousands of notes. A top-k over forty candidates always returns something, so "the best match cleared the bar" stops meaning "a good match exists". A bar calibrated for best-of-thousands is cleared by best-of-forty as arithmetic, not relevance. - Rules are short imperative technical English, far more homogeneous than note prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it; a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely inherited, it sits below where this corpus's noise lives. 0.72 errs deliberately toward silence on an asymmetry that is also structural: this hint fires on EVERY write. A missed rule is recoverable — it is still in Scribe and the agent can search it. A hint that cries wolf is not: it teaches the reader to skip the whole block, and the true positives go with it. The arm's own comment already said "noise on a hint that fires on every write is how a hint gets ignored". Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting the relationship — which would silently reinstate #3311 — does not. RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2 means the second line is almost always the second-best noise wearing the same confident framing as the first; halving k halves that regardless of the bar. It stays a constant because it is a decision about how loud one hint may be, not a per-install tuning question — and a knob nobody turns only adds a way to misconfigure the surface. Reachable from Settings, no restart (rule 25), with copy that says which way to move it and points at retrieval_telemetry's rule pull-through — which step 3 made readable — to tell "arriving unread" from "never arrived". Every config stand-in in the suite gained the key, not just the one that noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so a missing key raises inside its fail-open except and turns the arm into a silent no-op — indistinguishable from it running and finding nothing. That is the same vacuous-pass shape that bit step 2, one layer down (rule 33). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 18:05:00 -04:00
bvandeusenandClaude Opus 5	70761b16d9	test(telemetry): the rule-arm fixture never reached the arm — it returned at the guard (#3316) ... The product code was right; the test was wrong, and wrong in a way that made two assertions fail and two others pass vacuously. `build_write_path_hint` returns early when a write matched nothing at all — no staleness, no synced record, no prior-art menu, no shape signal. The rule arm sits deliberately on the FAR side of that guard, because it runs a semantic search and hoisting it would mean an embedding query on every write in the session. My fixture stubbed every other arm to empty, so it hit the early return and the rule arm never ran: `record_rule_surfaced` was called zero times, and "the recorder was not called" is also what two of the four tests were asserting for their own reasons. The fixture now supplies one prior-art hit — 0.72 against a 0.6 threshold, so it clears the band and the top_k slice — with a comment saying the hit is the arm's precondition rather than scenery. And the gate got its own test, because the fixture now depends on it: a write matching nothing must NOT reach the arm. Without that, a future change to the guard would make every assertion in this file pass without exercising anything. #3311 is explicit that the gate stays until the arm's precision is fixed, so the test says to go read that issue rather than update the assertion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 17:17:59 -04:00
bvandeusenandClaude Opus 5	8f7f447fda	feat(telemetry): the rule arm records what it showed, and get_rule records the read (#3316) ... Milestone 333 step 2. Step 1 built the table; a counter nobody calls reads zero and looks exactly like a surface nobody uses, which is #2663's shape. SURFACED — the standing-rule arm in build_write_path_hint, beside the record_retrieval it already made. Two tables, and the split is not arbitrary: retrieval_logs is one row per CALL keyed on the score distribution a threshold is tuned from; rule_usage_events is one row per RULE per event, the grain "was this hint ever acted on" needs and the grain a JSONB result_ids array cannot be indexed at. The comment there said rule ids had nowhere to go — that note_usage_events remaps ids on restore, so a rule id would return attached to whatever note took that number. Still true of the NOTE table, and precisely why step 1 built its own. Rewritten to say the gap is closed rather than leaving a stale rationale that would have someone re-derive the same dead end. Records `fresh`, i.e. AFTER exclude_rule_ids. A rule the session already holds was considered and not shown; counting it would inflate the denominator with claims the agent never saw, and the ratio would then fall for a reason that has nothing to do with whether hints land. PULLED — two doors, both after their access check so a refused read is not a pull. mcp_get_rule is the one that matters: the arm's own message ends "Read it with get_rule(N)", so that call is the exact action a landed hint produces. rest_rule carries the other prefix, and the prefix is load-bearing — "is this rule dead weight?" is served by any pull, "did that injected hint land?" by agent pulls only. NOT a pull: rule_history. It loads the rule for its title and its own output says "The current wording is on the rule itself — get_rule(N)", so counting it would credit a read of the history as a read of the rule and double-count anyone who then follows that pointer. list_always_on_rules and enter_project are likewise bulk resident loads, not somebody choosing to open one record. tests/test_rule_usage_wiring.py is cross-cutting on purpose: the surfaced end is in plugin_context, the pull end in two other modules, and "both ends meet" is a property no module-shaped file asserts. It covers the exclusion boundary, that a failing recorder cannot break the write, that a refused read records nothing, and two completeness guards — every door records, and the bulk loaders still do not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN	2026-09-02 17:15:03 -04:00