40189147d2bb3f85cc6895c7b38b6580b7188ec4
51
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
40189147d2 |
fix(rules): a shortened rule line must not decide what it says about holding (#3851)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m24s
CI & Build / integration (push) Successful in 40s
CI & Build / Build & push image (push) Successful in 26s
CI run 6485 was red. Six failures, three causes, and only one of them was a stale test. THE REAL DEFECT. The compact branch dropped the `seen` TAIL along with the trigger, so a rule the session had already been told rendered exactly like one it had not. #3750's whole argument is that those are different claims — a repeat is rendered precisely because the session may no longer HOLD what it was told — and the tail is the entire difference a reader can act on. test_a_rule_the_session_already_holds_is_referenced_not_re_offered caught it within one commit, which is that guard working as intended. Fixed by keeping the tail and dropping only the trigger, which is both the cheaper and the safer cut: a trigger runs 300-400 characters after #3855, a tail about 100. Re-measured on the real renderer — top-full-plus-references is ~299 tokens against ~568 for five full lines, so about 2x the old single line rather than the 1.4x claimed before, for four more rules and no lost information. The comments carrying the old figure are corrected rather than left to read as a decision nobody made. THE FIXTURE THAT STRADDLED THE BAND. `_THREE_HITS` spanned 0.81-0.74 against a 0.05 band, so the act arms dropped its lowest hit and four cases of test_both_recorders_report_the_same_rules_for_one_call failed reporting a count mismatch — under a message blaming the exclusion filter. A guard pointing confidently at the wrong subsystem costs more than no guard, because it is believed. Scores retightened to 0.81/0.80/0.79 and the precondition is now asserted by a named test, so a future band change is told where the problem is instead of through four confusing failures. THE STALE CONSTANT GUARD. test_the_rule_arm_asks_for_one_rule_not_two pinned RULEHINT_LIMIT == 1 — a real decision, correctly guarded, for a world with a resident set. Rewritten to pin what replaced it, as relationships rather than values (rule 115): the arm can return several, and rules are narrowed HARDER than the notes menu because they measured flatter, not sharper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
10343a6019 |
feat(rules): an act surfaces a banded SET of rules, quieter after the first (#3851)
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 53s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 45s
RULEHINT_LIMIT was 1. That was correct while retrieval SUPPLEMENTED a 33-rule resident set — one salient rule beside everything already loaded. Milestone 394 removes residency, and then this arm is the whole delivery: `git push origin dev` is governed by rules 1, 2, 9 and 140 simultaneously, and each alone permits the mistake the others catch. A cap plus a band, not a bigger cap. The old argument's real content is that a fixed k invents lines — it fills slots whether or not anything deserves them. A band keeps only what scored close to the top, so one clearly relevant rule still shows one and four competing rules show four. The corpus decides; the cap is a ceiling on the worst case, not the usual answer. MEASURED, AND IT CORRECTED THE PREDICTION. The expectation was that rules would rank sharply, since rule_document() shapes them like snippets and note 2485 measured snippets separating their top hit by 0.153 against 0.010-0.023 for every other kind. Three probes against real act queries say otherwise: `git push origin dev` top 0.757, gap 0.022 `docker compose up -d` top 0.685, gap 0.016 a bare-owner-filter query top 0.656, gap 0.020 Dev-log territory, not snippet territory — rules arrive as a packed block, so shaping alone did not buy separation. The band is therefore narrow: at 0.10 (the notes menu's value) every one of the top eight on the push probe falls inside, including a CI-registry rule and another project's branch policy. 0.05 admits about three ranks. COST, MEASURED RATHER THAN ASSUMED. The old comment claimed a line costs ~40 tokens. It is ~143 once the trigger is rendered, and #3855 roughly tripled trigger lengths, so five full lines are ~646 tokens before EVERY Bash call. Hence rank decides volume: the top hit keeps the full rendering, later hits are cited (~198 tokens total, 1.4x the old single line, for four more rules). The old paragraph's instinct — a fourth voice at full volume is where a reader stops reading — is answered by making later lines quieter rather than by refusing to have them. Band before dedup, deliberately. The band is a statement about scores; letting the ledger reorder it would make "you were told this already" change what counts as relevant. Same axis independence the renderer already keeps between `kind` and `seen`, and `rule_ids` stays fresh-only (#3752) so #3668's identity between logged results and surfacing rows survives. `suppressed` now covers both causes and says so. Both act arms take the same band: their score distributions are the same shape, and only the query differs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
d5f96563fd |
feat(rules): a slot a preference cannot lose (#3894)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 1m1s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Successful in 1m44s
CI & Build / Build & push image (push) Successful in 35s
Milestone 399 step 4. A rule and a preference are not equally served by one
ranking, because their losses are not equal:
- a RULE crowded out at the prompt boundary still fires at an act arm. A
push reaches pre_tool_rule, a write reaches write_path_rule. The prompt
hit is a preview of a second chance.
- a PREFERENCE about how to answer has no second chance. The response IS
the act, so crowded out there it is never delivered at all.
A straight ranking therefore favours the record whose loss is recoverable
over the one whose loss is total, and does it INVISIBLY: the rule that won is
a legitimate hit, the telemetry reads healthy, and the only symptom is a
preference that quietly never arrives. reuse_slot exists for the same shape
one corpus over (#2463).
`semantic_search_rules` gains a `kind` filter, so the slot's query can only
answer with what the slot is for. Verifying afterwards would be weaker — an
unfiltered search that happened to return a rule would spend the slot on it,
and that line would be indistinguishable from one that earned its place.
THE SLOT BUYS POSITION, NOT A LOWER BAR, matching reuse_slot. A weak
preference cannot buy it, so silence stays the default. The task asked for a
separate threshold; I did not add one, and the reason is that the worry
behind it — reading a miss rate as a fact about preferences — is answered by
`preference_slot` being its own logged source, where best_available_id names
which preference was refused. A knob added on a guess is a way to
misconfigure the surface; a bar moved on evidence is an argument. The
evidence arrives on its own now.
IT EXTENDS, IT NEVER DISPLACES — and here it parts from reuse_slot, which
evicts its menu's weakest hit. A displaced hit sits in prompt_rule's
retrieval_logs row while never being surfaced, so that source's two tables
stop agreeing and #3668's identity breaks for a reason nothing in the data
explains. Milestone #379 is what losing that identity costs: five steps
planned against two counters disagreeing, not a write path dropping rows. One
extra line in a rare case is the cheaper price.
It also runs BEFORE the bail-out. An empty general result is not proof no
preference qualifies: that search overfetches by distance then collapses, so
a preference ranked below the window is invisible to it while a kind-filtered
query finds it at once. Bailing first would make the slot dead in exactly the
corpus it exists for.
One existing assertion repinned from a bare call_count to a per-source
filter: the slot logs its own query on the same call, and a count would pin
the number of arms rather than the property — going red the next time one is
added, which is rule 167's false alarm about the thing it protects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
|
||
|
|
44e0b0541f |
feat(rules): rules retrieve against the operator's message (#3852)
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Successful in 1m35s
CI & Build / Build & push image (push) Successful in 38s
The third rule arm, and the one the other two cannot reach. `write_path_rule` is keyed on code, `pre_tool_rule` on a command — both things the session is about to DO. A rule that governs what to SAY has no such trigger: extract intent from loose phrasing, raise a conflict before acting, hand off an action with its reason, end a finding with an offer all bind on a RESPONSE, and no tool call precedes one. The operator's message is the only query that exists before a response is composed. That hook searched notes alone, so no rule had ever been retrieved against a thing the operator actually said — and residency was the only surface those rules had, which is what milestone 394 removes. A SEPARATE FUNCTION, not a branch in build_autoinject_hint, because of its early returns. That arm bails when auto-inject is disabled, when the query is blank, when nothing clears the note bar — every one a statement about NOTES. Folded in, an operator who turned the awareness menu off would silently lose their rules, a coupling with no symptom since both look like a quiet hook. Two functions, two sets of gates, composed in the route. Guarded as "the rule arm never asks the notes arm's config", which is the structural fact. Joins _ARMS rather than getting its own test file. #3497's history is that the pre-tool arm inherited a defect from its sibling by being MODELLED on it instead of sharing with it, and a third arm modelled on two is two chances to repeat that. Repeat rendering, fresh-only counting, log-before-bailout, the kind register and the two-recorders identity are properties of every arm or of none. The bar is INHERITED and says so. 0.72 was tuned against code and commands; prose is a different query shape against the same documents, and triggers are written in the vocabulary of the moment — which for most rules is act vocabulary. Starting at the only number with evidence behind it and logging every call from the first deploy is what makes it settleable; guessing lower would put an unmeasured bar in front of a corpus that binds. k=3, anchored on this hook's own budget rather than the act arms'. RULEHINT_LIMIT is 1 because that arm fires before every Bash call; this one fires once per turn, beside a notes menu already spending three slots. And a prompt genuinely contains more than one act — "merge to main and then start on X" is two — where a command is one thing. `prompt_rule` added to RANKED_SOURCES: a ranker picked it, and a ranked source missing from that tuple is silently counted as bulk delivery and drops out of the pull-through denominator. The hook reads and writes the SHARED rule ledger under scribe-priorart, not a private one — one session keeps one list, aged (#3751), so a rule named here is not re-announced before the next Bash call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
26e0dff706 |
feat(rules): a preference does not speak in a rule's voice (#3849 step 3)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m27s
CI & Build / integration (push) Successful in 1m31s
CI & Build / Build & push image (push) Successful in 41s
Two independent axes on one hint line. `kind` decides the head, `seen` decides the tail, and neither reads the other — which is what let a second kind arrive without reopening #3750's repeat question. Whether a record is already on the exclusion ledger has nothing to do with how much force it carries, so the seen branch is shared verbatim. The noun carries the whole visual difference, deliberately. A reader skimming an injected block gets one word to place the register, so the word that moves is the one naming force: "Standing rule" / "Preference". Everything structural after it is identical, so the kinds read as one set rather than two formats. Force is asserted in exactly one other place, and that moves too. A rule's line says to read it BEFORE DECIDING IT DOES NOT APPLY, because dismissing a rule unread is how the thing it prevents happens. A preference makes no such claim: it says where to find HOW THIS HAS BEEN DONE BEFORE, and following it buys consistency rather than correctness. Guarded on both places at once. Pinning the noun alone would pass a line reading "Preference … before deciding it does not apply" — label swapped, instruction kept — which is worse than not distinguishing them, because it looks handled. And a guard on the independence claim itself, exercising all four combinations: the way this breaks silently is a seen branch that grows a kind test, leaving one combination rendered by nobody's intention. Noted, not fixed: plugin/skills/using-scribe/SKILL.md still says "Standing rules are binding" with no room for a kind that does not. That surface is step 6's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
c4908f093f |
feat(rules): a repeat is referenced, not withheld (#3750, #3752)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 40s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 37s
Both arms used to drop a hit already on the session's exclusion ledger and emit nothing. That is correct only while the session still HOLDS what it was told, and a compaction breaks exactly that: the earlier injection is summarized away while the id stays on the ledger, leaving the rule absent from context AND unreachable for the rest of the session. #3749 closed the compaction half by clearing the ledger; this closes the ordinary half, where a session simply stops holding a line it read an hour ago. Only ONE CLAUSE of the existing line is false on a repeat — the claim that the rule is not in the session's loaded set — so only that clause changes. The fresh line is byte-identical to what it was. Both tails now come from one `_rule_hint_line`. The arms phrase their heads differently on purpose; everything after must not differ, and #3497's history is that the pre-tool arm inherited a defect by being modelled on its sibling rather than sharing with it. THE BUDGET DECISION, recorded at RULEHINT_LIMIT. A repeat competes for the single slot on rank alone: nothing is fetched behind it, and it never rides alongside as a second line. Promoting a fresh rule past a better-ranked repeat would reinstate the withholding one rank deeper, and a second line is the one thing the limit exists to forbid. The consequence is deliberate — a rule that keeps ranking first for a recurring situation keeps being referenced, and its decay belongs to exclusion ageing (#3751), not to a first-place rule being demoted for having won before. THE TELEMETRY, decided before shipping rather than after a number moved (#3752): nothing changes. A reference is a RENDERING decision, not a retrieval outcome. `results` stays `fresh`, `suppressed` stays len(hits) - len(fresh), and `record_rule_surfaced` still counts only what the arm freshly chose. This matters more than it reads: the naive implementation drops the `fresh` filter and takes suppressed_count to zero everywhere — and #3739's near-miss fix identifies repeat-caused zeros by `suppressed_count > 0`, so the contamination corrected on 2026-09-08 would return by a different route, in the same field, with the fix still in the code and no longer working. A test asserts the counters as unmoved, because "nothing changed" is only worth something if it is checkable. Also corrects two comments that outlived #3702 — both arms still claimed CONDITIONAL ONLY while the module-level note above RULEHINT_LIMIT said the opposite, in the exact code this change edits. GUARDS - an already-held hit produces a rule line at all (the regression), asserted on `get_rule(<id>)` rather than a truthy context: the write-path arm fills its context from four other sources, so truthiness passes under the OLD behaviour and pins nothing there. - the two tails are distinguishable, each excludes the other, and neither injects the rule statement — the budget claim, both branches. - the counters are unmoved, per #3752. - test_a_rule_the_session_already_holds_is_not_re_offered asserted the old contract (`"161" not in context`). Repinned rather than deleted: the telemetry half of what it protected still holds. Repinned the #3497 log-placement guard on structure (rule 167). It read `body.index("if not fresh:")` — a local variable NAME, not the property. This change renames that guard to `if not hits:`, so the old assertion would have raised ValueError and reported #3497 as back while the arm was entirely correct. Now walks the AST for the first early return after the search and asserts the call row is written before it. Falsified both ways: it fails on the #3497 mutation, and it refuses to pass when no early return exists at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
d5ac8408f6 |
feat(telemetry): record WHAT the bar turned away, not only how close it came (#3807)
CI & Build / Python lint (push) Successful in 8s
CI & Build / Plugin hooks (push) Successful in 16s
CI & Build / integration (push) Successful in 40s
CI & Build / TypeScript typecheck (push) Successful in 43s
CI & Build / Python tests (push) Successful in 1m14s
CI & Build / Build & push image (push) Successful in 2m59s
#3670 added `best_available_score` so a threshold could be judged from its rejections. It records how CLOSE the bar came to firing and not WHAT it refused, and that is the half a decision actually needs. Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 — about 117 declines a day within 0.013 of firing. Dropping to 0.707 would take that arm from 22 hits a day to roughly 139: six-fold, on a surface that runs before every Bash call. The percentile says the mass is there. Nothing said whether it was worth showing. NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected rule line already carries title and trigger, so a session can comply without ever calling get_rule, and rule pull-through understates usefulness by construction. Reading the rejected records can — and `result_ids` holds only what was RETURNED, so on a zero-result call the near-missed record had no name at all. So the id, from the SAME ranked candidate as the score. Both searches unpack `best` once and read both fields off it, because splitting that into two expressions is exactly how a later edit pairs a score with its neighbour's id — and a score attached to the wrong record is worse than no id, since it invites judging the wrong one and concluding the bar is fine. write_path withholds the id on the same condition it withholds the score (#3739): a surviving id beside a null score names a record without saying what it scored, the pair disagreeing in the other direction. THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled, and a reader tuning a bar needs to go and read the records. Opt-in via `near_miss_samples` (0-20, default 0) so the ordinary readout keeps its size, and deliberately NOT a window function: this module's one production outage was a grouped query Postgres rejected, swallowed by the broad except, every counter reading zero while the mocked tests passed (#2663). One flat ordered query, overfetched, bucketed in Python — the shape that lesson prescribes. Migration 0097, nullable and unbackfilled. Not a foreign key: the table spans record types and `source` says which, exactly as result_ids works. The integration guard pins the listing as PER SOURCE. A global LIMIT would let a noisy source eat the whole quota and leave the surface being tuned showing nothing — which reads as "nothing was close", the misreading this milestone has spent itself correcting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
623464323e |
fix(telemetry): a search that never ran is not a decline (#3765)
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 46s
CI & Build / integration (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m21s
CI & Build / Build & push image (push) Successful in 36s
`best_available_score` was added by #3670 so a bar could be judged from what it rejected, and it arrived null on four unrelated causes: the corpus offered nothing, the query was empty, the embedder was down, or the DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the #2663 shape — a swallowed failure rendering as a clean zero — inside the field added to fix an instance of the #2663 shape. Found while trying to explain why reuse_slot returned nothing on 45 of 45 calls, and auto_inject on 153 of 161. That investigation is still open; what it established first is that the readout could not answer it. THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so every remaining null means one thing: searched, and nothing came close. That is the convention the pre-tool arm already follows for a blank command — "a row here would report a call that never happened and drag the clear-rate down with phantom declines" — extended from the case a caller can see in advance to the ones only the search knows about. Both searches stamp `report["searched"]` FALSE before anything can return and True only where a real result set exists, so every early return leaves it false. It has to be the first thing done to the dict: a return added above that line would leave the key absent. ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no report cannot know, and the safe reading there is the old behaviour. Only a real search can report False, so absent means "nobody asked" and never "it failed" — which is also why 66 existing mocked searches across twelve test files keep working unchanged rather than being rewritten to simulate a flag they do not care about. A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a WARNING on a query failure, which is where a broken search belongs: a counter cannot say "I am broken" without a reader already trusting it. Three tests, and the middle one is what makes them discriminate — a blanket `return` passes the first and fails the second, because a call that searched and came back empty is the only evidence a threshold is too high (#3497). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
277f5df515 |
fix(telemetry): write_path reported records it withheld itself as near misses (#3739)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 24s
Found by verifying the previous fix on live data — the check that fix was
meant to make possible.
write_path near_misses.max 0.822 p90 0.7521
top_score.min 0.6857 so the bar is at or below this
A "rejection" that outscored every acceptance, and not one outlier: the p90
is above the bar too.
#3739's fix keyed on `suppressed_count`, and I justified its NULL branch as
"null means the caller passed its exclusions INTO the search, so the score
is already post-exclusion". That holds for auto_inject and reuse_slot, both
of which log the RAW search output and do their Python filtering after. It
does not hold for write_path, the one note arm that filters TWICE:
`exclude_ids` takes `seen - pulled_seen` into the search, but the
pulled-and-seen ids stay in the query on purpose — the arm's query doubles
as the resemblance test — and are dropped afterwards in Python. So the row
carries a POST-filter count beside a PRE-filter score.
The suppression column cannot rescue it the way it does for the rule arms.
This arm's count would be PARTIAL — covering the drops made here and not
the ones `exclude_ids` made inside the search — and a partial number under
a name that reads as complete is the substitution this milestone exists to
stop.
So it reports null whenever its own filter removed anything: not measured
on this call, because the bar was not the only thing that turned something
away. Calls that withheld nothing keep reporting, which is most of them.
Both directions are asserted. Without the second test, setting the field to
null unconditionally would pass the first while deleting the measurement
#3670 was built for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
|
||
|
|
e7c1af32a0 |
fix(telemetry): the bar can only be judged from what it rejected (#3670)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 28s
`cleared_threshold` was documented as the number to read first. It was a tautology. The search applies the threshold before returning, so every returned result cleared it by construction and a call with no results has no top_score to compare — the condition was true exactly when `result_count > 0`. It was `calls - zero_result_calls` under a name that promised a second opinion, and `zero + cleared == calls` held on all nineteen source/window readings ever taken, today's live seven included. The reading procedure built on it asked the reader to compare a number with itself, and a threshold change was unobservable through it: raise the bar and both numbers move together, so the field could never show a bar set too high. REPLACED, NOT JUST REMOVED. The question the table exists to answer is whether the bar is in the right place, and that is only answerable from the calls that returned NOTHING: how close did the best rejected candidate come? A 0.72 bar turning away a stream of 0.71s is set too high by a hair; the same bar turning away 0.30s is working. Both render as a zero-result call today and nothing separates them, because the losing score is discarded inside the search. So both searches now rank WITHOUT the bar and apply it in Python. The qualifying set is provably identical — rows arrive ordered by distance, so every above-bar row sorts ahead of every below-bar one, and an over-fetch that returned N above-bar rows returns the same N plus some losers. What changes is that the losers are visible instead of dropped in the query. `report` carries the score out without changing what a search RETURNS: eight of eleven call sites want hits and nothing else. New column (migration 0096), nullable and unbackfilled. A row written before this genuinely does not know, and a 0.0 would read as "the corpus held nothing remotely relevant" — a claim invented out of a caller's silence, which is the substitution this whole milestone corrects. The new aggregate is a percentile_cont WITHIN GROUP over a CASE, one step from the shape that produced #2663, where a rejected query was swallowed by the broad except and every counter read zero. It carries an integration guard for that reason: only real Postgres can say it parses, and the symptom of failure is silence. Also adds a guard that no int field in a bucket equals `calls - zero_result_calls`. That identity is what `cleared_threshold` satisfied for its whole life, and it survived because it had its own name and nobody added the two numbers beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
277aea58e4 |
test(telemetry): pin the identity that falsified this milestone (#3668)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 27s
CI & Build / Build & push image (push) Successful in 23s
Reverts the deliberate break from |
||
|
|
5e19a1b028 |
test(telemetry): FALSIFICATION — prove the identity guard can fail (#3668)
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 34s
CI & Build / Python tests (push) Failing after 45s
CI & Build / Build & push image (push) Skipped
Deliberately broken, reverted in the next commit. Rule 167 requires every guard be falsified against the regression it names before it is trusted, and rule 10 puts CI as the only place that can run it — so the failure has to be made to happen here rather than argued for. The guard: both rule arms feed one `fresh` list to two recorders, so the call log and the surfacing log cannot disagree about what one call showed. The regression: `results=fresh` becomes `results=hits` in both arms, so the call log counts what the ranker found while the surfacing log counts what was shown. That is not a hypothetical shape. It is exactly the divergence that would make a correct system report a lost write when the two tables are later compared in aggregate — the reading that scoped this milestone at five steps against a defect that did not exist. Expected red: the one-already-held and all-already-held cases on both arms. The nothing-held case must still PASS — with no exclusions both recorders see the same list however wrongly they are wired, which is why it could never have been the discriminating case. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
dd1e6e2645 |
feat(rules): the rule arms stop filtering the corpus to one tier (#3702)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 1m3s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 1m29s
Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:
PRESENT IN CONTEXT the rule was delivered at session start
SALIENT AT THE MOMENT the rule is in front of the reader when the action
it governs is about to be taken
A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.
Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.
MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.
Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.
The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
|
||
|
|
8be555d6dd |
feat(telemetry): tell a ranker decline from a repeat before the observation window opens (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 28s
Making the rule arms log every call exposed a second ambiguity in the same
row. `result_count == 0` is two unrelated events wearing one number:
- the ranker found nothing above the bar — the only evidence a threshold is
set too high; and
- the ranker found only what this session had already been shown — which
says nothing whatever about the bar.
A long session excludes its way into the second, so the arm reads worse the
longer it runs correctly. Rows written now carry the ambiguity permanently,
which is why this lands before any watch period rather than after.
`retrieval_logs.suppressed_count` (0095, nullable) holds what the caller
dropped as already-shown. Both rule arms report it; they filter in Python and
always know. The note arms pass exclusions INTO semantic_search_notes and
never see what was dropped, so they store NULL.
THE NULL IS LOAD-BEARING. It means "not measured here", and the readout
renders it as `suppression: null` rather than a zeroed dict. Defaulting to 0
would let an unmeasured surface read as a perfectly clean one — the same
substitution of an artifact for a measurement that #3311 made. No backfill,
for the same reason: existing rows genuinely do not know.
`retrieval_telemetry`'s `sources` gains `suppression` with `measured_calls`,
`calls_with_suppression` and `zero_because_already_shown`; subtract the last
from `zero_result_calls` for the true ranker declines. The MCP tool docstring
says to read the two together and warns against reading the null as a zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
|
||
|
|
154a5de13e |
fix(telemetry): both rule arms logged only their hits, so the clear-rate could only read 100% (#3497)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Failing after 48s
CI & Build / Build & push image (push) Skipped
`write_path_rule` reported `zero_result_calls: 0` and `cleared_threshold: 133/133` — a perfect record no other surface comes near (`write_path` 421 zeroes of 613, `reuse_slot` 124/199, `auto_inject` 114/326). #3311 read that as a measurement and milestone 333 was scoped on it. It was an artifact. Both arms called `record_retrieval` inside a guard on having results — the write-path arm behind `if fresh:`, the pre-tool arm below `if not fresh: return out` — so a call that found nothing wrote no row. The statistic was a fact about the shape of the code, true at any threshold whatsoever. The call log moves out of the guard in both arms. The surfacing log stays in it: nothing was shown, so no surfacing occurred. `results=fresh` is kept deliberately — the note arms pass exclusions into `semantic_search_notes`, so what they log is already post-exclusion, and logging `hits` here would make this row mean something other than every other row in the same readout. The defect bites hardest on the pre-tool arm, which fires on every Bash call: with no rows at all, a ranker that declined is indistinguishable from a hook that never fired — the silent failure the arm exists to stop. Tests cover both arms behaviourally (found nothing; found only what the session already held; searched nothing at all, which must stay silent) plus a structural guard, because this was one level of indentation and it appeared independently in two places. #3311 and the `rule_usage` docstring corrected rather than quietly rewritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
2ee24b9d2b |
feat(rules): rules before tools — a PreToolUse arm keyed on the action (#3476)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Successful in 1m9s
CI & Build / Build & push image (push) Successful in 25s
The only just-in-time rule surface was registered on `Write|Edit` and queried with `code or path`, so a rule could be retrieved at the moment of a code write and nowhere else. Every rule about which tool to reach for — don't curl the forge, don't stand up a stack, don't run the suite locally, don't branch — was unreachable exactly when it mattered, and residency in the always-on preload was the only surface it had. That is the pressure that grew the resident set to 31 against #3089's ceiling of ~23; it was never a judgment anybody made. A reflex generates no query, so an instruction to check the rules cannot catch one. A mechanical trigger can: the tool call IS the query, and a reflex has to become a tool call before it can do anything. `build_tool_rule_hint` is deliberately tool-agnostic — a name and a string — so widening the matcher later is a hooks.json edit with no server change. The hook starts on Bash, which is where the action reflexes live. The two pre-tool arms share ONE session ledger of already-named rules (`<state>/<sid>.rules.ids`). Two ledgers would mean a rule named by one arm gets re-offered by the other, and the hint that fires most often is exactly the one that must not repeat itself. A test asserts both scripts build the same path, and another checks the shell hook and the Python route agree on every query-arg name (rule 33) — a rename there fails silently, looking like a surface that never finds anything rather than a broken one. Deliberately silent on outage, unlike the prior-art hook: a write is occasional, a Bash call is not, and an outage line before every command is what gets a channel muted. `tier="conditional"` matches the write arm and is the transition point — an always-on rule is already resident, so re-tier one and it starts arriving here instead of in every session's preamble. `pre_tool_rule` joins RANKED_SOURCES: this arm chose what it showed, so a pull can settle whether the choice landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
8b9b3a1d9b |
feat(telemetry): the preload emits, and the always-on set stops being unfalsifiable (#3473)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 23s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 27s
The ranked rule arm became measurable in M333. The preload did not — and that is the surface whose value is actually in question. `list_always_on_rules`, the SessionStart block and every `rules_payload` caller handed rules over wholesale and emitted nothing, so the resident set's token cost was certain and its usefulness could not be tested even in principle. Bulk deliveries now record as AMBIENT, beside the ranked count and never inside pull-through. Folding them in would mean growing the always-on set depressed the arm's measured precision and trimming it flattered the arm, neither for any reason to do with the arm. `RANKED_SOURCES` inverts the note twin's `AMBIENT_SOURCES` deliberately: there is one ranked rule source and this change adds seven bulk ones, so naming the rare half makes a forgotten surface default to ambient — under-counting it — rather than padding the denominator with surfacings nobody chose. Two lookalike call sites are deliberately left silent, with a test to keep them that way: the write-path etag arm and `rules_etag_for` read the rules to build or compare a MARKER and show nobody anything. No migration — `event` and `source` are plain Text with no CHECK (rule 36 does not apply). Snippet #2858 updated to the new `rules_payload` contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ |
||
|
|
238510080e |
feat(retrieval): the standing-rule arm gets its own bar, and asks for one rule not two (#3318)
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 31s
CI & Build / Python tests (push) Successful in 1m4s
CI & Build / Build & push image (push) Successful in 35s
Milestone 333 step 4 — the split #2223 made one surface down, now made for the third corpus. The arm inherited WRITEPATH_DEFAULT_THRESHOLD = 0.68, a number measured against code-vs-note-PROSE and never re-derived for code-vs-RULE-TEXT. THE DEFAULT IS ARGUED STRUCTURALLY, NOT READ OFF A HISTOGRAM (rule 115). Two facts hold on any install, including one with six rules and no telemetry: - The eligible corpus is tiny — conditional rules only, a handful to a few dozen against thousands of notes. A top-k over forty candidates always returns something, so "the best match cleared the bar" stops meaning "a good match exists". A bar calibrated for best-of-thousands is cleared by best-of-forty as arithmetic, not relevance. - Rules are short imperative technical English, far more homogeneous than note prose. #2223 put the code-vs-prose floor at 0.55-0.63 and set 0.68 above it; a more homogeneous corpus has a HIGHER floor, so 0.68 is not merely inherited, it sits below where this corpus's noise lives. 0.72 errs deliberately toward silence on an asymmetry that is also structural: this hint fires on EVERY write. A missed rule is recoverable — it is still in Scribe and the agent can search it. A hint that cries wolf is not: it teaches the reader to skip the whole block, and the true positives go with it. The arm's own comment already said "noise on a hint that fires on every write is how a hint gets ignored". Pinned as an INEQUALITY, not a value: test_the_rule_bar_defaults_above_the_code_bar asserts RULEHINT > WRITEPATH, so tuning the number stays free while inverting the relationship — which would silently reinstate #3311 — does not. RULEHINT_LIMIT = 1, and deliberately not a knob. With a corpus this small, k=2 means the second line is almost always the second-best noise wearing the same confident framing as the first; halving k halves that regardless of the bar. It stays a constant because it is a decision about how loud one hint may be, not a per-install tuning question — and a knob nobody turns only adds a way to misconfigure the surface. Reachable from Settings, no restart (rule 25), with copy that says which way to move it and points at retrieval_telemetry's rule pull-through — which step 3 made readable — to tell "arriving unread" from "never arrived". Every config stand-in in the suite gained the key, not just the one that noticed. The arm reads `rule_threshold` while BUILDING its search arguments, so a missing key raises inside its fail-open except and turns the arm into a silent no-op — indistinguishable from it running and finding nothing. That is the same vacuous-pass shape that bit step 2, one layer down (rule 33). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN |
||
|
|
8901c904a9 |
feat(telemetry): retrieval_telemetry reports rule pull-through where it reported nothing (#3317)
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / integration (push) Successful in 32s
CI & Build / Build & push image (push) Successful in 29s
Milestone 333 step 3, the read half. Steps 1 and 2 built the table and filled it; until now nothing read it, and `usage` — sourced entirely from note_usage_events — described notes only while `sources` happily listed a write_path_rule row above it. A reader takes the aggregate as covering everything named above it. It did not. A SEPARATE `rule_usage` BLOCK, not folded into `usage`. Two reasons, and the second is the one that bites: the corpora differ by orders of magnitude, so a blended ratio would be the note ratio with noise on it and the rule arm would stay invisible inside it; and `usage` is what existing callers already read and compare across windows, so silently changing what it counts would move a number nobody was told had changed meaning. There is a test asserting rule events stay out of the note block. No `ambient` key, unlike the twin. Nothing surfaces a rule un-ranked — list_always_on_rules and enter_project hand rules over wholesale but emit no event — so there is no ambient class to subtract. The absence is a fact about the data, not an oversight, and it returns when a bulk loader starts emitting. Guarded separately, like `by_source`. This table did not exist a commit ago, and an instance running upgraded code against un-migrated schema would otherwise take down two readouts that work perfectly in order to report a third that cannot. On failure the FLAG is added and the SHAPE is kept — a caller must not have to choose between crashing on a missing key and quietly rendering zeros it has no right to. `pull_through` is None rather than 0.0 on an empty window, matching the note block. A ratio of zero asserts "rules were shown and none opened"; with an empty numerator and denominator that is a claim the data does not support, and it is the reading that would make a brand-new install look like a broken one. Also fixed, from #3311: the rule arm never timed its search, so it was the one source in the readout reporting a null p90_duration_ms — a gap that reads as "this surface is somehow not measurable" rather than "nobody passed the number". Both docstrings updated in the same change. The tool's is the agent-facing contract (rule 119) and it explicitly said rule surfacings were absent and had "no usage counter at all". Leaving that would have had a reader conclude the arm has zero pull-through rather than a separate one. Tests are integration for the reason the block above them is: real GROUP BYs and count(distinct) against a table a commit old, in a module whose one production outage was a SQL shape the database rejected inside a broad except. A mock would agree with whatever the code does, including nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN |
||
|
|
8f7f447fda |
feat(telemetry): the rule arm records what it showed, and get_rule records the read (#3316)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 37s
CI & Build / Python tests (push) Failing after 53s
CI & Build / Build & push image (push) Skipped
Milestone 333 step 2. Step 1 built the table; a counter nobody calls reads zero and looks exactly like a surface nobody uses, which is #2663's shape. SURFACED — the standing-rule arm in build_write_path_hint, beside the record_retrieval it already made. Two tables, and the split is not arbitrary: retrieval_logs is one row per CALL keyed on the score distribution a threshold is tuned from; rule_usage_events is one row per RULE per event, the grain "was this hint ever acted on" needs and the grain a JSONB result_ids array cannot be indexed at. The comment there said rule ids had nowhere to go — that note_usage_events remaps ids on restore, so a rule id would return attached to whatever note took that number. Still true of the NOTE table, and precisely why step 1 built its own. Rewritten to say the gap is closed rather than leaving a stale rationale that would have someone re-derive the same dead end. Records `fresh`, i.e. AFTER exclude_rule_ids. A rule the session already holds was considered and not shown; counting it would inflate the denominator with claims the agent never saw, and the ratio would then fall for a reason that has nothing to do with whether hints land. PULLED — two doors, both after their access check so a refused read is not a pull. mcp_get_rule is the one that matters: the arm's own message ends "Read it with get_rule(N)", so that call is the exact action a landed hint produces. rest_rule carries the other prefix, and the prefix is load-bearing — "is this rule dead weight?" is served by any pull, "did that injected hint land?" by agent pulls only. NOT a pull: rule_history. It loads the rule for its title and its own output says "The current wording is on the rule itself — get_rule(N)", so counting it would credit a read of the history as a read of the rule and double-count anyone who then follows that pointer. list_always_on_rules and enter_project are likewise bulk resident loads, not somebody choosing to open one record. tests/test_rule_usage_wiring.py is cross-cutting on purpose: the surfaced end is in plugin_context, the pull end in two other modules, and "both ends meet" is a property no module-shaped file asserts. It covers the exclusion boundary, that a failing recorder cannot break the write, that a refused read records nothing, and two completeness guards — every door records, and the bulk loaders still do not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TcCs1CcQ1ormdnzSshKqvN |
||
|
|
efabba58dd |
fix(rules): the staleness signal must not wait for prior art to match (#3244)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Failing after 52s
CI & Build / Build & push image (push) Skipped
CI caught two things, and the first is the feature not working rather than a test being wrong. THE SIGNAL WAS GATED ON A COINCIDENCE. build_write_path_hint returns early when no prior art, stamp, divergence or derive matched, and that guard sat ABOVE the new arm — so a session whose rules had changed was told only if the file it happened to be editing also matched something else. A staleness signal that fires on that coincidence is not a staleness signal. The arm now runs above the guard, collecting into its own list that `lines` is seeded from, and the guard accounts for it. The standing-rule arm (milestone 307) is deliberately LEFT below that guard, and this is a finding rather than a fix: it has the same gating and probably should not, but it runs a SEMANTIC search, so lifting it would put an embedding query on every write in every session. That is a cost decision, not a bug fix, and not this task's to make. THE MARKER MUST NOT BREAK THE PAYLOAD IT DECORATES. rules_etag is computed on the SessionStart path, where `max()` raising costs the whole context payload — every rule title, the project, all of it — to save a hint. A row with no usable timestamp is now skipped and a set with none degrades to a count-only marker, which still catches a rule added or deleted and only loses edits. That is the right way round to lose information. CI found it because build_session_context's tests pass MagicMock rules and `max()` over those raises TypeError. Also: list_always_on_rules on an install with no always-on rulebooks returns `rules_etag: "empty|0"`. Its exact-dict test is updated rather than loosened — the key being present on an empty install is the behaviour, not noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
5c9bb40777 |
feat(rules): a session is told when its rules move under it (#3244, milestone 323 step 5)
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / integration (push) Successful in 32s
CI & Build / Python lint (push) Successful in 6s
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python tests (push) Failing after 1m0s
CI & Build / Build & push image (push) Skipped
The rules payload carries a marker; the write-path hook hands it back; the server says which rules moved. Nothing is said when nothing moved. THE COUNT IS NOT DECORATION. max(updated_at) alone cannot see a DELETED rule — it moves no timestamp — and that is the single change that takes an instruction OUT of force, which is the one a session most needs to hear about. The marker is `<max updated_at>|<count>`, and a deletion is reported through the count because there is no row left to name. THE HOOK IS THE CARRIER because it already fires before a write, which is the moment acting on a stale rule costs something. One comparison, no payload, no extra round trip. WHERE THE MARKER IS CAPTURED, and it could not be anywhere else: the SessionStart hook, from /api/plugin/context. The model also receives one from list_always_on_rules, but a hook cannot see an MCP tool's result — so the value the write path compares has to be stored where a shell script can reach it. Keyed by session id in the state dir the prior-art hook already uses, so "changed since" means since THIS session loaded its rules. NOT ON rules_payload, against the task's letter. Those are applicable_rules — a different, subscription-derived set. One key name over two sets is how a comparison starts reporting phantom changes, and the write path compares against the always-on set. WHAT IT CANNOT SEE is stated in both the service and the write-path arm as a table, because a reader who finds an etag will assume it covers staleness generally: another session edits a rule mid-flight | caught the session is misremembering a rule read hours ago | caught compaction summarised the rules out of context | NOT caught The third is the most common, and the marker is blind to it — the etag was in context too and went with the rules. The SessionStart nudge is that case's only mechanism and must not be softened because this shipped. A test asserts both modules still explain that. Instance-agnostic (rule 115): an install with no rules produces a stable marker rather than an error, and "no rules" reads as a state rather than as a change. An unreadable or absent marker reports nothing — a signal that cries wolf is worse than none, because it trains a reader to skip the line that will one day be true. The arm fails open like every other arm on this hook. The delivery is tested through the real build_write_path_hint rather than the helper alone: the feature IS a line arriving in a session, and the arithmetic being right proves nothing about that. Live acceptance is deploy-gated and not yet recorded on the task. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
02c1e37620 |
feat(rules): the write path can notice a standing rule it was never given (#3031, milestone 307 step 5, hook arm)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 26s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 24s
A conditional rule is not resident, so a session can be about to violate one it was never handed. This arm notices: when what is being written resembles a rule's trigger, the hint names it and says to read it before deciding it does not apply. A SUGGESTION, and the plan was wrong about why it could be more. It claimed the hook "already resolves a path to an area" — it does not, and nothing in Scribe maps a path to a System or a canonical area (build_write_path_hint resolves paths against snippet LOCATIONS, a different index; the learned-alias idea belongs to another project). Correction logged on the task. Rather than invent path→area inference to make a stale claim true, the arm does what D7 already decided and what this surface already IS: tags bind at enter_project, meaning suggests here. The header of the hook says NEVER BLOCKS; dressing a hint up as binding would have been the actual mistake. CONDITIONAL RULES ONLY. An always-on rule is already in the session, so re-offering it is noise — and noise on a hint that fires on every write is how a hint gets ignored. Telemetry goes to retrieval_logs, NOT note_usage_events, and that is a correctness call rather than a preference: note_usage ids are REMAPPED on a backup restore, so a rule id written there would come back attached to whatever note took that number — silently corrupting the evidence the next true-up is supposed to read. retrieval_logs is never restored and `source` already separates surfaces. record_retrieval's `results` type widened to match what it actually needs (an `.id`), instead of passing a Rule to something annotated Note. The rule dedup gets its OWN state file and query parameter, like the three channels before it — #2708's lesson was that one shared channel lets a hint of one class silence a different class that had never been shown. Plugin version bumped: a hook change clients cannot see did not ship (#1040). The stub is autouse in conftest rather than added to forty-odd call sites: the arm loads an embedding model, and every existing test that stubs the NOTES search would otherwise pull a real model in through the one arm it had no way to know about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
8664d8ad14 |
feat(ledger): the consumer map surfaces — every css row carries used_by on list_shapes, flag="unused-css" (the map's negative space, surfaced never deleted), derive groups and the write-path family carry consumers, the derive line says "used by N template(s)", coverage payload unused_css + the standing block; docs, SKILL, plugin 0.1.44; the two step-2 tests expected 5 edges where fan-out makes 6 (milestone 302 step 3, #2936)
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 38s
CI & Build / integration (push) Failing after 44s
CI & Build / Python tests (push) Successful in 1m26s
CI & Build / Build & push image (push) Successful in 31s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
85111442a6 |
feat(ledger): CSS derive families are names, never bodies — name floor 2 for css, dup: grouping sym-only; derive line says "repeated name" and dismisses scoped-css; plugin 0.1.41 (note #2917)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 25s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m0s
CI & Build / Build & push image (push) Successful in 25s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2324c15418 |
feat(write-path): the derive arm — the hint names a duplicate family (no canon) or a canon elsewhere for the shapes being written; exclude_derive channel (#2900, milestone 299 step 2)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Failing after 9s
CI & Build / integration (push) Successful in 25s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 23s
shape_ledger.write_time_derive asks the ledger what it knows about each named (kind, symbol): a derive-grouped family (identical body / same name in N other files) -> "derive it now, do not add a copy"; a canonical row at another path -> "canon #N at <path>, reuse". Judged rows at the path and the canon own file stay silent. Rendered by _derive_line beside the divergence line; keyed (group id / canon:<id>) on a third per-session dedup channel in the hook (.derive.ids -> exclude_derive=). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ff5f6438c4 |
feat(inception): always-on exclusions reach every rule surface — list_always_on_rules(project_id), get_applicable_rules, rules_payload.excluded_always_on, session context, exclude/include tools (#2880, milestone 297 step 2)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 36s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Failing after 1m12s
CI & Build / Build & push image (push) Skipped
A project that opted out of an always-on rulebook at inception must not see
it anywhere: list_always_on_rules(project_id=) and the subscription-derived
set skip it (an exclusion is total), get_applicable_rules / rules_payload
carry `excluded_always_on` as the seventh key so the departure is visible
wherever the rules are, and the SessionStart block built for a bound project
names it ("Excluded for this project by its inception decision …"). MCP:
list_always_on_rules takes project_id; exclude_always_on_rulebook /
include_always_on_rulebook mirror suppress/unsuppress (owner-only; the
rulebook must be always_on — subscribed rulebooks are left by unsubscribing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
efc650390e |
fix(plugin-context): the empty write-path payload carries the divergence key too
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 19s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Successful in 1m3s
CI & Build / Build & push image (push) Successful in 42s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d74a244b3a |
feat(ledger): divergence readout — button B where button A is canon, shape history, and judged-shape recheck (#2793, milestone 294 step 7)
CI & Build / TypeScript typecheck (push) Failing after 2s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 28s
CI & Build / Python tests (push) Failing after 37s
CI & Build / Build & push image (push) Skipped
Every judgment now goes through one helper that remembers the fingerprint judged (classified_sha) and writes a code_shape_events row; the sync writes vanished / reappeared / drifted events and flags recheck_at when a body moves under an instance/variant. The refresh flags diverges_from on shapes new since the previous computation that sit where one canon dominates the judged siblings of their directory+kind and were not proposed as that canon (a first seed flags nothing); the write-path hint asks the same question in-band for the shapes the hook names. list_shapes(flag=divergence|recheck), shape_history(project_id, path, symbol) (read-only), coverage line/payload/ card carry divergent + recheck. Backup v8 carries the history. Plugin 0.1.36. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
475f0857c9 |
feat(ledger): write-path stamping — a pulled canon the session then instantiates lands as a hook instance row (#2791, milestone 294 step 5)
CI & Build / Plugin hooks (push) Failing after 2s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Failing after 28s
CI & Build / TypeScript typecheck (push) Successful in 36s
CI & Build / Python tests (push) Successful in 1m5s
CI & Build / Build & push image (push) Successful in 39s
The prior-art hook now names the shapes being written (shapes=kind:name — every definition in the payload, or the one enclosing an Edit found by walking the file upward) and the server stamps them as instance rows when the session PULLED a snippet inside PULL_WINDOW that the payload references by symbol or that the semantic arm scored for this very payload. classified_by=hook, evidence in reason; never overrides a judgment or a canonical row, overridable by classify_shapes. Offered-but-unopened stamps nothing. Pulled-and-already-seen snippets stay in the semantic query as evidence without re-entering the deduped menu. A brand-new shape gets a provisional row the next sync confirms or vanishes. Read-scoped keys get the hint, never the stamp. Plugin 0.1.34. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
3162332a13 |
feat(prior-art): edit-time record-sync nudge — the sync class (#2708)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 26s
CI & Build / Python tests (push) Successful in 56s
CI & Build / Build & push image (push) Successful in 43s
A snippet recorded AT the exact file being edited is not a reuse suggestion — it IS the record of the file being changed. The write-path hint now renders those as their own SYNC class: 'snippet #N records this file — updating the record is part of the edit (update_snippet / verify_snippet)'. Nearby and semantic hits stay the reuse menu. The two classes dedup on separate per-session channels (exclude_ids vs exclude_sync_ids, .ids vs .sync.ids in the hook), so a reuse hint shown early in a session can no longer silence the record-sync nudge when the recorded file itself is edited later. Sync surfacing is measured under its own note_usage source (write_path_sync) — its pull-through rate is the scoreboard for whether edit-time sync actually happens, per decision #2707 (no forge connection; records stay current in the session that has the context). Plugin 0.1.31. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9c68faa0bd |
feat(processes): composition contract — the process is the skeleton, the conversation supplies the parameters
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 47s
CI & Build / Build & push image (push) Successful in 24s
Extends #2582's fix per the operator's direction: a stored Process must incorporate the context it was triggered with, not displace it. No new plumbing needed — the live context is already in the session; what was missing is the stated contract. get_process now carries it: live constraints/scope/focus fold into the procedure and override its defaults where they disagree; a clarify step asks only what the conversation has NOT already answered (confirm interpretations, don't re-ask); stated concerns become lenses the procedure applies, not text it discards. The sync-generated skill stubs state the short form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
66c4fd1b88 |
fix(sync): process-skill triggers invoke by name only — resemblance offers, never substitutes
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / Python tests (push) Successful in 47s
CI & Build / Build & push image (push) Successful in 26s
Scribe issue #2582: the sync wrote "Use when {title}-type work is requested" into every generated process skill, so a bespoke live prompt that merely resembled a stored Process got the Process's canned procedure — and inherited approvals embedded in its body (the Drift Audit's fan-out opt-in) that the conversation never granted. Observed live: an explicit systems-review request with its own constraints was answered with the Drift Audit clarify menu recommending ~15 parallel auditors. New trigger contract, both variants: follow verbatim only when invoked by name; on mere resemblance the live instructions govern — offer the process, ask before following, and never inherit embedded approvals the operator hasn't granted in this conversation. (The Drift Audit process record itself was also rewritten to a sequential in-context walk — data change, live already, recorded in the same issue.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2d1e26f38f |
feat(telemetry): ambient surfacings count, apart — enter_project and the skill sync emit
CI & Build / Python lint (push) Successful in 5s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 16s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 47s
CI & Build / Build & push image (push) Successful in 25s
#2477, option (a) as decided, with the readout changed in the same commit. ## The two silent surfaces enter_project returns open tasks + recent notes on every project entry — probably the largest surfacing by volume — and emitted nothing, so the pulls it caused floated unattributed and the surfaced:pulled ratio ran against a denominator missing its biggest contributor. Now source "enter_project". build_process_manifest installs every reachable Process as an auto-surfacing skill on the operator's machine — its own docstring calls it the most consequential passive surface Scribe has — and emitted nothing, so a Process matched on every relevant turn and never opened was indistinguishable from one never installed. Now source "process_skill_sync": the honest event is "installed", which is a surfacing in effect since the description sits in front of the model each session. ## The readout, same commit — the condition option (a) carried Both surfaces are AMBIENT: top-N-by-recency and install-everything are not ranked choices. Pooling them into surfaced_count would make a note's number dominated by "recently updated in a project you opened", and dead-weight detection would read that as popularity — the wrong number read confidently, which is the corrupts-data tier the survey ranked above everything else. So usage_for_notes splits: surfaced_count stays RANKED-ONLY (every existing consumer's reading — "surfaced often, never pulled → dead weight" — keeps meaning what it meant), and ambient_count is new. Classified in SQL via a CASE on AMBIENT_SOURCES so the group count stays three rows per note, not one per distinct source. Pulls stay pooled: "did anyone ever open this?" does not depend on how it was found. #1038 and #2085 read agent pulls and ranked surfacings; both are unaffected by ambient volume, which is the point. Refs #2477 |
||
|
|
45ba4aab25 |
fix(telemetry): the two invisible retrievals log, and /api/search takes scope
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Python tests (push) Failing after 28s
CI & Build / Build & push image (push) Skipped
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 22s
#2463, the remaining findings (finding 3 landed with
|
||
|
|
f20c019f2a |
feat(supersession): demote what a later note overtook, and label it
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / TypeScript typecheck (push) Successful in 10s
CI & Build / integration (push) Successful in 18s
CI & Build / Python tests (push) Failing after 30s
CI & Build / Build & push image (push) Skipped
Step 3 of #278. First step with visible effect. ## Where the demotion happens, and why not in SQL Applied AFTER the pgvector fetch, over an over-fetched candidate set, not as part of the ORDER BY. Ordering by `distance + penalty` would be exact and would turn an indexed top-k into a scan-and-sort of every embedded note — the HNSW index from migration 0067 can only serve a raw-distance ordering. So the query fetches 3x the requested rows by raw distance and the re-rank happens in Python. Demoting after a LIMIT k with no over-fetch would have been theatre: the cut already happened, so a superseded record pushed down still sits in the results and the live record that should have replaced it was never fetched. The cost is stated in the code: a live record outside the over-fetch window cannot be promoted in. With a 0.05 penalty against neighbours ~0.014 apart, that needs the true answer more than three ranks down, which no observed query approaches. ## Demote, never hide — enforced in three places The penalty applies to the RANKING score, not to the relevance threshold. The floor decides whether a record is relevant at all; the penalty decides which relevant record comes first. Applying it to the floor would drop a superseded record out of the results entirely, which is the one thing this must not do. It is small on purpose. Supersession is a claim about SOME of a record's content, so one that strongly answers a question nothing else answers still surfaces — just behind anything comparable that is current. Its test asserts both bounds, the upper one citing the operator's constraint rather than an optimisation. And every test here that could be satisfied by dropping a record instead asserts the record is still present. ## The dedup gate opts out A superseded record is still a duplicate of what you are about to write — the claim is that it is no longer current, not that it is gone. Demoting it there would let the same note be recorded a second time, and the second copy would be the one nothing warns about. ## The label Auto-inject marks a superseded line SUPERSEDED with a pointer to check the later record. One query for the whole menu. An agent handed stale material with nothing marking it acts on it with full confidence, which is worse than never having surfaced it — the ranking is only half the fix. Fails open: a supersession lookup error returns unpenalised results rather than none, because unpenalised ranking is the behaviour that shipped for months and a broken search is not. Refs #278 |
||
|
|
da2383b079 |
fix(retrieval): verify the reserved slot's kind instead of trusting the query
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / integration (push) Successful in 18s
CI & Build / TypeScript typecheck (push) Successful in 32s
CI & Build / Python tests (push) Successful in 58s
CI & Build / Build & push image (push) Successful in 34s
CI caught six failures on
|
||
|
|
f8522fb28f |
fix(retrieval): give reuse a slot, and let experience reach the write path
CI & Build / TypeScript typecheck (push) Failing after 2s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 22s
CI & Build / Python tests (push) Failing after 35s
CI & Build / Build & push image (push) Skipped
Two mirror-image scoping mistakes, neither deliberate (#2246). AUTO-INJECT let every kind compete on raw cosine. That is fatal rather than merely imperfect here, because Scribe's project records are ABOUT software work: a task titled "surface snippets before the agent writes code" is a near-perfect lexical match for "write a function…" while answering none of it. Measured live, a prompt asking for a helper returned three records about BUILDING the retrieval system and zero snippets. Snippets are ~0.5% of the corpus, and the ratio worsens as the project record grows — which is the direction Scribe is meant to grow, so no threshold tuning fixes it. Now the best snippet or process takes the LAST slot when none won on score. Deliberately NOT held to the margin band: that band measures distance from the top overall score, and the top score is the very thing snippets lose to. It still must clear the configured threshold, so a weak snippet cannot buy the slot — silence stays the default. Skipped entirely when reuse already won, so the fix is invisible in the case it isn't needed. WRITE-PATH was snippets-only — the same mistake inverted. An issue recording "we tried this and it deadlocked" could never reach the moment that code was about to be written, though it is arguably the better prior art: it says what NOT to do. Widened to snippets plus recorded experience. That needed a filter the search layer couldn't express. "Experience" is issues plus dev-logs, which differ on is_task, so neither note_type nor is_task alone covers it. semantic_search_notes gains task_kind, which restricts TASKS to the given kinds while leaving non-task notes untouched — so note_type=("snippet", "note") + task_kind="issue" yields snippets, fixed problems and durable notes, without the open to-do list. note_type now accepts a sequence too. Non-snippet hits are labelled with their kind, because an unlabelled issue on that menu reads as "here is code to reuse", the opposite of what it says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
731ca284c3 |
feat(design-systems): give a design system a way to reach the session
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 28s
CI & Build / TypeScript typecheck (push) Successful in 37s
CI & Build / Python tests (push) Successful in 51s
CI & Build / Build & push image (push) Successful in 31s
Storing a design system never made a session aware of one. Rules get pushed into every session by the SessionStart hook and returned by enter_project; a design system had neither, so its standards were reachable only by an agent that already knew to call resolve_design_system — the same silent failure as a token nobody declares. That gap was invisible while the operator's visual standards also lived in a rulebook. Retiring that rulebook (which is what this unblocks) would have deleted design guidance from every session with nothing to say so. - services/design_systems.design_context() — the delivery side. Guidance is chain-merged ANCESTOR-FIRST: a child system holds only what it CHANGES, so its own guidance describes a departure from a house style it never restates, and the leaf alone is a fragment. Tokens are summarised (count + group names), not listed — a hundred declarations would crowd out the context they are meant to inform. - enter_project returns `design_system`, null when the project has none. - The SessionStart context gains a Design system block with pointers to the values, alongside the always-on rules. - server.py's entity list gains Design system, including the negative: do NOT record one as a rulebook, because a token kept as prose cannot be resolved, inherited, rendered or checked. - The rulebook-tier passage used "a design-system rulebook" as its worked example of a subscribed rulebook — it now teaches the opposite, plus a new "is this a rule at all?" test pointing at design systems, processes and snippets. - using-scribe gains a section on building UI against the project's system. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
390846a3d5 |
fix(write-path): disclose cross-language prior art instead of hiding it
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 16s
CI & Build / Python tests (push) Successful in 42s
CI & Build / Build & push image (push) Has been skipped
CI & Build / TypeScript typecheck (push) Failing after 2s
CI & Build / Python lint (push) Successful in 3s
Closes #2244. Retrieval matches on concept, and concepts are language-agnostic: asking about a TypeScript union-find scores 0.72-0.73 against a PYTHON snippet, comfortably over the 0.68 bar. That is useful — a different-language solution gives you the shape even when the code isn't reusable — but the menu line said nothing about it, so the reader either dismissed a good structural reference or pasted Python into a .ts file. Worth noting this predates the concept-query change: raw TS code already matched the Python snippet at 0.73, because the embedder reads identifiers and structure semantically rather than syntactically. The fail state has been shipping quietly; #2242 only made it an intended use rather than an accident. - knowledge._note_to_item projects `language` from the data mirror, same shape as the existing verification projection — a plain column read, no body parsing. - The semantic arm carries language through on the item it builds; it is the arm where these arise, since a snippet recorded AT the path you're editing is almost never in another language. - _prior_art_line folds it into the marker: [similar 0.72 · python]. Together with the score rather than after the title, because the two jointly are the judgement being offered. - One explanatory line is added to the menu, and only when something on it is actually tagged. Two deliberate calls: LABEL, DON'T FILTER. A stricter threshold for foreign-language hits would suppress exactly the shape-borrowing this exists for. They were never the problem; their being undisclosed was. ONLY CLAIM A MISMATCH YOU CAN ESTABLISH. _foreign_language returns "" when either side is unknown — unrecognised extension, or a snippet with no recorded language. A wrong "· python" is worse than no tag. Same-language hits stay unlabelled, so the common case keeps a clean line and the preamble stays off the menu entirely. Operator-typed language names fold through an alias table first (py/python3 → python, tsx → typescript, c++ → cpp); unrecognised names pass through lowercased, which still makes an unknown-but-equal pair compare equal. Trap found while building: _note() in the tests is a MagicMock, so `note.data` auto-created a truthy mock that would have rendered its repr into a menu line. Both test helpers now set data = None explicitly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
57781770c3 |
feat(write-path): query snippets by concept, not by raw code
CI & Build / Plugin hooks (push) Successful in 7s
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 20s
CI & Build / Python tests (push) Successful in 42s
CI & Build / TypeScript typecheck (push) Successful in 33s
CI & Build / Build & push image (push) Successful in 26s
Implements #2242, from the operator's question: would it make more sense to search by the concept of the snippet than by the code itself? It would, measurably. A snippet's embedded text is f"{title}\n{body}", and a snippet's body is composed markdown — When to use / Signature / Location, then the fenced code — so `when_to_use` appears TWICE in the vector and the document is prose-forward. The semantic arm was interrogating it with raw code carrying no prose at all. Measured on the deployed instance against snippet #2222: query built from score best unrelated separation raw code body 0.743 0.630 0.11 name + docstring 0.823 0.602 0.22 hand-written concept prose 0.835 0.583 0.25 A 12-word description beats a near-verbatim reimplementation of the function, and code-as-query RAISES the noise floor. It's also the cleanest explanation for the fragment miss recorded on #2223: a short excerpt carries almost no prose to match a document that is mostly prose. So build the query from what the code says it's FOR — declarations plus the first docstring / JSDoc / leading comment block — shaped as "name(params) — what it does", mirroring a snippet's own title, which is the form that measured 0.823. Server-side rather than in the hook: no manifest bump, so installed 0.1.20 plugins get this immediately; multi-language parsing in bash would be miserable; and it's unit-testable here. Two rules worth calling out, both measured rather than chosen: - NO DOC, NO REWRITE. A bare identifier is not a concept and scored 0.671 vs the code body's 0.743. Separation from noise is identical either way (0.113), but the absolute drops under the 0.68 bar, so preferring a bare name would convert a comfortable hit into a miss. Undocumented code keeps the raw payload. - The 48-char floor still judges the RAW payload, before the rewrite. A concept query is allowed to be shorter than the floor — that is the point, the best queries are short — but a sub-floor edit stays silent even with a docstring. Applying the floor after extraction would discard the best queries. Regex, not a parser: this is on a PreToolUse critical path and an Edit's new_string is rarely a valid module, so a miss must cost only a fallback. Every unrecognised language (Vue SFC, config files) degrades to exactly the previous behaviour. Telemetry now logs the concept query rather than the code, since retrieval_logs is what the threshold gets tuned from and the two aren't comparable. 0.68 is left alone: signal rises to 0.82 while noise FALLS to 0.58, so the bar sits mid-gap instead of near the edge. To be re-measured against the deployed instance rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
1fb883b72f |
fix(write-path): give the semantic arm its own threshold + a payload floor
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 6s
CI & Build / TypeScript typecheck (push) Successful in 22s
CI & Build / integration (push) Successful in 20s
CI & Build / Python tests (push) Successful in 41s
CI & Build / Build & push image (push) Successful in 49s
Closes #2223. The write-path prior-art trigger's semantic arm shared auto-inject's 0.55 threshold, which was tuned on prose. Code embeddings sit on a much higher similarity floor — any two Python-shaped payloads share keywords, indentation and structure — so 0.55 landed INSIDE the noise band. Measured against the live instance: near-duplicate of a recorded helper 0.73-0.74 true positive unrelated colour math / Vue SFC / CSS 0.55-0.63 false positive `x = 1` 0.58 false positive 6 of 8 probe payloads produced a nudge; 4 were noise. The margin gate couldn't help — _AUTOINJECT_BAND is relative to the top hit, so with a single hit it never engages. Two gates are now the write-path arm's own: - kb_writepath_threshold, default 0.68 — above every measured false positive, still 0.05 below both true positives. Auto-inject keeps 0.55; it was tuned on prose and is not implicated. The comment this replaces explicitly reserved the split for when telemetry showed the surfaces wanted different values, so this is the change it described, not a reversal of it. - WRITEPATH_MIN_CODE_CHARS = 48 non-whitespace chars, below which the semantic arm doesn't run at all. Whitespace is excluded so a deeply indented one-liner can't pass on padding. 48 sits under the smallest plausible reusable helper (~60) and well over a degenerate edit, so it errs toward keeping recall — precision is the threshold's job. This is the cheap half of the operator's #89 idea; the full length<->threshold curve stays open there, since they asked to brainstorm it rather than have a scale invented for them. top_k stays shared — "how many titles at once" means the same thing on both surfaces. The existing tests were passing `code="x"` / `code="def f(): ..."` into the semantic arm, i.e. exactly the payloads the floor now drops, so the gate tests were never exercising a realistic payload. They now use a REAL_CODE fixture, plus new coverage for the floor (trivial payload, padding, place-arm unaffected, real helper passes) and a guard on the constant itself. Settings UI carries the new knob with the reasoning in its hint, and the write-path checkbox no longer claims it shares the threshold. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
2b85443dd1 |
feat(snippets): usage signal — was a surfaced record ever actually pulled?
CI & Build / Python lint (push) Successful in 2s
CI & Build / Python tests (push) Successful in 56s
CI & Build / integration (push) Successful in 31s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Build & push image (push) Successful in 44s
retrieval_logs answers "what did the ranker return, at what scores" — the right substrate for tuning a threshold. It cannot answer the question the snippet corpus actually needs: did anyone open this? A snippet nobody opens is not neutral. It takes a slot in every future auto-inject menu and crowds out something useful. Adds note_usage_events (migration 0071): one row per note per event, either 'surfaced' (we put its title in front of an agent) or 'pulled' (someone opened it in full), tagged with which surface produced it. Closes the gap #2082 recorded against this work. The write-path PLACE arm carries no score, so it has no home in retrieval_logs — folding it in would corrupt the score distribution that table exists to capture. The result was that the arm firing on the STRONGEST claim ("there is already a canonical helper in this exact file") was the one arm nobody could measure. Both arms now emit usage events under distinct sources, so their pull-through rates are finally comparable. Deliberate departures from the task as written: - Not in-session correlation. The original framing was "correlate result_ids against a later get_note in the same session." There is no session identity server-side — the MCP endpoint is stateless and the hooks send no session id — and adding one would mean threading an opaque client-supplied token through every read path. Two independent counters answer the question without it: surfaced 40×, pulled 0 is dead weight regardless of how those events distribute across sessions. - Pulls record at the ENTRY POINTS (MCP tools, REST detail route), not in snippets_svc.get_snippet, which update and merge also reach. Counting those would inflate precisely the number meant to say "someone chose to look at this." - get_note records for every note kind, not just snippets. The auto-inject menu surfaces tasks and processes too; scoping this to snippets would pin those at zero pulls forever and make them read as dead weight next to snippets that merely had a counter. Surfaced in the Snippets list as an "N/M used" badge, warning-toned once a record has been offered 3+ times and never opened, with the tooltip saying what to do about it (usually: its "when to reach for it" doesn't say when). No badge at all below one surfacing — "0/0" reads as a verdict when it's an absence of evidence. Also returned from MCP list_snippets so the agent can see dead weight without opening the UI. Telemetry keeps the retrieval_telemetry contract throughout: writes are fire-and-forget, reads degrade to zeroes, and no path can raise into the surface it observes. Refs #2085 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UaYUaouG9jjhATyuxCKrQs |
||
|
|
e0328f2b1c |
feat(plugin): write-path trigger — offer prior art before code is rewritten
CI & Build / TypeScript typecheck (push) Successful in 35s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 27s
CI & Build / Python tests (push) Successful in 47s
CI & Build / Build & push image (push) Successful in 1m2s
The milestone headline. Auto-inject fires on the operator's prompt; the moment reuse is actually lost is later, when the agent decides mid-task to write a helper. A PreToolUse hook on Write|Edit now fires there. Channel: `additionalContext` with NO permissionDecision, so the note reaches Claude beside the tool result and the write is never blocked — a recall aid must not be able to stop the operator's work. Plain stdout would have been invisible to the model, and deny/ask would have made a nudge into a gate. Two arms, different in kind: - BY PLACE — a snippet recorded at this path (or its directory) is prior art by definition, not resemblance, so it is neither scored nor thresholded. This is what #2083's reverse lookup was built to answer. - BY MEANING — semantic search restricted to snippets (new `note_type` filter on semantic_search_notes) over the code about to be written. Place ranks first; the top-k cap spans both arms. Gates carried over from milestone 93 verbatim: threshold, margin, session dedup, titles-never-bodies. Own `source='write_path'` in retrieval_logs so precision is tunable separately — the docstring records that the place arm is unlogged and hands that to #2085. Its own on/off in Settings but the SAME threshold/top-k: one "how loud may Scribe be" knob is easier to reason about than two that drift, and splitting them later is then a data-backed change rather than a guess. Details worth keeping: the hook sends a REPO-RELATIVE path because that is how locations are recorded; the git remote resolves to a project and is never used as the location `repo` filter (different namespaces, would silently match nothing); the endpoint stays a GET because a read-scoped API key cannot POST and every other hook depends on that. plugin.json 0.1.17 -> 0.1.18. Refs #2082, milestone #232. |
||
|
|
8977bed28d |
feat(inject): label the record kind on each auto-inject menu line
Every injected hit rendered identically, so a recorded snippet was indistinguishable from a stray dev-log in the one place prior art most needs to stand out. Each line now carries its kind — [snippet], [process], [task], [issue], [note] — and the header says "records" rather than "notes", which it can no longer claim. Task-ness wins over note_type in the marker: "there's an open issue about this" is the more useful thing to know at a glance. Still title-first: the marker is metadata already on the ORM object, so no extra query and no bodies (#2084). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt |
||
|
|
ef1dbdfc86 |
feat(acl): unify the retrieval scopes; mark shared rows in Knowledge browse
CI & Build / Build & push image (push) Has been skipped
CI & Build / Python lint (push) Successful in 3s
CI & Build / Python tests (push) Failing after 31s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / integration (push) Successful in 34s
Closes #2092 and the Knowledge-browse provenance gap. The two halves of a hybrid search disagreed: the keyword half honoured shares while the semantic half was pinned to NoteEmbedding.user_id, so a shared record was findable by wording and invisible by meaning — the case a semantic search exists to serve. semantic_search_notes now scopes on Note via a `scope` parameter, and each of its five callers declares which kind of act it is: mcp/tools/search.py read the agent asked routes/search.py read the user typed it knowledge.py (semantic) read matches the keyword half beside it plugin_context.py browse nobody asked; never a one-to-one share dedup.py own a verdict that blocks a write must not hinge on another person's notes That last one is the reason this isn't a single global widening: the dedup gate returns "update the existing one instead", so matching a stranger's record would refuse a legitimate create and point at something the caller can't edit. Scope defaults to "own" so a caller that forgets is wrong in the safe direction, and an unknown scope raises rather than falling back — a typo there would be a data-exposure bug. Auto-inject keeps the browse scope, which still admits a collaborator's note via a shared project. Its menu line is the only provenance an agent sees, so a foreign hit now reads: #12 "Title" (0.71) - shared by alex, treat as a suggestion. MCP and REST search results carry shared/owner too. Knowledge browse: the feed hydrates cards from /api/knowledge/batch rather than the list route, so both paths label rows now, and KnowledgeView shows "by <owner>" on records the viewer doesn't own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RLwAaV4DQEmVyn496HnEvt |
||
|
|
04b58ce01e |
feat(acl): shared records are search-only and always labelled as someone else's
CI & Build / Python lint (push) Successful in 3s
CI & Build / TypeScript typecheck (push) Successful in 28s
CI & Build / integration (push) Successful in 35s
CI & Build / Python tests (push) Failing after 41s
CI & Build / Build & push image (push) Has been skipped
Narrows the |
||
|
|
8126db3203 |
feat(plugin): knowledge auto-inject (Path A) — title-first per-turn awareness
CI & Build / integration (push) Successful in 12s
CI & Build / Python tests (push) Successful in 43s
CI & Build / Python lint (push) Successful in 2s
CI & Build / TypeScript typecheck (push) Successful in 21s
CI & Build / Build & push image (push) Successful in 55s
New UserPromptSubmit hook (scribe_autoinject.sh) + GET /api/plugin/retrieve that surface the TITLES (never bodies) of the few notes clearing four anti-bloat gates: a per-user confidence threshold (stricter than pull search), a margin gate, per-session dedup (exclude_ids), and a top-k ceiling. Each retrieval is logged to retrieval_logs as source=auto_inject so the threshold can be tuned from data. Per-user config (enable / threshold / top-k) is DB-backed via /api/settings with a Settings UI card; defaults enabled, threshold 0.55, top-k 3 (conservative — tune once auto_inject telemetry accrues). Scribe: project 2, milestone 93, task 1033. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xz4j1H7pjYSjKsEpgcNH5E |
||
|
|
33f9a0a4d4 |
feat(plugin): Phase 4 — Scribe Processes auto-surface as local skills
#755 Phase 4. Saved Scribe Processes (DRY pass, Drift Audit, …) now surface as auto-triggered Claude Code skills instead of pull-only get_process calls. Design correction vs the plan: stubs live in the USER's ~/.claude/skills/, NOT plugin/skills/_instance/. The plugin is git-cloned and identical per install, so instance-specific generated files can't ride in it; personal skills are live-detected within the session (verified via claude-code-guide). MCP prompts were the alternative but are pull-only (no relevance auto-surface), so skills are the right primitive. - backend: GET /api/plugin/processes manifest (services/plugin_context. build_process_manifest) — {name, slug, description} per Process; description is the auto-surface trigger (title + preview); slugs deduped, blanks skipped. - plugin: scribe_sync_processes.sh writes ~/.claude/skills/scribe-proc-<slug>/ SKILL.md (body = "call get_process(name), follow verbatim") and PRUNES stale scribe-proc-* stubs. Fail-open + silent; a transient fetch failure never wipes existing stubs. Runs as a 2nd SessionStart hook + via the /scribe:sync command. - plugin.json 0.1.7 → 0.1.8; README updated. - tests: build_process_manifest (render, slug dedupe, blank-title skip, preview truncation). Sync script's write+prune validated in isolation (plugin/** is not CI-covered): correct stubs created, stale pruned, unrelated skills untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
8fe571e175 |
feat(plugin): resolve session project from git remote, not a pinned project_id
The SessionStart hook asked for a project_id via plugin userConfig, which pins one install to a single project — wrong for an operator working across many repos/projects. Resolve the active project server-side from the working repo's git remote instead (a stable identifier, not a dir-name guess). - repo_bindings table (migration 0064) + RepoBinding model: (user, repo_key) -> project, FKs CASCADE. - services/repo_bindings: normalize_repo_key collapses ssh/https/scp/creds/port/ .git to host/owner/repo; resolve/set/list/delete. - GET /api/plugin/context takes ?repo=<remote>; unbound repo -> a "bind this repo" hint with a ready bind_repo() call. project_id kept as manual override. - MCP tools: bind_repo / list_repo_bindings / unbind_repo. - Hook sends ?repo=$(git remote get-url origin) URL-encoded; all project_id handling removed. plugin.json drops the project_id userConfig (0.1.2 -> 0.1.3). - Tests: normalize equivalence classes + unbound-hint rendering. Refs task 755 (Scribe-as-plugin push channel). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |