155 Commits

Include renames

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5	fdc07f2a2b	fix(search): show the passage that matched, not the opening of the body (#4243) ... Raised by the operator: are we limiting what comes back by character count, and how do we verify the pertinent part is the part displayed? We were not. mcp/tools/search.py sent (note.body or "")[:240] — a head cut, with no marker that anything had been removed, so a 240-character preview of a 4000-character record was indistinguishable from a complete short one. The opening is the wrong span. The match is semantic and per chunk, and semantic_search_notes collapses to best-chunk-per-note — its own comment at the collapse says "the first appearance of a note is its best chunk". So the system identified the passage that earned the hit and then discarded it: select(Note, distance) kept no chunk column. A record could rank first on its sixth paragraph, be previewed by its first, and be judged irrelevant on a span the search had already scored lower. That biases against long records, and it is self-concealing — the caller who does not open it never learns the preview was misleading. - embeddings: chunk_index/chunk_text ride along in the select, and the collapse records the winner in report["best_chunk"]. Carried in `report`, NOT by widening the return tuple: ten callers unpack (score, note) at ~18 sites and nothing would catch the misses (lesson #4207). `report` is the side-channel this function already uses for best_available_score. - search(): excerpt / excerpt_is / body_length, and read_full when there is more. A caller that cannot tell a matched passage from a document opening cannot judge whether to look deeper, which is the only decision the field supports. elide() moves to services/text.py so both callers share one copy, and it keeps BOTH ends with a stated gap — it is the fallback for when nothing identifies a better span than "all of it", not the goal. Also fixes a guard that produced a false failure on the previous commit: test_pull_telemetry checked `"project_id: int = 0" in body.split("\n")[0]`, which sees only the first line, so wrapping get_task's signature over four lines made it report a function that does take the project as one that does not. Parsed with ast now, and proven to still reject an absent or wrongly-typed parameter rather than being appeased by reflowing the code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 08:50:16 -04:00
bvandeusenandClaude Opus 5	4f2977b848	fix(tasks): add_task_log wrote to a surface no agent could read back (#4241) ... The work log reached the web UI through routes/task_logs.py and nothing else. get_task returned only the body — a claim written once, before the work — with the record written during it invisible beside it. So a stale body arrived with nothing to contradict it, and this session rebuilt work that had already shipped, with the evidence sitting in the task's own logs. Read side, scoped through the access layer (rule 78): - logs_for_task / count_logs_for_task / log_counts_for_tasks in services/task_logs.py. Scoped by who may read the TASK rather than by who wrote the entry: list_logs filters TaskLog.user_id == user_id, which hands a shared collaborator an empty list reading as "no work has been done". The page query folds readable_notes_clause into the same statement so the permission does not become an N+1. - get_task returns work_log; list_tasks and get_milestone steps carry log_count, zero-filled so "none" is a count and not a missing key. Elision keeps both ends. The newest entry arrives whole to 4000 chars because it answers "where does this stand"; older ones are shortened from the MIDDLE, never the head. A head cut selects what a reader sees by character position, which is uncorrelated with what matters — an entry closing with "so this shipped in 04775c3" loses the one sentence that answers the question, and a truncated flag says something went, never whether it mattered. The gap states how many characters it covers. conftest gains an autouse stub for the new read arm, same reasoning as _no_rule_arm: three widely-called tools grew a database read, and the existing call sites should not each have to learn about it. Raised while reviewing this: search() has the same shape and worse — body[:240] with no marker at all, while the chunk that actually matched sits unused in the row that won. Filed as #4243, not fixed here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 08:45:43 -04:00
bvandeusenandClaude Opus 5	f8e53c1c35	fix(telemetry): a warning fired on an arm whose decline rate is arithmetic, not evidence (#4232) ... Found by reading a live `retrieval_telemetry` readout after milestone 419 deployed, not by inspection. The readout said: cannot_decline / report_preference — "45 calls, 0 of them returned nothing. An arm that fires unasked has to be able to say nothing; this one never has. Check that it applies its floor at all." And printed, beside it, that arm's band: p10 = p50 = p90 = min = max = 0.791. FIVE IDENTICAL PERCENTILES IS THE TELL. That is not a ranking, it is one record at one score on every call — because `report_preference` searches a fixed string (`reply_preferences.COMPLETION_QUERY`, a module constant, and deliberately so). For a fixed query against a stable corpus the top score is a CONSTANT, so the arm's decline rate is 0% or 100% and never in between; which of the two it is depends only on where the bar sits relative to that one number. "Never returned nothing" is therefore arithmetic, not evidence, and the warning's own remedy — check whether it applies a floor — cannot be answered from it. The arm already knew this about itself; the warning did not: "a fixed query makes this arm's score a constant and a floor a hair above it produces a dead arm no amount of traffic will ever reveal" — services/reply_preferences.py THIS CLASS OF BUG ALREADY HAS A GUARD, which is the argument for the shape of the fix. `Point.logs_unconditionally` exists because of #3497: both rule arms once logged only their hits, so their zero count was structurally 0 and this same warning would have fired on a LOGGING property while sending the reader to move a threshold that was never involved. This is that one step over — a QUERY-SHAPE property — and gets the same treatment: a declared field on `Point`, and exclusion rather than trust. AND THE WARNING THAT WOULD BE INFORMATIVE HERE DID NOT EXIST. For a fixed-query arm the dangerous state is the mirror image: every call empty, meaning the bar is above the constant and no further traffic will ever move it. The arm is off rather than quiet, and nothing in the readout said so — `expects_traffic` covers an arm with NO calls, not one with calls and a 100% decline rate. That state is real and reached: `report_preference` once logged 69 consecutive declines at 0.0006 under its bar. So `fixed_query_never_clears` sends the reader to `near_miss_samples` and not to the dial — because that incident is also the one where the statistic and the correct action pointed opposite ways. Every percentile said lower the floor; opening the refused record showed it was rule 77 arriving as a false positive, and lowering it would have delivered that rule on every completion report ever written. Guards in tests/test_retrieval_warnings.py, including the falsifier that matters most here: `cannot_decline` must still fire on an arm whose query varies, or this change is a disabled check wearing a narrowed one's clothes. Both boundaries tested from both sides, per that module's own standard. The new code is documented in the `retrieval_telemetry` tool docstring beside the others (rule 33) — an undocumented code in a readout is a reader meeting a verdict with no way to disagree with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 08:05:52 -04:00
bvandeusenandClaude Opus 5	edbc31f8ca	feat(lessons): the kind whose whole question is "is this trigger right" was the one kind that could not see its own counts (#4196) ... The lesson slot has recorded surfaced-vs-opened since it shipped. Nothing showed it. `get_lesson`'s REST door attached `usage` to the payload and no view rendered it; the listing did not attach it at all, and neither MCP door did. #4196 asks when a lesson that keeps getting followed should become a rule, and names the trap in the same breath: raw frequency cannot separate "this should bind" from "this trigger is too broad", and the second is the commoner reading by a wide margin. Surfaced-AND-opened can separate them. Neither question is answerable by a reader who cannot see the numbers, which is why this is the first step and not the threshold. NO THRESHOLD IS PROPOSED HERE, deliberately. The corpus today is 10 lessons with 7 recorded surfacings and 3 opens, over about fifteen hours of usage data. A promotion rule fitted to that would be fitting noise — #3311's failure, and the warning lesson #4228 was written to carry. `UsageBadge` already declines to render a verdict under three surfacings for the same reason. So #4196 stays open: its subject, the promotion path, is still unbuilt. What lands is the evidence it needs. - REST `GET /api/lessons` and MCP `list_lessons` attach `usage` to every row, from one aggregate per page rather than a per-row read, which would be N+1 by construction. Every row carries the key zero-filled, so "never surfaced" is a state a reader can see rather than a missing field they have to interpret. - MCP `get_lesson` attaches it too, and reads it BEFORE recording its own pull. That door records a pull on every open — it has to, or the kind sits permanently at zero — which makes the order load-bearing in a way it is not for a kind that only counts. The REST detail door already ordered it this way; the two now agree about what the number means. - `LessonDetailView` renders `UsageBadge` (snippet #3460) rather than re-spelling the chip, with the advice keyed to this kind: a lesson that is repeatedly offered and never opened is usually keyed to a situation nobody is in, so it points at re-keying `when_to_apply`, not at deleting the claim. Guards, in the two styles this pair of doors already uses: the MCP side driven behaviourally through mocks, including the call ORDER for `get_lesson`; the REST side on structure like its siblings in test_lesson_rest_door.py, because the route is decorated and returns a Quart response. Rule 167's falsifier is included. KNOWN GAP, not fixed here: `KnowledgeView` is the only lesson LIST in the UI and it reads `/knowledge`, not `/lessons` — so the REST listing change reaches `frontend/src/api/lessons.ts::listLessons`, which currently has no consumer. The agent-facing listing does reach a reader today. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 02:33:33 -04:00
bvandeusenandClaude Opus 5	512d0326a0	fix(telemetry): a floor that moved inside the window makes the band check a comparison of two populations (#4225) ... `retrieval_telemetry(days=30)` reported, for write_path_rule: "the weakest tenth of what this arm returns scores 0.6984, only -0.0216 above its floor of 0.72" A negative distance above something. The tenth percentile of what an arm RETURNED cannot sit below the floor that gates what it may return — not inside one population. MEASURED CAUSE. write_path_rule's floor was 0.68 until 2026-09-02, when 2385100 (#3318) raised the shipped default to 0.72. The window opened 2026-08-22, so six days of it are calls made under the old bar; top_score.min for the surface is exactly 0.68, the old bar still in the sample. AND THE CHANGE LEFT NO TRACE THE READOUT COULD SEE. retrieval_tuning_events records dial turns — a person or a model choosing a number. It was silent about the other way a floor moves: somebody edits floor_default and ships it. retrieval_tuning_history returned {"events": []} and retrieval_surfaces said last_change: {}, source: "shipped". All true, and all of it silent about a floor that had in fact moved. THE RAISE ANNOUNCED ITSELF. A LOWERED FLOOR WOULD NOT: the gap comes out comfortably positive and reads as a clean bill of health on a sample that half predates the bar being judged. Both directions are now pinned. So the check is SUSPENDED, not softened. band_hugs_floor asks whether the scores are piled on the bar; that needs the scores and the bar to come from the same regime. Where they do not, the honest answer is that this sample cannot say, plus the date after which one can — floor_moved_mid_window replaces band_hugs_floor for that arm and never accompanies it. A reader told a number is unavailable goes and gets one; a reader handed a qualified number uses it. NO MIGRATION. `actor` is Text with no CHECK precisely so a new kind of actor is not one — the model's own comment says so, and this is the case it anticipated. "release" joins "model" and "human". user_id is already nullable, which is right: no user did this, a release acts on every account that has not overridden the dial, and a row per user would both multiply and misattribute it. Both readers now take the newest of (this user's change, the release's). THE FIRST SIGHTING IS A BASELINE, written with old_value NULL. Nothing moved; the row exists so the next release has a predecessor. That null is load-bearing: floor_moves_since asks for old_value IS NOT NULL, so a fresh install's baseline does not silently retire the check on every new install. UI: the tuning history rendered actor as `human ? 'you' : 'Claude'`, so a release row would have told the operator that Claude moved a floor it never touched — the one failure the actor column exists to prevent. Three-way now, with an unknown value printing itself rather than guessing. Recorded at startup, inline and awaited. What #4181 cost three hours was concurrency — a background task racing the hook for the same pool. Sequential creates no contention, and this is twelve single-row reads. It must finish before serving because a readout served before the change was recorded is the exact answer this exists to stop giving. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 01:17:37 -04:00
bvandeusenandClaude Opus 5	bb8013928f	feat(telemetry): the readout names rules that were opened and changed nothing (#4213) ... Milestone 419 step 2. Step 1 made an outcome recordable; this makes it readable. `retrieval_summary`'s rule block gains `applied`, `departed` and `distinct_rules_acted`, and `_compute_warnings` gains two codes. TWO CODES, NOT ONE WITH A ZERO IN IT. `read_and_unacted` reports rules that were opened and left no outcome, against the ones that did. It only fires once outcomes exist anywhere in the window, because a window with none cannot tell "every rule was ignored" from "nothing calls `rule_outcome` yet" — and on every install the day this ships, the truth is the second. Claiming the first there would be #3311's failure exactly: a statistic that could not vary being read as a fact about the corpus. The cold case gets its own code, `outcomes_never_recorded`, whose prose says in as many words that it does NOT mean the rules were ignored. `applied` AND `departed` ARE NOT SUMMED. A departure carries the reason the agent gave and is evidence about the RULE; an application is evidence about the agent. Folded together they would say only "an outcome exists", which is true of both and useful about neither. `distinct_rules_acted` counts either, because for the unacted arithmetic the distinction does not matter. An outcome is not a pull. The fold branches on OUTCOMES first and never routes an outcome through the surfaced/ambient split: `source` on an outcome row names the door the outcome came through, not a ranker, so the ambient distinction has nothing to say about it. An integration test holds that line — if an outcome leaked into the pull counters the silently-unchanged rule would vanish into a compliant-looking total, which is the confusion #4212 was opened to end. Verified by lifting the shipped `_compute_warnings` out of source with `ast` and exercising it against the six populations the new tests assert: cold instrument, warm instrument, departures-only, full compliance, nothing opened, and a failed read. The integration tests for the new counts run against real Postgres in CI — count(distinct) with an IN over an unconstrained column is a SQL shape a mock would agree with whatever it did, which is what #2663 was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-21 00:13:45 -04:00
bvandeusenandClaude Opus 5	dfcb000719	feat(rules): a surfaced rule gets an outcome, not just a read (#4212) ... Milestone 419 step 1. `rule_usage_events` could say a rule was SURFACED and that it was PULLED. It could not say what happened next, so a rule that fires constantly and is always obeyed and a rule that fires constantly and is never obeyed left byte-identical telemetry. The second is far the more urgent and was the one the readout could not name — measured on a session where three of seven misses were caught by the operator and none by the system. Two new events, `applied` and `departed`, and a `detail` column carrying the why of a departure. No CHECK migration: `event` was created in 0094 as plain Text with no constraint, verified in the migration rather than assumed from the model, so rule 36 does not bite here — said in both places because the next person adding a value will reach for it. THE THIRD STATE IS DERIVED, AND THAT IS THE DESIGN. Read-and-silently- unchanged is the failure this milestone was opened on, and it cannot be reported: an agent that knew it was ignoring a rule would not be ignoring it. So nothing here asks. `applied` and `departed` are reported; the third state is a rule that was opened and left no trace. An `ignored` enum member would collect nothing while reading as though it had measured something, which is #3311's failure — a statistic that could not vary being taken for a finding. `detail` is a column rather than two more bare event strings because a departure stripped of its reason reads back as a miss, so the two states this exists to separate would collapse again one layer down, in the readout, where nobody would see it happen. Nullable: following a rule needs no argument, and an expensive event is one that stops being recorded. `outcome_state` is the single reading of the four states, taking the aggregate `usage_for_rules` already returns, so the badge, the readout and any later session summary cannot disagree about what "followed" means — the drift #3246 found across the rules system. A departure outranks an application: a rule both applied and argued with is a rule someone argued with, and the argument is the half worth surfacing. `rule_outcome` is the MCP door, classed as a WRITE. The read-only set tolerates getters that call record_pulled, but those are reads that leave a trace; this tool's entire effect is the row, and the row carries prose the agent authored. A read-scoped key that can put text in the operator's database is not read-scoped, whatever table it lands in. Backup carries `detail` on both sides. It is the one field here a fresh install cannot re-earn — counts come back by being used again, a stated reason exists once — and #4197 records that the column guard watches the export side only, so the round-trip test is the thing that would catch a one-sided add. Delivery is deliberately not settled here: how an agent gets prompted to record an outcome is step 3's subject, and the same record serves whichever answer that step reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 23:56:05 -04:00
bvandeusenandClaude Opus 5	0fe19a8440	fix(ledger): a canon may hold a class and the to_dict beside it (#4220) ... The review surface shipped yesterday reported two canons on its first live day and both were sound. Coherence was "do all judged rows share a form", which #2844 failed at 37/62 = 0.597 for containing a model class and the to_dict the canon's own text says the class must carry, and #2849 failed at 4/7 for pairing sync loop-starters with the async ticks they schedule. A review surface whose whole output is noise is one that stops being read. The obvious repair is a trap, and there is now a test standing in front of it. Grouping by family and keeping a majority test makes the check BLIND: before #2844 was cleaned by hand it held 37 classes and 56 callables, which as families is 56/93 = 0.602 — a clean pass, and the 31 rows that had no business being there (Vue functions, route handlers, a dozen tests) would never have been reported at all. A looser bar in the same shape is worse than the bug. So the verdict is inverted. Instead of asking whether most rows agree, it asks how many rows the canon CANNOT ACCOUNT FOR: a row in the majority family is accounted for; a callable defined in a file that also holds a majority-family `type` row is a method of a member, not a foreign body; and strangers above a fifth of the readable rows make the canon incoherent. The majority vote abstains those methods, so a class's own serialisers cannot outvote the classes and turn the members into the strangers. Measured on the real ledger before it was written, which is why it is this rule and not a nudge to the share: clean #2844 has 0 strangers in 62, #2849 has 0 in 7, and polluted #2844 had 31 in 93 — the same 31 withdrawn by hand this morning, named exactly. The entry now carries `families`, the majority `family`, `attached`, `stranger_count`, `unattended`, and `strangers` — THE ROWS THAT DO NOT FIT, replacing a sample of the first twelve members. The reader's question is which rows are wrong, and a sample of the agreeing majority cannot answer it. `unattended` is the discriminator between a check that is too strict and a ledger full of junk: both canons flagged on day one were entirely audit-judged, and nothing showed that without opening each one. Scoped to the review surface. `canon_form` still answers at the precise form level for stamping and divergence, where a sync helper beside an async canon is a fair question; nothing here changes what the ledger writes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 23:39:01 -04:00
bvandeusenandClaude Opus 5	400253d039	feat(ledger): the ledger can say "these look wrong" without acting on it (#4208) ... THE HALF THAT WAS MISSING. #4204 put a floor under what the write-path hook may assert. A floor only guards new writes; every row already stored stands (lesson #4202). Measured after that fix shipped: Portal carried 32 rows under one canon and 3 under another, all stamped on scores of 0.69-0.77 — below the 0.80 floor, so none of them could be written today, and all of them were still there. Scribe's own ledger carries 334 under #2860. `stamps_to_review` reports two things and changes nothing: weak — rows the hook stamped on a resemblance below the current floor, each with its score, signature and derived form. incoherent — canons whose own judged rows do not agree on a form. A canon claims some shapes are the same sort of thing; when its members are a class, three getters and a dozen tests, that claim has stopped being true and every base-rate reading built on it is reading noise. `canon_form` already made such a canon fall silent — nothing made it VISIBLE. IT DELIBERATELY CANNOT FIX ANYTHING, and that is the design, not an omission. The first version of this commit was an automatic sweep that reset rows by score. That is the original defect pointed the other way: what harmed the ledger was not one wrong score, it was a machine recording permanent classifications unattended. Un-recording them unattended is the same act with a wider blast radius. An agent reads the evidence, judges, and records the judgment under its own name through `classify_shapes`. `test_the_service_carries_no_machinery_for_bulk_withdrawal` asserts that structurally, so the next person to reach for an auto-retire has the argument again on purpose rather than in a diff nobody reads. A JUDGMENT IS NEVER LISTED AS WEAK, whatever its age. This is the measured correction to an assumption I nearly shipped: of Scribe's 334 rows under #2860, 302 are in `services/` — the canon's own home — and the ones sampled there are `classified_by="audit"` with no score at all. The legitimate bulk of that canon was never scored; it was judged by an agent in batch. Listing those as weak would invite an agent to withdraw the only real judgments in the ledger. An agent's decision is a different KIND of evidence, not a worse one. THE SCORE NOW HAS A PARSER. It lived only inside a prose sentence, so nothing could ask how strong the evidence for a row was without re-deriving it — which is how 32 rows sat unexamined for nineteen days. Format and reader are one constant apart (`_RESEMBLE_REASON` / `stamp_score`), with a round-trip test and a test pinned to reason strings taken verbatim from the two poisoned ledgers. `live_rows_for` is `live_rows` behind the project read gate, for callers that arrive from outside rather than from a job that already knows who is asking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 22:28:12 -04:00
bvandeusenandClaude Opus 5	e2c3a5c2b5	feat(telemetry): retrieval_telemetry says what is wrong (#3431) ... The tool returned distributions and left the reading to the caller, so every readout was the same four checks done by hand — #3430's baseline, #3835's rule near-misses, the #1038 rerank gate. Mechanical, and therefore forgettable. Tonight's acceptance pass on #3898 was the case for doing this. Reading it by hand meant catching that two surfaces had `covers_window: false`, that `prompt_rule`'s floor had moved three times inside the window (which made the readout self-contradictory: deliveries at 0.622 beside refusals at 0.7199), and that 15 of 20 near-misses were one record against text no operator wrote. Miss any of those and the obvious conclusion was "the bar is too tight" — a floor change that would have injected one preference into every notification. `warnings` is always present and empty when clean, so its emptiness is an answer rather than a gap. Each entry carries the numbers that produced it: "345 calls, 0 declined" is the analysis, "check write_path_rule" is an instruction to redo it. Five codes — cannot_decline, band_hugs_floor, no_duration, surfaced_never_pulled, unregistered_source. cannot_decline has three guards, each a bug it would otherwise cause. Asked surfaces are exempt (a search returning a list every time is working). An arm not known to log unconditionally is exempt — that is #3497 exactly, where both rule arms recorded only their hits, so a decline count of zero was a LOGGING defect and this warning would have sent the reader to a threshold that was never involved. Unregistered sources get numbers but no verdict. `silent_surfaces` is the half the rows cannot show: an arm that emitted nothing is invisible to every row-based check and looks exactly like an arm that does not exist. It is driven by a new declared registry, `retrieval_registry.POINTS` — deliberately NOT `retrieval_surfaces.SURFACES`, which answers "what can be tuned" and excludes the reserved slots because a budget of 1 is their feature. This answers "what can be measured", and the reserved slots belong in it precisely because they are judgeable without being tunable. A test asserts the two cannot drift apart. The registry test derives sources from the call sites with `ast`, not grep, and the difference is not theoretical: `wide_net` and `report_preference` reach their recorder as `source=SOURCE` through a module constant, so a grep for `source="` is blind to both — the narrowing #3191 warns about. Three sites pass `source` as a variable and are declared in FAN_OUT_SITES; the test pins those sites but not the values they can pass, which is why the `unregistered_source` warning exists to catch the rest at first fire. Thresholds are settings (rule 25) defaulted so a fresh install with almost no data produces no warnings at all (rule 115) — a new user's first readout naming five broken things would be describing the emptiness. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-20 20:24:56 -04:00
bvandeusenandClaude Opus 5	a4aae974a2	feat(telemetry): a usage event records which project the reader was in (#4196, #3735) ... `RetrievalLog` has carried `project_id` since it existed, so "this record was SURFACED on project B" was always answerable. `note_usage_events` had none, so "this record was OPENED on project B" was not — and the two cannot be joined to recover it, because there is deliberately no session identity server-side. NoteUsageEvent's own docstring rules that out. That gap sat exactly on the question milestone 385 exists to answer. A lesson's whole claim is that it reaches a session on a project it was not written on, and step 8's acceptance is "retrieved on a different project AND opened". Each half was answerable; the conjunction was not. WHICH project, because the name is ambiguous and the wrong reading makes the column useless: it is the project the READER was in, never the one the record belongs to. The record's own project is already on the note; copying it here would answer a question nobody asked while looking like it answered this one. The surfacing half is free — every arm already holds the scope it just searched, so auto_inject, lesson_slot, the write-path arms and enter_project now record it. process_skill_sync does not and should not: it installs every Process the operator can reach, which is not a project-scoped question, so a project there would be a fiction. The pull half needs the caller, since a getter knows only what it was handed. The five single-record getters take `project_id: int = 0` and pass it through, following the convention `search` and `create_*` already set. Null stays an ordinary answer meaning "not reported" — a pull with no project is still a pull and still counts toward dead weight; it simply cannot speak to transfer. The four REST detail views report none for now: a human opening a record in a browser is a different event from an agent recalling one, and #2245 left that asymmetry deliberately undecided. Guarded the way #2245 and #2476 taught: by source inspection, because a parameter that was never threaded through changes no return value and shows up only as a column that is mysteriously always null. Three guards — the signature, the pass-through, and the arms — plus the can-fail test rule 167 asks for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-19 23:26:55 -04:00
bvandeusenandClaude Opus 5	26a757ecfe	feat(lessons): a lesson is yours to keep current too (#4195) ... The lesson kind shipped with every mechanism for growing and nothing telling a session to use them. `learned_from` is a list on purpose, the dedup gate hands back an existing id rather than minting a twin, and `update_lesson` already names re-keying a bad trigger as the edit that pays most. None of that was reachable as a habit. The exclusivity claim was the bug. The skill said "a preference is the one record you keep current yourself", and by naming only preferences it put lessons outside the habit. That sentence is now "a preference is yours to keep current", which says the same thing about preferences without saying anything false about lessons. Beside it, a paragraph on what growing a lesson means: another incident added to what taught it, a claim stated more exactly, or a trigger re-keyed to the situation that really fired. Written as a practice rather than a prohibition (rule 165) — the reader is named as the one person placed to judge the trigger, because they are standing in the situation it claims to name. `get_lesson` carries the same prompt at the moment it bites: a session reading a lesson inside the situation it names is the only reader who can tell whether the trigger is keyed to what actually fired. The guidance-ownership registry gains the topic and re-points the preference topic's statement, since the phrase it pinned is the sentence this change rewrites — the module asks for exactly that, in the same commit. No index marker: the index names session-start reflexes and this one fires mid-work, so `_INSTRUCTIONS` stays at 1998/2000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-19 23:17:20 -04:00
bvandeusenandClaude Opus 5	d36d68a20f	feat(lessons): the REST door a human can actually reach (#3734) ... Step 7, part two. Milestone 385 built the lesson kind through the MCP tools, which is the agent's surface. The Vue app speaks REST, so a lesson was a record a person could not create, read, edit or retire — rule 27 failing at the door rather than in the view. `/api/lessons` now offers list, create, read, update and trash, plus `/api/lessons/taught-by/<id>` — the reverse of `learned_from`, which the task body calls the direction that gets forgotten and arguably the more useful one: a reader opening an old issue wants to know what was learned from it, and until now the relation was only navigable from the lesson's side. `lessons_taught_by` reads `data[taught_by]` through `path_exists`, the same jsonpath dialect the snippet location lookup uses, so both reverse lookups hit the GIN index (0070) the same way rather than scanning bodies. Share-aware via `readable_notes_clause`: it renders beside a record the caller can already see, so a lesson shared with them belongs there exactly as their own does. THE TRIGGER IS REFUSED WHEN EMPTY, at create and at update. This is the one place the door is not a thin wrapper, and it is deliberate: the service will store a triggerless lesson quite happily — it saves, reads correctly in every listing, and never surfaces. There is nothing to notice afterwards, because it looks exactly like a lesson that works. Better to refuse it than to hand back a record that looks finished. The refusal says why, so the next reader does not take it for a nag and delete it. `lesson_to_dict` moves into the service and the MCP tool's `_to_dict` becomes an alias for it. Both doors now return one shape — a payload spelled once per door answers the two of them differently the first time a field is added — and both compose through `services/lessons.py`, so a lesson written from the web ranks identically to one written by an agent. The document IS what ranks, so that parity is the whole reason the door is thin. The dedup gate matches the MCP path: two lessons under one trigger compete in a single ranked list for one reserved slot, so a duplicate here displaces rather than merely clutters. NOT DONE YET: this is the door, not the UI. #3734 stays in_progress until the Vue views, the router entries, the Knowledge browse badge and the both-ways sources panel exist — rule 27 is about the operator being able to touch it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-19 14:02:57 -04:00
bvandeusenandClaude Opus 5	0ed8e86cd5	feat(instructions): a rule proposal has five answers, and three of them route (#3733, #3896) ... Step 6 of both milestone 385 (lessons) and 399 (preferences). #3896 asked for the fourth and fifth answers in one pass, because two people each adding one branch to a three-way distinction produce a list that does not read as a set. create_rule now opens by asking what kind of thing is being held, with one question that sorts it — what happens if someone doesn't do this? Something breaks, a boundary is crossed: a rule. It gets done a way the operator didn't want: a preference. They lose time rediscovering it: a lesson. The closing question grew the two matching answers, and they are named as first-class outcomes rather than places a proposal lands when it fails. An observation that turns out to be a lesson has been routed, not dropped. Stated as a practice, not a prohibition (rule 165). #3557's first cut opened "NOT YOURS TO CALL UNPROMPTED" and cost the noticing; the wanted behaviour here is still more proposals, and what changes is only which door they go through. create_note says the same from its side, so routing does not depend on having opened create_rule first — and its existing rule test ("a mistake, not merely uninformed") turned out to name the lesson exactly. create_lesson names the fifth kind so the set is complete from every door. create_project_rule's citation of the loop names five answers, since it cites rather than repeats. The force axis has one owner (decision #4027): using-scribe states all three strengths, the sorting question, that updating a preference mid-work is the normal case, and that preferences shape how work is done and never what gets recorded. _INSTRUCTIONS carries the pointer — "Rules bind; preferences guide and you keep them current; lessons inform." It had 14 characters of headroom, so the clause is paid for by trimming atmosphere from three other lines; 1998 of 2000 now. Guards: the proposal-loop test learns the preference branch, the lesson branch and the force question, on both rule surfaces; guidance-ownership gains the force-axis topic (shared with the docstrings, for the moment a proposal is actually written) and the preference-scope topic; a new guard pins that the index names all three strengths and who keeps the middle one current, with its can-fail case being the omission that actually happens — a kind added to the product while the index still describes the corpus that came before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-19 12:12:14 -04:00
bvandeusenandClaude Opus 5	6a2476addb	feat(records): every typed kind gets all five doors and a duplicate report (#4164) ... Arising from #3731, which shipped a lesson with three of five tools and logged the rest rather than widening its own scope. The operator's framing on reading that: each kind deserves CRUD functions and to show up in the search and report functions. So this fixes the property, not the two instances. WHAT WAS MISSING FOR A LESSON: no delete, no list, and no duplicate report. `delete_lesson` is the #2250 situation exactly — the trash is kind-agnostic so `delete_note` always reached a lesson, but nothing said so, and a kind whose own tools offer create/read/update reads as one you cannot retire. `list_lessons` is the only way to ask what has been learned at all: `get_lesson` needs an id you already have, and semantic search returns what resembles a query, never the set. APPLYING THE RULE FOUND THE SAME REPORT GAP FOR PROCESSES, which have had full CRUD for months and have never been in `_REPORT_KINDS` either. Both are in now, each compared only against its own kind. The lesson report default is 0.90 — the general semantic floor, deliberately BELOW its own write-path bar of 0.96. The gate is permissive on purpose so it does not refuse two genuinely different lessons whose triggers read alike, and that tolerance is precisely what wants reviewing later, so the report looks at the band the gate was told to let through. Safe there and not at the gate, because a report proposes and the operator picks where the gate blocks a write. A BUG CAUGHT BEFORE IT SHIPPED: `list_lessons` first read the trigger from `it["data"]`, which `_note_to_item` does not carry — it projects named keys off the mirror (`language`, `verification`) rather than the column. Every row would have listed an empty trigger, which on a kind whose whole point is the trigger is the failure looking like the feature. `when_to_apply` is now projected there beside the others, so every listing surface gets it, including step 7's UI. The guard asserts the PROPERTY rather than the instances: for each typed kind, all five tools exist, are actually offered by register(), are classified for auth, and the kind has a duplicate report. Derived from the kinds themselves, so a fourth inherits the bar. A per-tool test cannot catch a missing tool, which is why four steps of milestone 385 went green over this. `find_duplicate_records` now validates against `_REPORT_KINDS` instead of its own literal — the second copy is what would have refused a kind the service already supported. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-18 18:35:13 -04:00
bvandeusenandClaude Opus 5	1d201d2ff7	feat(lessons): a lesson can be written, and it keeps every incident that taught it (#3731) ... Milestone 385 step 4 — the write path. ITS OWN TOOL MODULE, not create_note(note_type="lesson"), on the snippet and process precedent and for the reason that precedent exists: a kind whose value depends on one field being filled needs a door that ASKS for that field by name. create_note would take a lesson through a generic body parameter and the trigger — the whole of why a lesson is findable — would be something the writer had to know to include. THE TRIGGER IS REQUIRED, refused rather than flagged. Step 1 left the choice open. Refusing is right for the same reason create_rule makes enforcement the deciding question: a lesson with no trigger is not a weaker lesson, it is a note that will never surface, and nothing downstream can tell the difference — it saves, reads correctly in every listing, and is silently absent from the one moment it was written for. A flag is a warning nobody is present to read; the write path is where the writer still is. The message says SYMPTOM, because "required" alone produces a topic where a situation was wanted. The docstring carries the distinction this milestone exists to fix, in a line a reader can apply: the difference between a lesson and a rule is FORCE, not importance. If ignoring it would be a mistake it is a rule and needs the operator's yes; if ignoring it just means someone re-derives it the slow way it is a lesson, and nobody is bound. CARDINALITY: a LIST, in notes.data under `taught_by`. The founding example generalised three incidents into one claim about failure classes no CI lane can see — generalising across incidents is the shape a good lesson HAS, and arose_from_id holds one, so a single id keeps the first and drops two while reading as complete. It lives in `data` rather than a join table for the reason decision #4157 put the trigger there: a table would settle, for every note kind at once, whether provenance is multi-valued — a question nothing has measured. `arose_from_id` is filled only when there is exactly ONE source, because every surface that renders it renders it as THE origin, and one of three would make those surfaces state something false. THE DUPLICATE GATE, which step 4 asked to check: a lesson is judged at a bar ABOVE the sibling band, not the general 0.90. #2518 measured deliberately parallel variants at 0.92 on a document that is mostly prose about the thing, which is exactly a lesson's shape now — so at 0.90 two genuinely different lessons about one area ("CI cannot see this class of failure") would refuse each other. Its own constant rather than reusing the snippet's: the two are separate facts that coincide today, and this number is inherited from a structurally analogous corpus rather than measured on lessons, of which there are none yet. Follows canon #2846 including the third registration point it names and this change would otherwise have missed: get_lesson is in server._READ_ONLY_TOOLS and the two writers in _WRITE_TOOLS, which test_mcp_auth requires. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-18 18:19:28 -04:00
bvandeusenandClaude Opus 5	1361ed7200	feat(lessons): the document shape is the stored record, and it travels (#3730) ... Milestone 385 step 3 — the step where the kind either works or is cosmetic. THE DOCUMENT, and why there is no `lesson_document()` beside `rule_document()` in embeddings. The step expected one. The difference is where the sharp shape LIVES. A rule keeps its trigger in a column and its title is a plain name, so `{title} — {trigger}` has to be synthesised at embed time and exists nowhere else. A snippet — the only sharp record in the corpus by #2485's measurement, 0.153 top-to-second against 0.010–0.023 — gets there the other way: its STORED title is already the join and its stored body already opens with the trigger, so the ordinary `title\nbody` join IS the sharp document. Step 1 chose the snippet route and step 2 built it, so `lessons.lesson_document` composes what is STORED and the generic chunker does the rest. The consequence the step asked about: `chunk_document` is untouched, so CHUNKER_VERSION does not move and NOTHING re-embeds. The step's "Re-embed" section describes a change this design does not make. THE NARRATIVE stays in the body, departing from the step's instruction to keep it out. `rule_document` excludes `why` because long dated narrative made sixteen dev-logs land on the centroid of "development" — but that finding predates chunking (#280). A body over budget is now split, and every chunk is prefixed with the title, which for a lesson carries the trigger. The story occupies its own vectors instead of averaging itself into the trigger's, and each of those is still anchored to when the lesson applies. A guard asserts exactly that. Holding the story out would cost the reader the only part that explains the insight, to buy a sharpness the chunker already provides. GLOBAL IN THE SEARCH is the real new code: `GLOBAL_NOTE_TYPES` and `include_global_kinds` on `semantic_search_notes`, widening the PROJECT filter alone. Off by default, because two callers depend on that filter holding — the near-duplicate gate compares a record only against its own project on purpose, and a globally visible kind there would let a lesson block an unrelated note's create on a project its author never touched. It composes with `note_type` rather than overriding it, so narrowing to snippets does not quietly acquire lessons, and it changes nothing about the ACL: `notes_visibility_clause` still gates every row. Wired into the explicit MCP search only — the operator asked, and there is no budget to spend. The unasked-for injection arms are step 5's subject (#3732) and the legibility of a lesson appearing on a foreign project is step 7's (#3734), so neither is turned on here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-18 16:13:15 -04:00
bvandeusenandClaude Opus 5	5c6175ad97	feat(placement): a record you only cite carries its status (#4154) ... Step 1 made placement cheap for a task whose status CHANGES: create_task and update_task return where it sits, and the report is written from that. It did nothing for a task a reply merely cites. This milestone's own step-6 review reported "#4014 is the open step of milestone 409". #4014 had been done for four days; the open step was #4015. The id did not come from a read — it came from a retrieval hint, which carries an id, a kind and a title and says nothing about status, while list_milestones said "8 of 9" and would not say which one. The gap was there to be filled and the nearest-looking id filled it. Two surfaces, one principle: the status arrives with the id. 1. get_project_milestone_summaries gains next_step — the earliest open step, {id, title, status} or None — carried through _BRIEF_FIELDS to enter_project, get_project and list_milestones. One extra flat query for the whole batch, so #2384's fan-out does not come back. OPEN_STEP_STATUSES moves to services/milestones.py and placement.py imports it; both surfaces now answer "what is next" and must not drift on what counts as open. Both step queries take the same readable_notes_clause (rule 78), so a row cannot name a step its own progress numbers exclude. 2. _record_kind renders a task's status: [task (done)], [issue (todo)]. A finished step and an open one read identically before, which is exactly the line the misreport was taken from. Only tasks — is_task IS status-is-not-None on the model, so there is no fallback branch. reporting-back gains the practice, owned and registered in the guidance ownership table: a record you only mention is a record to read. The guards are structural and each fails on the regression it names: the query count is asserted rather than the payload shape, and the two surfaces' agreement is pinned on the rendered ORDER BY, since a mocked session hands back whatever order the test chose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-18 12:08:00 -04:00
bvandeusenandClaude Opus 5	aee24c9c1c	feat(retrieval): a tuned number carries the space it was measured in (#4104) ... Milestone 416 step 6. A retrieval floor is a cosine similarity, which only means something inside one embedding model's geometry over documents cut one particular way. Change either and every floor on the install keeps applying while describing nothing — and nothing anywhere says so, because the scores simply come out different and the bar goes on cutting. `CHUNKER_VERSION` already solved this for documents: stamped per row, so the backfill re-embeds precisely what is stale. The same idea, applied to the numbers: - `calibration_stamp()` — embedding model + document shape, one definition. TWO fields, never a fused string (rule 149): a mismatch has to say WHICH half moved, because they call for different responses. - `retrieval_tuning_events` gains `embedding_model` / `shape_version` (migration 0104), stamped on every write. Nullable and NOT backfilled — "unstamped" is the honest answer for a row written before this existed, and it reports as `stale: null`, never as fine. - `current_settings` reports calibration per dial: tuned rows from their event, untouched dials from the registry default's own stamp. - `retrieval_surfaces` and the Settings panel show the mismatch. The panel renders ONLY when something is stale, so seeing it at all is the signal. - `migrate_floor` / `migrate_retrieval_floor` answers "a path for thresholds to be inherited by the next model so that they don't have to recalibrate a lot": the raw cosine cannot cross models, but the PERCENTILE it represented can. Measure what fraction of a surface's logged calls the old floor admitted, re-score those queries under the current model, take the value admitting the same fraction. Dry run by default; applying writes an ordinary tuning event with the arithmetic in its reason. Nothing auto-retunes. A stale stamp says a number is no longer a measurement; it does not say what the number should be, and #4102 measured the one case where the statistic and the correct action pointed opposite ways. The load-bearing test is an ABSENCE: no chat-model identifier may appear anywhere in the calibration path. Claude produces none of these scores, so a Claude upgrade must trigger nothing — a false alarm here teaches the operator to ignore the real one on the day bge-small becomes bge-base. Backup v17 carries both columns, unfilled on the way out and on the way back: a round trip must not turn "we don't know" into a stated fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 12:45:48 -04:00
bvandeusenandClaude Opus 5	381c90ca7e	feat(retrieval): the wide net becomes a pull — fifty candidates, no bar (#4103) ... Milestone 416 step 5. The operator's compromise — "if you're worried about excluding potentially important data let limit it to 50 entries" — moved to the surface where it is safe. Fifty in the push would be milestone 394 with extra steps; fifty in a pull crowds nothing out. `what_might_apply(query)` returns ranked rule candidates with NO threshold. The moment it serves is the one where the caller does not trust a bar to decide for them, so it does not have one — every row carries its score and the reader judges. WHY IT IS NOT A BIGGER `limit` ON `search` `_search_rules` returns statement, why and how_to_apply in full, on the stated reasoning that a caller who went looking deserves the whole record. That is the DEEP pull and should stay that way. This is the SHALLOW one — many candidates, each just enough to decide whether to open it. Opposite trade-offs, so it is a second tool. THE PREMISE THE STEP GOT WRONG The task said fifty "costs nothing". `_rule_hint_line` had already measured otherwise: ~143 tokens per line once the trigger is rendered, and #3855 tripled trigger lengths across the corpus. Fifty is ~7,000 tokens — cheap next to an arm firing before every Bash call, but not free, and a tool promising a free wide net gets reached for casually and then regretted. So it reuses the graduated shape #3851 measured for the push: the top few carry their trigger whole, the rest carry a cut of it. TRUNCATED, never dropped — the trigger is what lets a reader judge without opening, and a teaser without one is just an id. The cut borrows `_goal_line`'s technique including the fallback that matters (#4036): `textwrap.shorten` returns a bare "…" for one unbroken word. TELEMETRY Logged under its own pull source, asserted absent from both the tunable push registry and AMBIENT_SOURCES. That guard is load-bearing right now: the push arms' near-miss distributions are the evidence #4121 argues from, and a pull folded into them would move those numbers. INSTRUCTION SURFACES The using-scribe reflex and the MCP instructions both pointed at `search(content_type="rule")` for the consequential moment — the deep tool, at the moment you want breadth. They now point here, and keep `search` for reading a rule you already suspect. Written as a practice rather than a prohibition (rule 165). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 12:18:25 -04:00
bvandeusenandClaude Opus 5	c8bfa6947c	docs(retrieval): stop asking the operator to diagnose retrieval (#4102) ... The last item in this step's done-when: no wording anywhere asks an operator to tune for correctness. Four Settings hints told them to do exactly that — "raise it if rules keep arriving unread", "lower it if a git push arrives with nothing", "lower this if genuine duplicates go unnoticed". Every one of those asks the operator to diagnose a ranker from symptoms, which is the job the model now does from the records: the telemetry says what each bar refused, and reading those records is what separates a real miss from a bar doing its job. The hints keep the explanation of WHAT each number is — that is worth reading — and drop the homework. `plugin_context.py` said the defaults "are meant to be tuned from retrieval_logs once data accrues", which was true and had no owner. It now names who does it and with what. `retrieval_telemetry`'s docstring gained the warning that belongs beside it: this readout has been measured pointing the wrong way, so the ids it returns are the point, not its percentiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 11:38:16 -04:00
bvandeusenandClaude Opus 5	ca49a46c23	feat(retrieval): the model moves its own floors, and says why (#4102) ... Milestone 416 step 4's write half. `retrieval_surfaces.py` made the six push arms describe their `{floor, budget}` the same way; this adds the three MCP tools that let the model READ that and change it, and the backup sections that carry the reasons. The operator's decision, which this implements: "the floor should be chosen and adjusted by the model using it… the user should be able to touch it but the model should be the thing handling it 9 times out of 10." WHY A REASON IS REQUIRED, AND WHY THE TOOL ARGUES AGAINST PERCENTILES The milestone originally listed self-tuning as a non-goal on one measured case, and that case is now the tool's docstring rather than a prohibition: `report_preference` logged 69 consecutive declines with the refused record 0.0006 under the bar, and every percentile said "lower it". The refused record was rule 77 "Extract intent from loose phrasing" matched against a query about report layout — a false positive. Lowering would have attached that rule to every completion report ever written. What separated the statistic from the correct action was OPENING the record. So `tune_retrieval` refuses a blank or perfunctory reason, tells the caller to read `retrieval_telemetry(near_miss_samples=5)` and the record ids it names, and carries that 69-decline example — an abstract warning loses to a number. The non-goal that survives is *statistical* auto-tuning; nothing here reads a percentile and picks a value. BACKUP (v16), which is what CI caught `retrieval_tuning_events` was neither backed up nor excluded, and #2293's guard said so. It is backed up: `settings` already carried the numbers, so dropping this would restore an install with six moved dials and no argument for any of them — precisely the state the table exists to prevent, and worse now that the model is the one moving them. One `_retrieval_tuning_event_rows` builder called from both exporters (snippet #2851); `user_id` travels because a restore has to remap it, which is why the row builder is not the model's `to_dict()`. `surface` is a registry name rather than a foreign key, so the history survives a restore into an install whose ids all differ. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-17 11:28:17 -04:00
bvandeusenandClaude Opus 5	07bdbf1647	feat(rules)!: a rule cannot be created, or edited into, having no trigger (#4099) ... `when_to_apply` is not metadata. `rule_document` embeds a rule as `{title} — {trigger}` / `When to apply: {trigger}\n\n{statement}`, the trigger appearing twice so purpose dominates a short vector — the shape note 2485 measured on snippets (a 0.153 top-to-second gap against 0.010–0.023 for everything else). Without one the document silently becomes title + statement: a DIFFERENT shape, ranked against a corpus it does not match, with nothing to report it. Every bar and every rank in the system assumes one shape. `create_preference` has refused an empty trigger since it shipped. The two rule creators defaulted it to "" — so the shape was enforced for the record kind that guides and optional for the kind that binds. The guard lives in the SERVICE, because both doors reach it: the MCP tools and the frontend's fast path in routes/rulebooks.py. Written in either alone, the other could still create a rule that never fires. The route keeps a matching check for the STATUS CODE only (400, not the 404 it maps ValueError to). update_rule refuses to EMPTY an existing trigger, checked after the mutation so it covers `clear=[...]`, an emptied form input, and any route added later. Deliberately asked as "did this edit remove one" rather than "does one exist": a rule predating the guard has none, and refusing to save it would freeze precisely the unreachable records that most need fixing. Deliberately not following arose_from_id, which the human door exempts itself from because provenance is about auditing what the AGENT changed. That reasoning does not reach this field — a missing trigger is not a missing explanation, it is a rule that does not work, and it fails an operator as badly as a session. 15 test fixtures across 6 files were creating rules with no trigger. They now pass one; that they did not is the point — curation is not a guarantee. Step 1 of milestone 416 "Retrieval stops guessing a bar". First because every later step assumes one document shape, and it is much cheaper to guarantee before a corpus grows than to backfill after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-16 14:45:32 -04:00
bvandeusenandClaude Opus 5	5e4fd017ae	fix(mcp): list tools return rows that say what a record is, not what it says (#4061) ... list_tasks returned every row's to_dict(), body included: a project's todo list came to 93-165k characters, past what an MCP client accepts inline, so the list arrived as a file to page through (the #4045 failure, one call over). - notes.brief_row: id, title, type, project, tags, updated_at; for tasks, status, kind, priority, milestone id and title; description, parent and due date only when set. - milestones.titles_for: one query for the milestone titles a page of rows names. - Brief rows on list_tasks, list_notes, get_milestone's steps, get_system and list_system_records. get_task / get_note / get_snippet read a record in full, and each docstring says so. - tests/test_list_rows_brief.py pins the ceiling: 100 rows of ~5k-character step plans stay under 40k characters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 16:07:00 -04:00
bvandeusenandClaude Opus 5	2811fc9025	fix(planning): the plan gate joins step text through embedding_text (#4079) ... test_nothing_else_builds_the_embedding_document_itself caught start_planning building f"{title}\n{body}" inline for the plan gate's candidate text. That is the embedded-document shape; plan_candidate_text now takes (title, body) pairs and calls embedding_text, so the candidate moves with the corpus it is ranked against (#2486). Also carries the create_task / create_records milestone_id docstring lines from step 5 (#4080), which share the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 13:45:52 -04:00
bvandeusenandClaude Opus 5	59407728e6	feat(planning): start_planning hands back the active plan that already covers the work (#4079) ... Step 4 of milestone 415 "An existing plan is found before a new one is made". A session that could not see an existing plan made a second one beside it. start_planning and create_milestone now ask first: an ACTIVE milestone in the project with the same title, or one that reads as the same plan (title, design and steps against milestone embeddings), is returned with its progress and a pointer to create_records(milestone_id=...). Nothing is created; force=true bypasses. - dedup.find_matching_plan / plan_gate / plan_match_response; access-checked before either arm (rule 78), fail-open like the other gates. - Done milestones never block; the semantic arm needs 200+ chars of candidate. - kb_plan_match_threshold (default 0.90) is a setting, in the Settings view, and pinned against the Python default by test_settings_defaults_agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 13:43:18 -04:00
bvandeusenandClaude Opus 5	3a501c2cac	feat(search): milestones are searchable by meaning — "is there already a plan for this?" (#4078) ... `search` covered notes, tasks and rules, and a milestone — the record a plan lives in — could not be found. A project whose roadmap was written as milestones had every later plan opened beside the one that already described it, because nothing could have told the session it existed. - milestone_embeddings (migration 0102): the third sibling of note_ and rule_embeddings, for note 3163's reason — the search is milestone-specific. The document is title — description, then description and the plan body, so a roadmap milestone with no description is still found by its design. - Written on create, on a title/description/body update, and for a plan made through start_planning / create_records, fire-and-forget with the parent-row claim (#3262); a startup backfill covers every existing milestone. Derived, so it joins _NOT_INCLUDED beside the other embeddings. - semantic_search_milestones: a project's milestones when the caller can read it (access.can_read_project), otherwise the caller's own; optional status. - search(content_type="milestone"): id, title, description, status, project and progress. Its own shape, and not part of "all", whose results are note-shaped. The docstring says what it is for: ask before start_planning. - Integration test on real Postgres: found in its project and not another, status narrows, an unreadable project returns nothing. Milestone 415 step 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 13:34:03 -04:00
bvandeusenandClaude Opus 5	184a3e026d	feat(mcp): enter_project names active milestones with no steps as open work (#4076) ... A plan written as a milestone with a description and no steps was invisible to the session handshake: it lists the 5 most recently touched milestones (#4045), and touching is a step changing, so a step-less milestone can never qualify. FabledLibrarian's roadmap (nine such milestones) sat unseen while later plans were opened as new milestones beside the ones that already described them. enter_project adds `unplanned_milestones`: active milestones with no steps, in roadmap order, id/title/description, up to 10 with an omitted count, none repeated from the recent list, and absent when there are none. The docstring says what they are for: check them before starting a new milestone, and add steps to a match with create_records(milestone_id=...). Milestone 415 step 1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 13:28:05 -04:00
bvandeusenandClaude Opus 5	4e4020c040	feat(rules): move a rule between global and project scope, keeping its id, history, areas and edges (#4063) ... A rule's home is its reach: a rulebook topic makes it global, a project makes it that project's. There was no way to change one, so a project rule decided to be global could only be recreated and the original trashed — losing the id every record cites, its edit history, its area tags and its relations. - services.rulebooks.move_rule(rule_id, user_id, topic_id= | project_id=): exactly one destination (the model's CHECK), owned by the caller, not the rule's current home. A topic already holding a live rule with the same title is refused with a message naming that rule, instead of uq_rule_per_topic failing the commit. Someone else's rule reads as not found. - Deliberately NOT done, and said in the docstring: no version (a version is what a rule said, milestone 323 decision 4), no duplicate gate (nothing new enters the corpus), no re-embed (retrieval reads the home at query time). - Both doors: MCP move_rule, REST POST /api/rules/<id>/move (rule 33). - UI: RuleHomePicker, one component in the rule editor (a global rule) and a project's rules tab (a project rule), so the two cannot drift on what a destination is. - using-scribe names move_rule under "Where a new rule goes". Plugin 2026.09.15.1626. Milestone 414 step 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 12:26:39 -04:00
bvandeusenandClaude Opus 5	0bcd4b5540	feat(rules)!: retire rulebook subscriptions and per-project suppressions (#4052) ... A rule's home is its scope now: a rule in a rulebook topic is global, a rule on a project applies to that project, and retrieval reads that directly (#4074). A subscription had stopped changing anything a session received; a suppression muted rules from a subscription. Operator, 2026-09-15: "we have global and project scoped rules, we don't need the subscriptions now." What goes, whole (rule 22): - Migration 0101 drops project_rulebook_subscriptions, project_rule_suppressions and project_topic_suppressions, and strips subscribe_rulebooks (and 394's leftover exclude_always_on_rulebooks) from stored inception choices. - Service, MCP and REST: subscribe/unsubscribe and the four suppress/unsuppress operations. The Subscribers checklist, the subscribe chips, the skip buttons and the Suppressed section in the rules UI. - Inception asks two questions (design system, seed Systems). create_project and decide_project_inception lose subscribe_rulebooks. - Backup v15 stops exporting the three sections; older archives still restore, the keys simply unread. Trash no longer hard-deletes suppression rows. What changes meaning: - get_applicable_rules is a project's LISTING: its own rules, plus the global rules tagged to an area it works in. Untagged global rules apply everywhere and arrive by retrieval, so they are not listed. A co_surfaces partner on a different project is not dragged in. - list_rules(project_id) lists that project's own rules. - rules_payload drops subscribed_rulebooks and suppressed_*; the handshake's brief form is project_rules alone. - using-scribe's "Where a new rule goes" and inception sections, tool docstrings and docs say global vs project. Plugin 2026.09.15.1620. Milestone 414 step 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 12:20:57 -04:00
bvandeusenandClaude Opus 5	188e78bbcd	feat(rules): retrieval honours a rule's home — global everywhere, a project's rules only in that project (#4074) ... semantic_search_rules searched every rule the user owned, and every hook arm called it without a project, so each project's rules were injected into every other project's sessions and a project rule meant nothing a session could feel. The search now takes a scope: global rules by default (an unbound session, or a caller that forgets to say), global plus project N when given project_id (N's rules only if the caller can read that project, through access.can_read_project), and every owned rule with everywhere=True. The four hook arms and the report preference lookup pass the session's project; an explicit search(content_type="rule") scopes to its project_id, or asks the whole rulebook without one. Milestone 414 step 1. Guarded by an AST walk that every hook call site passes project_id, and an integration test on real Postgres that a rule is reached only from its home. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-15 12:06:34 -04:00
bvandeusenandClaude Opus 5	7f974d9749	feat(mcp): enter_project becomes a small primer: goal, recent work, open work, vocabulary (#4045) ... The handshake carried the whole project record, every milestone's plan, full rule text, the notes most recently edited and ~9k of design guidance. For project 2 that was ~222k characters, past what an MCP client accepts as a tool result. Each category was walked through with the operator and sized to what a session needs on arrival; each names the call that has the rest. - project: id, title, status and the full goal (session start's "full goal" pointer still lands here). get_project keeps the whole record. - milestone_summary: the 5 most recently touched milestones, any status, most recent first, without plans. Summaries gain last_touched_at: the later of the milestone's own edit and its newest step update, from the query that already counts steps. milestone_summary_omitted counts the rest and points to list_milestones. get_project and list_milestones list every milestone, also without plans. - open_tasks: the 10 most recently touched open tasks, with or without a milestone, each naming its milestone. list_notes gains sort="touched" (the later of updated_at and the newest work-log), because a log doesn't bump updated_at. - recent_notes: dropped. Retrieval surfaces notes by relevance, and get_recent covers recency. - systems: id and name. - design_system: summary plus guidance_call. get_design_system gains resolved_guidance, the chain-merged prose; its own guidance field is only the departures, so session start's old pointer to it led to a fragment. The session start pointer and using-scribe's "Building UI" section now name resolved_guidance. - rules: rules_payload(brief=True) gives project_rules as id and title plus subscribed_rulebooks, and records only what it shows. Retrieval delivers rules in full and ignores subscriptions (#4052). Other callers unchanged. - pattern_coverage, inception and systems_bootstrap: unchanged. Clients: the plugin's using-scribe skill, the compaction notice and session start are updated here; the REST project summary only gains last_touched_at. Plugin version minted. Tests: a size ceiling on the handshake for a large project; milestone and task selection and naming; brief rules; resolved_guidance; the session start pointer; and a real-Postgres test that a work-log touches its task and a step update touches its milestone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-14 22:04:38 -04:00
bvandeusenandClaude Opus 5	9b2de3552f	fix(mcp): project reads list milestones without their plans, and cap done ones (#4045) ... enter_project returned every milestone's full plan body. On a project with 39 milestones the handshake came to ~222k characters, 168k of them bodies (110k from done milestones). That is past what an MCP client accepts as a tool result, so the call meant to orient a session arrived as a file to page through. It grows with a project's history, so any long-lived project on any install gets there. - brief_milestone_summary (services/milestones.py) trims summary rows to the listing fields: id, title, description, status, order_index and progress. The plan is get_milestone's job. user_id, project_id and timestamps repeat what the caller knows. - enter_project and get_project share one block: every open milestone plus the 5 most recently updated done ones, in order. milestone_summary_omitted is attached only when older done ones were left out, and names list_milestones and get_milestone. - list_milestones lists every milestone, done included, without bodies. It is the call the omitted line points to, and it had the same size problem. - The REST project summary is unchanged; the web UI reads it. Tests: trimming, the done cap and its order, the omitted key present and absent, get_project and list_milestones, and a size ceiling on enter_project's milestone block for a 200-milestone history. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy	2026-09-14 19:48:24 -04:00
bvandeusenandClaude Opus 5	921565696c	feat(409): an operator's own reply shapes reach the reply they are about (#4013) ... The reporting-back skill ships default shapes; an operator's adjustments to them are preference records. Prompt-time retrieval matches the operator's message, and a shape preference is about the reply, so those preferences were on file and never arrived. Operator's decision (logged on #4013): the server delivers them for a completion report, and the skill asks for every other kind. - Completion reports (option C): closing a task with update_task runs a kind-filtered preference search for the moment "writing the completion report after finishing a task" and returns matches as `reply_preferences` ({id, title, statement, kind}), with a sentence added to `report_back` naming the key. A preference says it is about completion reports through its own when_to_apply; no tag or column. Omitted when nothing matches, and the lookup fails open. - Telemetry: every call logs to retrieval_logs under `report_preference` (empty calls included; a search that never ran writes no row) and hits are recorded surfaced. The source is ranked, so it counts toward pull-through. The bar is the prompt arm's setting until step 6 reads this source's near misses. - Every other reply (option A): reporting-back gains "The operator's own shapes come first". Before a finding, decision, handoff or "where are we", search(content_type="rule") in the words of that moment and follow what comes back. Registered in the ownership guard with reporting-back as owner. - Loading reply shapes at session start (option B) was rejected: it would be a small copy of the preloading milestone 394 retired. Domain-neutral query (pinned); works on an install with no preferences. Plugin version minted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 17:43:16 -04:00
bvandeusenandClaude Opus 5	6c1fd28179	fix(#4022): instruction surfaces stop describing the always-on tier milestone 394 removed ... Skill bodies and tool docstrings still taught the deleted model: using-scribe said always-on rules "arrive whether or not you ask" and that SessionStart may inject a rule index; create_project, decide_project_inception and enter_project said an undecided project gets "every always-on rulebook"; create_rule pointed standards at "the always-on one"; the verification sweeps and retrieval_telemetry listed always-on paths and a live preload. Every passage now describes the current model: every rule is retrieved, a rulebook binds only by subscription, an undecided project inherits nothing, and the preload survives only in telemetry rows older than 394. Also repairs three sentences left half-replaced by the 394 edits: the static context's "If you have not loaded the no rule has arrived", create_rule's "an A subscribed rulebook", and create_project's doubled subscribe_rulebooks entry. Plugin version minted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 10:39:27 -04:00
bvandeusenandClaude Opus 5	7239e3c479	feat(409): the reporting reflex reaches every surface a session reads (#4012) ... Step 3 of milestone 409 "Response shapes". Step 2's reporting-back skill only helps if it fires, and only exists in the Claude Code plugin. - scribe_static_context.md and using-scribe (new reflex 11) say: report back in a shape the operator can read, placed from the `placement` block rather than memory, and point at the reporting-back skill. using-scribe also lists it among the sibling process-skills. - update_task returns a one-line `report_back` cue when a task is closed (done or cancelled). A tool response is the only surface every MCP client sees, at the moment the report is about to be written. - _INSTRUCTIONS takes no line: there is no budget without trading out a session-start reflex. The decision is recorded in server.py's comment block so it is not re-litigated blind. - Guards: test_instruction_surfaces_agree pins the reflex and the placement pointer on both plugin surfaces; a tool test pins the cue on closing statuses and its absence on every other update. Plugin version minted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 10:11:58 -04:00
bvandeusenandClaude Opus 5	46d9134b10	feat(409): a task write returns where the task sits (#4010) ... Step 1 of milestone 409 "Response shapes". An agent reporting finished work is asked to say which milestone it belongs to, which step of how many, and what is next. Without those facts to hand it reconstructs them, and a reconstruction reads exactly like the truth when it is wrong. create_task and update_task (MCP) and the REST create/update task routes now return a placement block: project; and for a task in a milestone, the milestone, position {step, of}, progress {completed, total, pct} and next (the next open step, falling back to the earliest open one before it). - Step order is creation order, not get_milestone listing order, which reshuffles on every update. - Siblings are read through readable_notes_clause; position and progress are computed over that same readable set, so a collaborator is never shown a step title they cannot open, and a note share alone reveals no plan. - Fail-open and omitted when empty, like every in-band decoration. - _no_embedding moves into conftest as one opt-in fixture for both integration modules that need it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 08:43:04 -04:00
bvandeusenandClaude Opus 5	441a1ac31d	fix(#4016): records that cite each other are created together, and a guessed id is refused ... Sessions predicted the ids their next creates would get and wrote them into plan bodies and reference notes before the records existed. The database never collides; the sequence is shared by every session and user, so any concurrent create took the guessed numbers and the references pointed at someone else's records. - create_records (new MCP tool) and start_planning(body=, steps=) create their records in ONE transaction: insert, flush for the real ids, rewrite {{ref:N}} / {{ref:milestone}} placeholders as #id "title", commit. No prediction, no waiting, no stub records left behind when a batch fails. Ids need not be consecutive and nothing depends on it. - Every MCP create/update of a note, task or milestone refuses a #N sitting just above the highest assigned id (within 50): that can only be a guess. Refusal, not warning. Numbers far above the max (PRs, forge issues) pass. - notes.build_note splits validation out of create_note so the batch validates records exactly as a single create does. - writing-plans and using-scribe say to pass steps up front and never write an unassigned id; plugin version minted. Integration test runs six concurrent batches and checks each resolves its placeholders to its own records, and that a failing batch writes nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-14 08:33:56 -04:00
bvandeusenandClaude Opus 5	4eebe271ed	fix(rules): restore system_ids and clear the imports the deletions orphaned (#394) ... Three defects from the sweep, all caught by ruff. system_ids was REMOVED FROM create_rule AND create_project_rule — a real API regression, not a lint nit. The parameter shared a signature line with tier, so deleting the tier deleted it too, and the tools lost the ability to tag a new rule to an area. Areas are what let a rule reach a project after this milestone, so the one parameter that decides reach went missing from the two tools that create reachable rules. The other two are imports left holding nothing: services/rulebooks.py's module-level datetime and its IntegrityError were used only by functions this milestone deleted, and plugin_context lost four (select, async_session, RulebookTopic, rulebooks_svc) with the preload and _topic_titles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 16:24:38 -04:00
bvandeusen	c149ef31a3	wip(394): steps 6+7 — backend path and instruction surfaces	2026-09-11 15:15:33 -04:00
bvandeusenandClaude Opus 5	8e06cdf749	feat(rules): the trigger contract is shown as a worked contrast, and pinned (#3855) ... Follows 8c9f947, which taught the trigger shape on the two update_* surfaces but left the softer regression unguarded: guidance kept and abstracted back to "name the moment in session vocabulary" — advice about being concrete that is not itself concrete, which is the shape that was already on file while the corpus filled with categories. Two attempts to detect that in free prose were written and discarded: - Counting quoted multi-word phrases anywhere in a docstring measured ambient quotation rather than demonstrated triggers. It PASSED the abstracted version by scoring unrelated prose, and text with an odd number of quote characters produced matches spanning the gap BETWEEN two unrelated phrases. - Scoping that count to a window after each trigger mention then FAILED create_preference in its CORRECT state, its examples sitting further from the first mention than any defensible window reaches. Both were proxies inferring demonstration from prose. Where a property cannot be measured, changing the shape of the thing is cheaper than a cleverer measurement — so all five trigger-writing surfaces now carry a two-line labelled contrast: RETRIEVES: "the migration failed with a check violation on a column we just extended" COLLAPSES: "when working on migrations" Unambiguous to parse, free in its wording, and a better teaching form than the sentences it replaces: the labels name the mechanism, so they do work for the reader rather than only for the test. The guard now pins both halves — the field is documented, and the contrast is present, complete and non-identical. Falsified against three regressions before committing: the paragraph stripped, the examples abstracted away, and one half of the pair removed. All three fail; the current tree passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 11:48:14 -04:00
bvandeusenandClaude Opus 5	8c9f947f09	feat(rules): the update surfaces teach the trigger shape, not just the create ones (#3855) ... A trigger is two-thirds of a rule's embedded document, so one naming a CATEGORY rather than a moment collapses the record toward its title and it never arrives. #3835 measured that across 113 rules; #3855 hit it again on the eight preferences, where six named a category and two did not. The split was not carelessness, it was an uneven contract. create_rule has carried the full argument since c61925b (2026-08-27) and the two preferences authored that day got good triggers; the six written weeks earlier got categories. The guidance worked wherever it existed — and it existed on three of five write surfaces. Both update_* tools were silent, and the update path is where every RETROFITTED trigger is written, which is most of them: a trigger that already reads fine as English is the one nobody rewrites. So: - update_rule gains the retrofit case, which is a different trap from the create case. There the field is empty and the instruction is "write one". Here one exists, reads perfectly well, and the honest-looking verdict is that it is fine. - update_preference gains it too, plus why the field is load-bearing there specifically: preferences get a reserved slot filled by a kind-filtered query at limit=1, so the corpus ranks against ITSELF and the trigger is nearly all that separates one from the next. - create_preference and create_project_rule now SHOW a moment instead of describing one. Advice about being concrete that is not itself concrete is the shape that was already on file while the corpus filled up. The guard pins one property: a tool taking when_to_apply mentions it. That is exactly what update_preference failed. The surface list is derived from register() rather than hand-kept, so a write tool added later is in scope the day it lands. Two stronger predicates were written for the softer regression — guidance kept but abstracted — and both were discarded after falsification: counting quoted phrases measured ambient quotation and passed the broken version, and scoping that count to a window failed create_preference while correct. Rule 167 settles it; the discarded attempts are recorded in the test docstring so the next author does not repeat them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-11 11:39:00 -04:00
bvandeusenandClaude Opus 5	89d16d89a9	feat(rules): a preference updates without asking, and says what taught it (#3849 step 2) ... The write path, and the step where a preference stops being a relabelled rule. `create_preference` / `update_preference` on the MCP surface, plus `kind` on update_rule and both HTTP doors. SEPARATE TOOLS, NOT A `kind=` ARGUMENT. create_rule's docstring IS the approval gate (#3557): propose, offer three answers, wait. That is right for a rule — the person it binds should have agreed. A preference inverts it, and one reached through create_rule would be read through that prose, so the caller would hesitate over exactly the act this kind exists to make routine. Two doors, two contracts, one table. Reads stay shared: a preference IS a rule row, and "what governs this" wants both. Two required fields, each buying something: - `when_to_apply`, because the trigger is two-thirds of the embedded document. Without one the record is written, stored, and silently never delivered — indistinguishable from one nobody wrote. - `arose_from_id`, the price of the ungated write. A corpus that drifts with no record of what taught each change cannot be audited, and the operator's veto over drift is worth exactly as much as their ability to read why it happened. The near-duplicate gate is what lets this corpus be written freely and stay small: the second preference about a thing updates the first. It is title-scoped and kind-blind, so it also catches a preference restating a rule that already binds. The asymmetry is guarded as two PRESENCE facts — the rule door still asks, the preference door still says write it — never as an absence. An absence check passes against a docstring that was deleted or rewritten into something else, which is snippet #3352's warning and would read as coverage here while proving nothing. `_plain_detail` moved to tests/helpers on its second copy, per that module's own reason for existing (#2825). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-10 21:59:23 -04:00
bvandeusenandClaude Opus 5	d5ac8408f6	feat(telemetry): record WHAT the bar turned away, not only how close it came (#3807) ... #3670 added `best_available_score` so a threshold could be judged from its rejections. It records how CLOSE the bar came to firing and not WHAT it refused, and that is the half a decision actually needs. Live, pre_tool_rule sits at a ~0.72 bar with a near-miss p90 of 0.7071 — about 117 declines a day within 0.013 of firing. Dropping to 0.707 would take that arm from 22 hits a day to roughly 139: six-fold, on a surface that runs before every Bash call. The percentile says the mass is there. Nothing said whether it was worth showing. NEITHER OBVIOUS INSTRUMENT ANSWERS IT. Pull-through cannot: the injected rule line already carries title and trigger, so a session can comply without ever calling get_rule, and rule pull-through understates usefulness by construction. Reading the rejected records can — and `result_ids` holds only what was RETURNED, so on a zero-result call the near-missed record had no name at all. So the id, from the SAME ranked candidate as the score. Both searches unpack `best` once and read both fields off it, because splitting that into two expressions is exactly how a later edit pairs a score with its neighbour's id — and a score attached to the wrong record is worse than no id, since it invites judging the wrong one and concluding the bar is fine. write_path withholds the id on the same condition it withholds the score (#3739): a surviving id beside a null score names a record without saying what it scored, the pair disagreeing in the other direction. THE READ PATH IS A LISTING, NOT A STATISTIC — an id cannot be percentiled, and a reader tuning a bar needs to go and read the records. Opt-in via `near_miss_samples` (0-20, default 0) so the ordinary readout keeps its size, and deliberately NOT a window function: this module's one production outage was a grouped query Postgres rejected, swallowed by the broad except, every counter reading zero while the mocked tests passed (#2663). One flat ordered query, overfetched, bucketed in Python — the shape that lesson prescribes. Migration 0097, nullable and unbackfilled. Not a foreign key: the table spans record types and `source` says which, exactly as result_ids works. The integration guard pins the listing as PER SOURCE. A global LIMIT would let a noisy source eat the whole quota and leave the surface being tuned showing nothing — which reads as "nothing was close", the misreading this milestone has spent itself correcting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 21:24:32 -04:00
bvandeusenandClaude Opus 5	623464323e	fix(telemetry): a search that never ran is not a decline (#3765) ... `best_available_score` was added by #3670 so a bar could be judged from what it rejected, and it arrived null on four unrelated causes: the corpus offered nothing, the query was empty, the embedder was down, or the DATABASE QUERY FAILED. Only the first is a measurement. The fourth is the #2663 shape — a swallowed failure rendering as a clean zero — inside the field added to fix an instance of the #2663 shape. Found while trying to explain why reuse_slot returned nothing on 45 of 45 calls, and auto_inject on 153 of 161. That investigation is still open; what it established first is that the readout could not answer it. THE FIX IS NOT A NEW COLUMN. A call that never searched writes no row, so every remaining null means one thing: searched, and nothing came close. That is the convention the pre-tool arm already follows for a blank command — "a row here would report a call that never happened and drag the clear-rate down with phantom declines" — extended from the case a caller can see in advance to the ones only the search knows about. Both searches stamp `report["searched"]` FALSE before anything can return and True only where a real result set exists, so every early return leaves it false. It has to be the first thing done to the dict: a return added above that line would leave the key absent. ABSENT IS A THIRD STATE AND IT DEFAULTS TO TRUE. A caller that passes no report cannot know, and the safe reading there is the old behaviour. Only a real search can report False, so absent means "nobody asked" and never "it failed" — which is also why 66 existing mocked searches across twelve test files keep working unchanged rather than being rewritten to simulate a flag they do not care about. A FAILURE IS NOT MADE INVISIBLE. semantic_search_notes already logs a WARNING on a query failure, which is where a broken search belongs: a counter cannot say "I am broken" without a reader already trusting it. Three tests, and the middle one is what makes them discriminate — a blanket `return` passes the first and fails the second, because a call that searched and came back empty is the only evidence a threshold is too high (#3497). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-09 12:47:46 -04:00
bvandeusenandClaude Opus 5	a165483b92	fix(telemetry): a repeat is not a rejection, and near_misses counted it as one (#3739) ... Caught on the first live read after deploying #3670. The readout contradicted itself: pre_tool_rule top_score.min 0.7204 the lowest score ever RETURNED near_misses.max 0.7457 "rejected", but scored higher `best_available_score` is measured pre-threshold, which is right, but for the rule arms it is also PRE-EXCLUSION, which is not. The note arms pass `exclude_ids` into semantic_search_notes so their score is already post-exclusion and clean; `semantic_search_rules` takes no such parameter, so the rule arms filter in Python after the search and a rule that cleared the bar and was dropped as a repeat still reported its score on a zero-result row. That is #3497's distinction — a ranker decline versus a reader already ahead of it — reintroduced one level up, inside the field built to replace a tautology. The population now also requires `suppressed_count IS NULL OR = 0`. The NULL arm is principled rather than permissive: null means the caller filtered INSIDE the search, which is exactly the case where the reported score cannot be contaminated. Deliberately conservative — a call carrying both a repeat and a lower genuine miss is dropped whole, losing that point. It undercounts; it cannot corrupt, which is the right way round for a number read against a bar. It also makes `near_misses.max < threshold` true BY CONSTRUCTION rather than by fixture: an above-bar candidate nobody excluded would have been returned, so its call is not in the population at all. THE TEST DID NOT CATCH THIS, and that is the part worth keeping. The assertion `nm["max"] < 0.72` was already there, with exactly the right intent. It passed because the fixture contained no suppressed call — the guard held because the breaking shape was absent, not because the code was right. Rule 167's stated failure mode, in a test written while citing rule 167. The fixture now builds that shape: a 0.9 hit dropped as a repeat, which lands in the population and drags `max` above the threshold unless the predicate excludes it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 16:42:51 -04:00
bvandeusenandClaude Opus 5	e7c1af32a0	fix(telemetry): the bar can only be judged from what it rejected (#3670) ... `cleared_threshold` was documented as the number to read first. It was a tautology. The search applies the threshold before returning, so every returned result cleared it by construction and a call with no results has no top_score to compare — the condition was true exactly when `result_count > 0`. It was `calls - zero_result_calls` under a name that promised a second opinion, and `zero + cleared == calls` held on all nineteen source/window readings ever taken, today's live seven included. The reading procedure built on it asked the reader to compare a number with itself, and a threshold change was unobservable through it: raise the bar and both numbers move together, so the field could never show a bar set too high. REPLACED, NOT JUST REMOVED. The question the table exists to answer is whether the bar is in the right place, and that is only answerable from the calls that returned NOTHING: how close did the best rejected candidate come? A 0.72 bar turning away a stream of 0.71s is set too high by a hair; the same bar turning away 0.30s is working. Both render as a zero-result call today and nothing separates them, because the losing score is discarded inside the search. So both searches now rank WITHOUT the bar and apply it in Python. The qualifying set is provably identical — rows arrive ordered by distance, so every above-bar row sorts ahead of every below-bar one, and an over-fetch that returned N above-bar rows returns the same N plus some losers. What changes is that the losers are visible instead of dropped in the query. `report` carries the score out without changing what a search RETURNS: eight of eleven call sites want hits and nothing else. New column (migration 0096), nullable and unbackfilled. A row written before this genuinely does not know, and a 0.0 would read as "the corpus held nothing remotely relevant" — a claim invented out of a caller's silence, which is the substitution this whole milestone corrects. The new aggregate is a percentile_cont WITHIN GROUP over a CASE, one step from the shape that produced #2663, where a rejected query was swallowed by the broad except and every counter read zero. It carries an integration guard for that reason: only real Postgres can say it parses, and the symptom of failure is silence. Also adds a guard that no int field in a bucket equals `calls - zero_result_calls`. That identity is what `cleared_threshold` satisfied for its whole life, and it survived because it had its own name and nobody added the two numbers beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 13:50:05 -04:00
bvandeusenandClaude Opus 5	7a2aff7bc1	fix(telemetry): a surface that stopped recording is not one that never ran (#3720) ... `out["sources"]` was built only from the windowed aggregate, so a source with rows in `retrieval_logs` but none inside the window got no bucket at all. Absent is exactly how a source that never existed renders, so a surface that WAS recording and went silent became unreadable — #2663 one level up, the failure that looks like the correct answer. Two queries at different scopes, and only one shaped the output. `_complete_from` reads all-time and knows every source the table has ever held; the windowed loop dropped whatever it did not return. Every such source now gets a zero bucket. Zero is a real measurement here rather than a manufactured one: the all-time query proves the source was recording, and it made no calls across a window it fully covers. No `covers_window` special case is needed either — a source whose first row fell after `since` would have that row IN the window and already hold a bucket, so anything reaching this branch began before it. The counts are 0 and everything else is null. A sampled distribution is not the same claim as a call count, and rendering p50 as 0.0 for a source nobody sampled would assert a measurement — #3311's mistake, in the readout built to prevent it. Found while fixing #3712's fixture, which failed with KeyError for this exact reason. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 11:07:21 -04:00
bvandeusenandClaude Opus 5	21a5831479	feat(telemetry): every counter says when it started being recorded (#3712) ... A window that opens before a counter existed reports that counter as though it had been measured throughout. The reader cannot tell "zero because nothing happened" from "zero because nobody was counting yet", and — worse — cannot tell a partial count from a complete one. That middle case yields a plausible FRACTION rather than an obvious zero, which is what makes it dangerous. It is not hypothetical. A 7-day window opened while the ranked rule surfacing recorders were four days old produced an apparent 64% write loss, which survived a code review, four ruled-out alternative causes and a five-step milestone before an identity check falsified it in one read. Every counter block now carries `complete_from` and `covers_window`. THE GRAIN IS THE SOURCE. retrieval_logs accumulates for months, so a per-table earliest row says months for every source it holds — including an arm added days ago whose counter means something else entirely. The old source would vouch for the young one, which is the exact reading this prevents. A SECTION TAKES ITS LATEST CONTRIBUTOR, NOT ITS EARLIEST. A figure summing several sources is complete only once every one of them was being written, so "*" is a max. Using min would reproduce the original error in miniature. `covers_window` is null, never false, when nothing was ever recorded: "no measurement" is not "partial measurement" — the null convention #3497 established for `suppression`, one level up. Also corrects a stale claim in the tool docstring: it still taught readers that write_path_rule "has never once declined to fire" (#3311). That was the arm writing its retrieval_logs row only on calls that found something; #3497 fixed it, and the arm declines the large majority of its calls. _complete_from takes the caller's session rather than opening its own, departing from the services canon (#2860) because it runs inside an existing block; to be recorded against the ledger once it ingests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-08 10:29:02 -04:00
bvandeusenandClaude Opus 5	c3ecdf0972	feat(rules): the rule gate becomes a practice with a question, not a prohibition (#3557) ... The first cut opened "NOT YOURS TO CALL UNPROMPTED", and that is the wrong instrument. A caller reading a prohibition stops NOTICING rule-shaped things rather than noticing them and asking — which trades a small failure for a larger one. The wanted behaviour is more proposals, not fewer. So both docstrings now describe the practice: propose readily, state the four things, and close with a question the operator answers in one word — approve it as written / let's talk about it / no. Named options where the interface has them, three written-out options where it does not. "Approve it as written" is what makes element 1 load-bearing: they approved TEXT, so that text is stored verbatim. "Let's talk about it" is framed as the expected answer rather than a setback. "No" routes the observation to create_note, which records without binding. The argument for asking is also better than consent. The operator's yes is the one moment the rule is certainly in front of them: afterwards a conditional rule is not read aloud at session start, and a project rule is absent from an unfiltered list_rules(). The proposal IS the review. Guard gains the answers-offered-back element and drops the wording that forbade; its header records why the framing changed, so the prohibition does not get reintroduced as a tidy-up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ	2026-09-04 20:11:42 -04:00