Rules mount on moments, and one retrieval pipeline (milestones 458 steps 1–4, 456 steps 1–3) #201

Merged

bvandeusen merged 11 commits from dev into main 2026-10-05 12:41:56 -04:00

Conversation 0 Commits 11 Files Changed 57

+4733 -866

11 Commits

Author	SHA1	Message	Date
bvandeusenandClaude Opus 5.5	b29689d4de	fix(retrieval): declare the reply moment's surfacing site in FAN_OUT_SITES (milestone 458 step 4b, #4922) ... moment_delivery records the mounted half's surfacing under source=rp.MOMENT_RULE_SOURCE. That is an attribute the registry's extractor cannot resolve, so the site is now declared with the value it emits, read from the pipeline's own constant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 12:32:07 -04:00
bvandeusenandClaude Opus 5.5	c6cdfc2172	feat(moments): the reply moment holds a finished reply for one read (milestone 458 step 4b, #4922) ... The reply is the one act no tool call marks, and it is where "let me know if it works" gets said. A new Stop hook (scribe_reply_check.sh) sends the finished reply to POST /api/plugin/reply-rules, which checks it twice: - mounted: every unopened RULE on reply.report, plus reply.ask when the reply asks a question. Deterministic. - semantic: the reply's head and tail against every rule's trigger, on a new ranked surface, reply_rule. It is the backstop for whatever the earlier arms missed. Its floor is its stop bar (default 0.80, budget 1), with its own Settings dials. The new stop_only stage records surfacing for the rule that holds and nothing else, because nothing else reached anyone. Following the operator's ruling from 456 step 8, a rule that holds blocks once, in the server's words. The hook blocks only on a reason it was given, so an unreachable instance never stops a session, and it never holds the rewrite. The ledger is the act checkpoint's own, so a rule holds a session once across both doors and the per-session cap counts both. The turn reader moved from the report check into scribe_defs.sh (scribe_turn_facts / scribe_turn_fact), so the two Stop hooks read a turn the same way. The output was checked identical on a real transcript. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 12:30:07 -04:00
bvandeusenandClaude Opus 5.5	b3b616b20a	fix(moments): the tools import the delivery module, and the tool-list cache leaves the swept directory (milestone 458 step 4a, #4922) ... Two guards caught 7865313: - test_mcp_tool_processes reads every coroutine in a tool module's namespace as a tool, and a name-imported attach_moment_rules looked like one. All seven modules now call moment_delivery.attach_moment_rules, and the parity guard accepts the attribute form. - The session-ledger convention: a file in a swept directory must be a .ids ledger. The tool-list cache describes the install, not the context, so it moves to its own directory, scribe-moment, where a compaction does not sweep it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 12:24:30 -04:00
bvandeusenandClaude Opus 5.5	78653130d6	feat(moments): mounted rules arrive when their moment happens, through every door (milestone 458 step 4a, #4922) ... A rule mounted on a moment now reaches the session when an act reaches that moment, with no semantic match involved: - run_moment_arm on the pipeline: a lookup, not a ranked search. Each line names the moment and the act that reached it ("at work.deliver, reached by `git push`"), so a misfire is visible where it lands and can be unmapped in-session. A repeat is cited, not quoted; fresh rules are recorded surfaced under source moment_rule with the moment in detail. No retrieval_logs row, as for the other lookups, so no latency is persisted for this arm. - rule_scope: a rule's home clause, moved out of semantic_search_rules so the moment lookup scopes by the same one. - rulebooks.rules_on_moments / mounted_moments. - The plugin door: a catch-all PreToolUse hook (scribe_moment.sh). It keeps /moment-tools' answer on disk for five minutes, so a call to a tool that cannot reach a mounted rule sends nothing, and an install that has mounted nothing sends one request per window. It shares the rules ledger with the other arms and fails open silently. - The MCP door: Scribe's own tools named by the shipped mappings carry moment_rules in their response, so a client without the plugin gets them too. The hook skips those tools. A guard pins the attach on every one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 12:21:45 -04:00
bvandeusenandClaude Opus 5.5	cc26054437	feat(moments): rules mount on moments, through every rule door (milestone 458 step 3, #4921) ... rule_moments (migration 0117) records which moments a rule arrives at, by catalog name, cascading with the rule. rule_detail, the one seam every rule door already returns through, gains moments beside system_ids: None leaves the mounts alone, a list replaces them. get_rule and both list_rules doors read them back, batched per page. All five MCP rule/preference writes and the three REST ones take moments and validate them before their create or update. An unknown name is refused with the catalog listed and leaves no half-made rule behind; a parity test pins that ordering on every door. Backup v22 carries the mounts as a join table remapped through the rule map; a real-Postgres round trip checks they land on the restored rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 11:03:51 -04:00
bvandeusenandClaude Opus 5.5	cf3de5bae1	feat(moments): actions map onto moments, with in-session corrections (milestone 458 step 2, #4920) ... moment_actions.resolve(tool, input) names every moment a call reaches and the action that reached it. One call can reach several: kubectl apply is a run, a deliver and a reach outside the workspace. Command tools match by how each segment of the line starts, with a word boundary; other tools by field=value arguments. The MCP server prefix and case are ignored. 56 shipped defaults cover the harness tools, Scribe tools and common command shapes. moment_mappings (migration 0116) holds what an install adds and the defaults it switches off. A removal is a stored row, so an upgrade does not switch the default back on. Per the operator ruling, corrections happen in the session: map_action and unmap_action (write tools) return now_reaches so the fix can be confirmed in the same reply. list_moments now shows each moment's actions on this install. REST mirrors both doors, recorded as human. Backup v21 carries the mappings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 10:58:00 -04:00
bvandeusenandClaude Opus 5.5	2ff7f2f34f	feat(moments): the moment catalog rules will mount on, readable in-session (milestone 458 step 1, #4919) ... Fourteen generic moments of work (session.start, work.start … reply.ask) plus the skill.<name> family, each with what it means and the kinds of action that reach it, written for any kind of work rather than software alone. The catalog is code because every install needs the same mount points; which actions reach a moment is per-install data (step 2). require_moment refuses an unknown name with the catalog listed, so a typo cannot become a mount that never fires. list_moments (read-only) and GET /api/retrieval/moments hand out the same catalog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 10:50:45 -04:00
bvandeusenandClaude Opus 5.5	0720ab6dcf	refactor(retrieval): the completion-report preferences run through the pipeline (milestone 456 step 3, #4905) ... reply_preferences.completion_preferences was the last hand-written rule search + record_retrieval + record_rule_surfaced triple outside the pipeline. It is now REPORT_PREFERENCE, a RuleArm with kind="preference" over COMPLETION_QUERY, read back as records rather than as lines. - RuleArm gains `kind`. _ranked asks the ranker for that kind and drops anything else on the way out, before the row is logged. - RuleResult gains `shown`, the (score, rule) pairs behind the lines, for callers that return records. - RuleMoment.project_id may be None: logged as given, searched as `project_id or None`. report_preference rows keep their NULL project. - Guards: reply_preferences now has 0 direct rule searches. The registry constant-source example moves from report_preference to preference_slot, the pipeline slot that records through a module constant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 08:38:05 -04:00
bvandeusenandClaude Opus 5.5	b72c9a92e2	test(retrieval): two source guards read the pipeline where the rule arms now live (milestone 456 step 2, #4904) ... CI run 8137 had two failures, both source-scanning guards whose property moved to the pipeline. The property itself still held: - test_every_surface_name_is_a_real_telemetry_source looked for source="<name>" literals. The rule arms now record under their spec field, so the spec is the join key, read from RULE_ARMS. - test_every_hook_rule_search_says_which_project_it_is_for counted 4 direct searches in plugin_context. It now expects 0 there, so a copied arm fails the count. It also asserts that the pipeline has exactly one search and that its keyword literal carries project_id and never everywhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 08:07:26 -04:00
bvandeusenandClaude Opus 5.5	f6b824b214	refactor(retrieval): the three rule arms run through one pipeline (milestone 456 step 2, #4904) ... prompt_rule, pre_tool_rule and the rule half of the write path each wrote the same steps out by hand: search, band, split fresh from repeats, log the call before any early return, reserve a slot, render, record surfacings. The copies drifted, and #3497, #3750 and #3752 were fixed one copy at a time. - New services/retrieval_pipeline.py. run_rule_arm runs the stages in one order. RuleArm is the spec (source, band, compact_tail, checkpoint, preference_slot). RuleMoment is the query and the session ledger. - The I/O (ranker and recorders) is passed in as RuleIO. plugin_context resolves it from its own names at call time, so existing patch points still apply. - _rule_band, _rule_hint_line, checkpoint_for, checkpoint_reason, the band constant and the preference slot moved into the pipeline unchanged. plugin_context re-exports them. - A recorder that raises now costs only its row, never a rendered line. - The flags reproduce today exactly. Whether prompt_rule should band, and whether the act arms should reserve a preference, are step 7. - Registry: the pipeline call sites are declared in FAN_OUT_SITES, with values read from the specs. - The #3497 structural guard now checks the one implementation, and that plugin_context writes no rule-source row of its own. plugin_context.py: 3,558 -> 2,980 lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 08:01:23 -04:00
bvandeusenandClaude Opus 5.5	be62abf142	perf(retrieval): one embedding per query per plugin request (milestone 456 step 1, #4903) ... A single plugin request fans one query out to several ranked arms. The operator message is searched by auto_inject, its reuse and lesson slots, prompt_rule, the preference slot and rule_via_lesson, and each arm called get_embedding on its own: up to six model calls for one vector. - embeddings.query_embedding_memo(): a request-scoped ContextVar memo that get_embedding consults. Outside a scope the default is None, so every other caller is unchanged. A failed embedding is not remembered. - memoized_query_embeddings decorates /retrieve, /tool-rules and /prior-art. - get_embeddings (document chunks) is untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-05 07:55:06 -04:00