Merge pull request 'feat(rules): the rule arms stop filtering the corpus to one tier (#3702)' (#142) from dev into main

Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:

  PRESENT IN CONTEXT    the rule was delivered at session start
  SALIENT AT THE MOMENT the rule is in front of the reader when the action
                        it governs is about to be taken

A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.

Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.

MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.

Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.

The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ

src/scribe/mcp/tools/rulebooks.py

@@ -315,6 +315,61 @@ async def create_rule(
 ) -> dict:
     """Create a new rule in a rulebook (a SHARED rule — keep it general).
 
+    PROPOSE RULES READILY, AND WRITE ONE WHEN THE OPERATOR SAYS YES. Noticing
+    that something has hardened into a standing instruction is valuable work,
+    and a session that notices it and says nothing has thrown the observation
+    away. So raise it whenever you see one. The single step that belongs
+    between noticing and writing is the operator's yes: a rule binds every
+    future session, and they are the person it binds.
+
+    Their yes is also the only moment the rule is reliably IN FRONT of them.
+    After the write it may not be again for months — a conditional rule is not
+    read aloud at session start, and a project-scoped one does not appear in
+    an unfiltered list_rules() at all. So the proposal is the review.
+
+    When the operator asks for a rule in so many words, that IS the yes —
+    write it and move on. The loop below is for the rule you thought of.
+
+    A PROPOSAL CARRIES FOUR THINGS, and the fourth is the one that decides it:
+
+      1. WHAT it would require — the statement, in the words it would carry,
+         not a gloss of them. The operator is agreeing to text.
+      2. INTENT — what it changes about how work gets done, and what goes
+         wrong today without it. "Be careful about X" is not an intent; the
+         behaviour that would differ tomorrow is.
+      3. WHY NOW — the incident, observation or decision behind it. Pass that
+         record as arose_from_id, and say it in the conversation too: the
+         field is for the reader six months out, the sentence is for the
+         person deciding.
+      4. HOW IT WOULD BE ENFORCED — a test, a CI check, a hook, a schema
+         constraint, a duplicate gate, a review step... or nothing, in which
+         case say so plainly: "nothing — this is prose a session has to
+         remember." Answer this one honestly and it will sometimes dissolve
+         the rule, which is the point rather than a side effect. What a test
+         can assert should BE that test; a rule is what remains when nothing
+         mechanical can hold the thing. A rulebook grows by default and
+         shrinks only on purpose, so a question that prevents a rule is worth
+         more than any question that improves one's wording.
+
+    THEN CLOSE WITH A QUESTION THEY CAN ANSWER IN ONE WORD. Offer three
+    answers, and make the middle one the easy one:
+
+      * "Approve it AS WRITTEN" — you create it with the statement exactly as
+        shown. This is what makes element 1 load-bearing: they approved TEXT,
+        so that text is what gets stored, verbatim.
+      * "LET'S TALK ABOUT IT" — the wording, the scope, the tier, whether it
+        wants to be a rule at all. Most good rules arrive this way, so treat
+        this answer as the expected one rather than a setback.
+      * "NO" — let it go. If the observation is still worth keeping, it is a
+        note (create_note): recorded, findable, and binding on nobody.
+
+    Where the interface offers structured choices, ask it that way — a
+    question with named options is answered in a click, while the same
+    question inside a paragraph is answered by scrolling past. Where it does
+    not, write the three options out as three options. Either way ask once
+    and let the answer stand; re-raising a declined proposal argues a rule
+    into existence, which is the thing this whole loop exists to prevent.
+
     A rulebook rule is shared by every project that gets the rulebook: an
     always_on rulebook binds ALL your projects; a subscribed rulebook binds the
     projects that opt in. So a rulebook rule must read as a general standard —
@@ -433,6 +488,16 @@ async def create_project_rule(
     the rule is returned in get_project's applicable_rules (under
     project_rules) and in list_rules(project_id=...).
 
+    PROPOSE, THEN WRITE ON A YES — create_rule's opening carries the whole
+    loop: the four things a proposal states (what it would require, its
+    intent, why now, and how it would be enforced) and the one-word question
+    that closes it (approve as written / talk about it / no). All of it
+    applies here unchanged. Reach for that loop MORE readily on this surface,
+    not less: a project rule stays out of an unfiltered list_rules(), and a
+    conditional one stays out of session start too, so the operator's yes is
+    the one moment this rule is certain to have been seen by the person it
+    binds.
+
     Check first whether a rule is the right shape at all — create_rule's
     opening asks that question and it applies identically here. A visual
     standard is a design system; a procedure is a process (create_process);

src/scribe/services/embeddings.py

@@ -774,10 +774,17 @@ async def semantic_search_rules(
     is the surfacing question, and it has its own machinery
     (get_applicable_rules) rather than a second, subtly different copy here.
 
-    `tier` narrows to one tier. The write-path hint passes "conditional",
+    `tier` narrows to one tier, and NONE is the ordinary case. The write-path
-    because an always-on rule is ALREADY in the session — surfacing it again as
+    and pre-tool hints deliberately pass nothing: an always-on rule is already
-    a suggestion is pure noise, and noise on a hint that fires on every write
+    in the session, but being in a list from turn zero is not the same as being
-    is how a hint gets ignored.
+    in front of the reader when the action it governs is taken, and filtering
+    on tier made a whole class of rules permanently ineligible for the one
+    mechanism that surfaces a rule AT the moment. Relevance is the threshold's
+    job; see the block above RULEHINT_LIMIT in services/plugin_context.py for
+    the argument and for what the resulting scores are being read against.
+
+    Pass a tier when a caller genuinely wants one class — a listing, an audit,
+    a UI that renders the tiers apart. Not to approximate relevance.
 
     Collapses to best-chunk-per-rule like the note search, so a long rule split
     across chunks competes once rather than crowding the results with itself.

src/scribe/services/plugin_context.py

@@ -94,13 +94,17 @@ WRITEPATH_DEFAULT_THRESHOLD = 0.68
 # THE STRUCTURAL ARGUMENT, which is the only kind admissible here (rule 115).
 # Two facts hold on any install, including one with six rules and no telemetry:
 #
-#   1. The eligible corpus is TINY. The arm searches `tier="conditional"`
+#   1. The eligible corpus is SMALL — every rule an install owns, still only
-#      rules only — a handful to a few dozen documents against thousands of
+#      a few dozen documents against thousands of notes. A top-k over a small
-#      notes. A top-k over forty candidates always returns something, so
+#      pool always returns something, so "the best match cleared the bar"
-#      "the best match cleared the bar" stops meaning "a good match exists"
+#      drifts from "a good match exists" toward "N things were ranked". A bar
-#      and starts meaning "forty things were ranked". A bar calibrated for
+#      calibrated for best-of-thousands is cleared by best-of-forty as
-#      best-of-thousands is cleared by best-of-forty as arithmetic, not
+#      arithmetic rather than relevance.
-#      relevance.
+#      This argument WEAKENED when the arms stopped filtering to one tier
+#      (see the note on that below): a larger pool makes clearing the bar
+#      mean more, not less. The threshold was deliberately left where it was
+#      anyway — moving two variables at once would make the resulting
+#      distribution unreadable, and this one errs toward silence on purpose.
 #   2. Rules are short imperative technical English — a far more HOMOGENEOUS
 #      corpus than note prose. #2223 measured the floor for code against prose
 #      at 0.55-0.63 and set 0.68 above it. A more homogeneous corpus has a
@@ -128,9 +132,14 @@ RULEHINT_DEFAULT_THRESHOLD = 0.72
 # ONE rule per write, not two — and this is deliberately NOT a knob.
 #
 # With a corpus this small, top-k does as much damage as the threshold: k=2
-# over forty candidates means the second line is almost always the second-best
+# over a few dozen candidates means the second line is almost always the
-# noise, arriving with the same confident framing as the first. Halving k
+# second-best noise, arriving with the same confident framing as the first.
-# halves that regardless of where the bar sits.
+# Halving k halves that regardless of where the bar sits.
+#
+# It also BOUNDS the blast radius of widening the pool (below): with k=1 a
+# wider corpus can change WHICH rule surfaces and how often one does, but it
+# can never make a single hint longer. The loudness of one hint and the
+# eligibility of a rule are separate controls, and only one of them moved.
 #
 # It stays a constant because it is a decision about how LOUD one hint may be,
 # not a per-install tuning question. The hint already carries prior art, shape
@@ -140,6 +149,45 @@ RULEHINT_DEFAULT_THRESHOLD = 0.72
 # adds a way to misconfigure the surface (rule 25 cuts both ways).
 RULEHINT_LIMIT = 1
 
+# WHY THE ARMS NO LONGER FILTER TO ONE TIER (#3702).
+#
+# Both arms used to pass `tier="conditional"`, on the reasoning that an
+# always-on rule is already in the session, so surfacing it again is pure
+# noise. That reasoning conflates two different things:
+#
+#   PRESENT IN CONTEXT  — the rule was delivered at session start.
+#   SALIENT AT THE MOMENT  — the rule is in front of the reader when the
+#                            action it governs is about to be taken.
+#
+# A rule handed over in a list at turn zero is present while a session writes
+# a config value three hundred turns later. It is not surfaced. So the filter
+# did not merely skip a redundant hint — it made a whole class of rules
+# permanently ineligible for the only mechanism that puts a rule in front of
+# an agent AT the moment, and the more important a rule is, the more likely
+# it was in that class.
+#
+# The deeper defect is that the filter was doing the THRESHOLD's job. Whether
+# a rule belongs in this hint is a relevance question, and a similarity bar is
+# the control for relevance. A categorical exclusion standing in for a
+# relevance judgment cannot be tuned, cannot be measured, and cannot be wrong
+# in a way anybody notices.
+#
+# THIS IS A MEASURED CHANGE, NOT A SETTLED ONE. The old comment's fear is
+# real — a hint that fires on every write and says obvious things teaches the
+# reader to skip the block, and the surface is then lost along with its true
+# positives. That fear had simply never been checked. `retrieval_logs` already
+# records top_score, result_count and the query for every call, so the
+# evidence now arrives on its own:
+#
+#   - rules clear the bar often and at high scores  -> the fear was justified,
+#     the filter was a crude proxy for a bar set too low, and the WORK IS THE
+#     BAR. Any reinstated filter should then carry a measured reason.
+#   - rules clear rarely, in a thin band near the bar -> the filter was never
+#     the right instrument and relevance was always sufficient.
+#
+# Only the eligibility moved. The bar and k=1 were both left exactly where
+# they were, so the resulting distribution has one cause.
+
 # How much of a command reaches the embedding (#3476). A shell call is not a
 # file: most are short, and the ones that are not are usually a heredoc or a
 # pasted script whose bulk says nothing about which rule applies. The VERB AND
@@ -1205,7 +1253,7 @@ async def build_write_path_hint(
         rule_t0 = time.perf_counter()
         hits = await semantic_search_rules(
             user_id, code or path, limit=RULEHINT_LIMIT,
-            threshold=cfg["rule_threshold"], tier="conditional",
+            threshold=cfg["rule_threshold"],
         )
         rule_ms = (time.perf_counter() - rule_t0) * 1000.0
         fresh = [(score, rule) for score, rule in hits if rule.id not in already]
@@ -1337,7 +1385,7 @@ async def build_tool_rule_hint(
         t0 = time.perf_counter()
         hits = await semantic_search_rules(
             user_id, query, limit=RULEHINT_LIMIT,
-            threshold=cfg["rule_threshold"], tier="conditional",
+            threshold=cfg["rule_threshold"],
         )
         duration_ms = (time.perf_counter() - t0) * 1000.0
 

tests/test_rule_creation_asks_first.py

@@ -0,0 +1,120 @@
+"""Both rule-creation tools run the propose-then-approve loop (#3557).
+
+WHY THIS EXISTS
+
+Every other gate on `create_rule` and `create_project_rule` is about SHAPE:
+is this a rule or a process, is it one thing you could violate, is it general
+enough for a rulebook, is it a near-duplicate. All of those improve a rule
+someone has already decided to write. None of them asks the prior question —
+whether the person the rule will bind has agreed to be bound by it.
+
+That question belongs at the tool, because the tool is the last surface a
+caller reads before the write, and because the write is less reversible than
+it looks. The operator's yes is not merely consent; it is the one moment the
+rule is certainly IN FRONT of them. Afterwards it may not be again for
+months: a conditional rule is not read aloud at session start, and a
+project-scoped rule does not appear in an unfiltered `list_rules()` at all.
+The proposal IS the review, so there had better be one.
+
+WHY IT IS PHRASED AS A PRACTICE AND NOT A PROHIBITION
+
+The first cut of this guidance opened "NOT YOURS TO CALL UNPROMPTED." That is
+the wrong instrument, and the failure it invites is worse than the one it
+prevents: a caller reading a prohibition stops NOTICING rule-shaped things,
+rather than noticing them and asking. The wanted behaviour is more proposals,
+not fewer — spotting that something has hardened into a standing instruction
+is valuable work, and the only step that was ever missing came after it.
+
+So the docstrings describe what to DO: propose readily, state four things,
+close with a question the operator answers in one word. This test is written
+the same way — it asserts the parts of the loop are present, and has nothing
+to say about any wording that forbids.
+
+The fourth element — how the rule would be ENFORCED — is not ceremony. It is
+the part that sometimes dissolves the rule: a thing a test can assert should
+be that test, and a rule is what is left when nothing mechanical can hold it.
+A rulebook grows by default and shrinks only on purpose, so the question that
+prevents a rule earns more than any question that improves one's wording.
+
+WHAT THIS PINS, AND WHAT IT DOES NOT
+
+STRUCTURE, never wording — the same bargain the disambiguator guard (#3123)
+strikes next door. Each element matches a family of synonyms, so the prose
+stays free to be rewritten, reordered or sharpened; only DELETING one fails.
+Pinning phrasing would make every improvement a red build, and a test that
+punishes editing is a test someone deletes.
+
+It cannot tell whether an agent actually proposes. Nothing in a docstring
+can. It catches the regression that really happens: guidance tidied away in
+a later pass by someone who read it as throat-clearing in front of the Args.
+"""
+import pytest
+
+from tests.helpers import tool_doc as _doc
+
+# Both surfaces, because the one that needs it most is the one that looks
+# minor. A project rule is the least visible record the system can hold —
+# absent from an unfiltered list_rules(), and absent from session start too
+# whenever it is conditional — so the surface that writes one carries the
+# larger risk while reading as the smaller act.
+_SURFACES = [
+    ("scribe.mcp.tools.rulebooks", "create_rule"),
+    ("scribe.mcp.tools.rulebooks", "create_project_rule"),
+]
+
+# The loop, element by element, each as a family of ways to say it. A
+# docstring satisfies an element by containing ANY member — that is the room
+# left for rewriting. The families deliberately exclude bare words a
+# docstring would hold by accident ("why", "how", "reason", "rule"), which
+# would let the assertion pass on prose that says nothing of the kind.
+_ELEMENTS = {
+    "the invitation to propose": ("propose", "proposal"),
+    "the operator's approval": ("approve", "approval", "says yes", "a yes"),
+    "the rule's intent": ("intent", "what it changes about how work"),
+    "why it is being proposed now": (
+        "why now", "arose_from_id", "the incident", "prompted it",
+    ),
+    "how it would be enforced": ("enforc",),
+    "the answers offered back": ("as written", "talk about it", "discuss"),
+}
+
+
+@pytest.mark.parametrize(("module", "name"), _SURFACES)
+@pytest.mark.parametrize("element", sorted(_ELEMENTS))
+def test_a_rule_surface_carries_every_part_of_the_proposal_loop(
+    module, name, element
+):
+    """Each element of propose → state four things → ask survives."""
+    doc = _doc(module, name).lower()
+    assert any(token in doc for token in _ELEMENTS[element]), (
+        f"{name}'s docstring no longer mentions {element}. A caller reads "
+        f"this immediately before writing a rule that will bind every future "
+        f"session, and the proposal is the one moment that rule is certain to "
+        f"be seen by the operator. Say it in whatever words you like; this "
+        f"guard only checks it is still said. See create_rule's opening."
+    )
+
+
+@pytest.mark.parametrize(("module", "name"), _SURFACES)
+def test_the_proposal_loop_comes_before_the_parameter_contract(module, name):
+    """It has to be read to work, and the Args: block is where reading stops.
+
+    A caller who has decided to make the call skims down to the parameters.
+    Guidance parked below them — or folded into one argument's description —
+    arrives after the decision it was meant to inform, which is the same as
+    not being there.
+    """
+    doc = _doc(module, name).lower()
+    args_at = doc.find("args:")
+    assert args_at > 0, f"{name}'s docstring has no Args: block"
+    loop_at = min(
+        (doc.find(t) for t in _ELEMENTS["the invitation to propose"]
+         if doc.find(t) >= 0),
+        default=-1,
+    )
+    assert 0 <= loop_at < args_at, (
+        f"{name} introduces the proposal loop at or after its Args: block "
+        f"(loop {loop_at}, args {args_at}). Move it to the opening — a "
+        f"caller who has already decided to write the rule reads the "
+        f"parameters, not the prose under them."
+    )

tests/test_rule_usage_wiring.py

@@ -160,7 +160,15 @@ async def test_the_arm_searches_on_its_OWN_bar_not_the_code_one():
     kw = search.await_args.kwargs
     assert kw["threshold"] == 0.81, "the arm is still using the code threshold"
     assert kw["limit"] == pc.RULEHINT_LIMIT
-    assert kw["tier"] == "conditional"
+    # NO tier filter (#3702). The arms search every rule the caller owns,
+    # because "already in the session" is not the same as "in front of the
+    # reader at the moment it applies" — and relevance is the threshold's
+    # job, not a category's. If this assertion is failing because a tier
+    # argument came back, read the block above RULEHINT_LIMIT first: the
+    # filter may legitimately return, but only carrying a measured reason.
+    assert "tier" not in kw or kw["tier"] is None, (
+        "the arm is filtering the rule corpus by tier again"
+    )
 
 
 @pytest.mark.asyncio