feat(rules): a slot a preference cannot lose (#3894)

Milestone 399 step 4. A rule and a preference are not equally served by one
ranking, because their losses are not equal:

  - a RULE crowded out at the prompt boundary still fires at an act arm. A
    push reaches pre_tool_rule, a write reaches write_path_rule. The prompt
    hit is a preview of a second chance.
  - a PREFERENCE about how to answer has no second chance. The response IS
    the act, so crowded out there it is never delivered at all.

A straight ranking therefore favours the record whose loss is recoverable
over the one whose loss is total, and does it INVISIBLY: the rule that won is
a legitimate hit, the telemetry reads healthy, and the only symptom is a
preference that quietly never arrives. reuse_slot exists for the same shape
one corpus over (#2463).

`semantic_search_rules` gains a `kind` filter, so the slot's query can only
answer with what the slot is for. Verifying afterwards would be weaker — an
unfiltered search that happened to return a rule would spend the slot on it,
and that line would be indistinguishable from one that earned its place.

THE SLOT BUYS POSITION, NOT A LOWER BAR, matching reuse_slot. A weak
preference cannot buy it, so silence stays the default. The task asked for a
separate threshold; I did not add one, and the reason is that the worry
behind it — reading a miss rate as a fact about preferences — is answered by
`preference_slot` being its own logged source, where best_available_id names
which preference was refused. A knob added on a guess is a way to
misconfigure the surface; a bar moved on evidence is an argument. The
evidence arrives on its own now.

IT EXTENDS, IT NEVER DISPLACES — and here it parts from reuse_slot, which
evicts its menu's weakest hit. A displaced hit sits in prompt_rule's
retrieval_logs row while never being surfaced, so that source's two tables
stop agreeing and #3668's identity breaks for a reason nothing in the data
explains. Milestone #379 is what losing that identity costs: five steps
planned against two counters disagreeing, not a write path dropping rows. One
extra line in a rare case is the cheaper price.

It also runs BEFORE the bail-out. An empty general result is not proof no
preference qualifies: that search overfetches by distance then collapses, so
a preference ranked below the window is invisible to it while a kind-filtered
query finds it at once. Bailing first would make the slot dead in exactly the
corpus it exists for.

One existing assertion repinned from a bare call_count to a per-source
filter: the slot logs its own query on the same call, and a count would pin
the number of arms rather than the property — going red the next time one is
added, which is rule 167's false alarm about the thing it protects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ

src/scribe/services/embeddings.py

@@ -818,6 +818,7 @@ async def semantic_search_rules(
     limit: int = 5,
     threshold: float = _SIMILARITY_THRESHOLD,
     tier: str | None = None,
+    kind: str | None = None,
     report: dict | None = None,
 ) -> list[tuple[float, "Rule"]]:
     """Return up to *limit* (score, rule) pairs most relevant to *query*.
@@ -855,6 +856,16 @@ async def semantic_search_rules(
     Pass a tier when a caller genuinely wants one class — a listing, an audit,
     a UI that renders the tiers apart. Not to approximate relevance.
 
+    `kind` narrows to `rule` or `preference`, and NONE is likewise the ordinary
+    case: a caller asking "what governs this" wants both, because the reader
+    needs to know what binds AND how the operator wants it done. The one place
+    it is passed is a RESERVED SLOT — a query that may only return a
+    preference, so the slot cannot be spent on something else. That is the
+    same reason `note_type` exists on the sibling search, and the same failure
+    it prevents: a slot silently filled by the wrong kind is worse than no
+    slot, because the line is indistinguishable from one that earned its place
+    on score.
+
     Collapses to best-chunk-per-rule like the note search, so a long rule split
     across chunks competes once rather than crowding the results with itself.
 
@@ -895,6 +906,7 @@ async def semantic_search_rules(
                         Project.user_id == user_id,
                     ),
                     *( [Rule.tier == tier] if tier else [] ),
+                    *( [Rule.kind == kind] if kind else [] ),
                 )
                 # Overfetch so collapsing chunks to their best row still fills
                 # the page — the same reason the note search overfetches.

src/scribe/services/plugin_context.py

@@ -678,6 +678,109 @@ async def build_autoinject_hint(
     return {"context": "\n".join(lines), "note_ids": note_ids, "config": cfg}
 
 
+async def _reserve_slot_for_preference(
+    user_id: int,
+    query: str,
+    hits: list,
+    *,
+    threshold: float,
+    project_id: int,
+    already: set[int],
+) -> tuple[list, int | None]:
+    """Guarantee a preference one slot, if one clears the bar (#3894).
+
+    THE ASYMMETRY THIS EXISTS FOR. A rule and a preference are not equally
+    served by a shared score contest, because their losses are not equal:
+
+      - a RULE crowded out here can still fire at the act arm. A `git push`
+        reaches `pre_tool_rule`, a file write reaches `write_path_rule`. The
+        prompt hit is a preview of a second chance.
+      - a PREFERENCE about how to answer has no second chance. There is no
+        later act — the response IS the act — so crowded out here it is never
+        delivered at all.
+
+    A straight ranking therefore favours the record whose loss is recoverable
+    over the one whose loss is total, and it does so INVISIBLY: the rule that
+    won is a legitimate hit, the telemetry looks healthy, and the only symptom
+    is a preference that quietly never arrives. `reuse_slot` exists for the
+    same shape one corpus over (#2463), where snippets kept losing to project
+    records that merely resembled the query.
+
+    THE SLOT BUYS POSITION, NOT A LOWER BAR — same as `reuse_slot`, which also
+    reserves at `cfg["threshold"]`. A weak preference cannot buy the slot, so
+    silence stays the default and the reserved line is never worse than the
+    ones it sits beside. If `preference_slot` later shows a stream of
+    near-misses, `best_available_id` (#3807) names which preference was
+    refused and a separate bar becomes an argument with evidence behind it
+    rather than a knob added on a guess.
+
+    LEDGER REPEATS STILL COUNT AS REPRESENTED. A preference already on the
+    session's ledger occupies the slot rather than being skipped for a fresh
+    one: it is still rendered (#3750), just with the tail that says so, and a
+    preference is the kind of record where being reminded is the point.
+
+    Returns the possibly-extended hit list, and the id the slot spent — the
+    caller needs that to keep each source's surfaced set matching its own log
+    row (#3668), since the slot logs under its own name.
+    """
+    if any(rule.kind == "preference" for _s, rule in hits):
+        return hits, None
+
+    _t0 = time.perf_counter()
+    _rep: dict = {}
+    # KIND-FILTERED, so the query can only answer with what the slot is for.
+    # Verifying the kind afterwards would be weaker: an unfiltered search that
+    # happened to return a rule would spend the slot on it, and the line would
+    # be indistinguishable from one that earned its place.
+    found = await semantic_search_rules(
+        user_id, query, limit=1, threshold=threshold,
+        kind="preference", report=_rep,
+    )
+    fresh = [(s, r) for s, r in found if r.id not in already]
+    # ITS OWN SOURCE, and both sides of the trade logged. #2463's own finding
+    # is the warning rather than the precedent here: the hit that slot pushed
+    # OUT was in retrieval_logs while the query that pushed it out was not, so
+    # the slot could never be judged against what it displaced. `results` is
+    # fresh-only, matching what gets recorded as surfaced below (#3752/#3668).
+    record_retrieval(
+        user_id=user_id, source="preference_slot", query=query,
+        threshold=threshold, limit=1, project_id=project_id,
+        is_task=None, results=fresh,
+        best_available=_rep.get("best_available_score"),
+        best_available_id=_rep.get("best_available_id"),
+        searched=bool(_rep.get("searched", True)),
+        suppressed=len(found) - len(fresh),
+        duration_ms=(time.perf_counter() - _t0) * 1000.0,
+    )
+    seen = {rule.id for _s, rule in hits}
+    slot = [(s, r) for s, r in found
+            if r.kind == "preference" and r.id not in seen][:1]
+    if not slot:
+        return hits, None
+
+    slot_id = int(slot[0][1].id)
+    if slot_id not in already:
+        record_rule_surfaced(
+            user_id=user_id, rule_ids=[slot_id], source="preference_slot",
+        )
+    # IT EXTENDS, IT NEVER DISPLACES — and here it parts company with
+    # `reuse_slot`, which evicts its menu's weakest hit. The reason is the
+    # ledger rather than taste. A displaced hit was RETURNED by the general
+    # search and is sitting in that call's `retrieval_logs` row, but would not
+    # have been shown — so `prompt_rule`'s surfaced set would stop matching
+    # its own log row, and #3668's identity would break for a reason nothing
+    # in the data explains. That identity is the cheapest true statement
+    # available about this pair of tables, and milestone #379 is what it costs
+    # to lose it: five steps planned against a gap that was two counters
+    # disagreeing, not a write path dropping rows.
+    #
+    # The price is one extra line, only when the general search already filled
+    # the limit AND a preference cleared the bar without placing. Cheap, and
+    # it buys a surface whose two tables can always be checked against each
+    # other.
+    return hits + slot, slot_id
+
+
 async def build_prompt_rule_hint(
     user_id: int,
     query: str,
@@ -758,6 +861,18 @@ async def build_prompt_rule_hint(
             searched=bool(_rep.get("searched", True)),
             suppressed=len(hits) - len(fresh),
         )
+        # THE RESERVED SLOT RUNS BEFORE THE BAIL-OUT, and that ordering is
+        # load-bearing rather than tidy. An empty general result is not proof
+        # that no preference qualifies: the general search overfetches by
+        # distance and then collapses, so a preference ranked below that
+        # window is invisible to it while a kind-filtered query finds it at
+        # once. Bailing first would make the slot dead in exactly the corpus
+        # it exists for — one where rules outnumber preferences.
+        hits, slot_id = await _reserve_slot_for_preference(
+            user_id, q, hits, threshold=threshold,
+            project_id=project_id, already=already,
+        )
+
         # `hits`, not `fresh` (#3750): a call whose only hit is a repeat still
         # has something to say, it just says it differently.
         if not hits:
@@ -770,6 +885,11 @@ async def build_prompt_rule_hint(
         # FRESH-ONLY (#3752). A reference is a rendering decision, not a
         # retrieval outcome, and counting one here would inflate the
         # denominator pull_through is read from.
+        #
+        # `fresh` is the PRE-SLOT list on purpose: it is exactly what this
+        # call's own `retrieval_logs` row recorded, so the two stay equal
+        # (#3668). The slot's hit is surfaced under `preference_slot` by the
+        # helper, against that source's own row.
         rule_ids = [rule.id for _score, rule in fresh]
 
         # RANKED, not ambient: this arm chose what it showed. The name is also

src/scribe/services/rule_usage.py

@@ -97,7 +97,15 @@ logger = logging.getLogger(__name__)
 # surfacing is a claim ("this rule may apply to what you are doing") that a pull
 # can confirm or refute, while an ambient one is a delivery nobody decided on.
 # Add a source here only when a ranker picked it.
-RANKED_SOURCES = ("write_path_rule", "pre_tool_rule", "prompt_rule")
+RANKED_SOURCES = (
+    "write_path_rule", "pre_tool_rule", "prompt_rule",
+    # A reserved slot is a ranker's choice twice over — it ran a query AND
+    # decided a kind was worth guaranteeing a place. Left out, its line would
+    # be counted as bulk delivery and drop out of the denominator, so the one
+    # surface built because a record class kept losing would be the one whose
+    # hits nobody could confirm.
+    "preference_slot",
+)
 
 
 def is_ambient(source: str) -> bool: