feat(409): an operator's own reply shapes reach the reply they are about (#4013)

The reporting-back skill ships default shapes; an operator's adjustments to
them are preference records. Prompt-time retrieval matches the operator's
message, and a shape preference is about the reply, so those preferences were
on file and never arrived. Operator's decision (logged on #4013): the server
delivers them for a completion report, and the skill asks for every other kind.

- Completion reports (option C): closing a task with update_task runs a
  kind-filtered preference search for the moment "writing the completion
  report after finishing a task" and returns matches as `reply_preferences`
  ({id, title, statement, kind}), with a sentence added to `report_back`
  naming the key. A preference says it is about completion reports through
  its own when_to_apply; no tag or column. Omitted when nothing matches, and
  the lookup fails open.
- Telemetry: every call logs to retrieval_logs under `report_preference`
  (empty calls included; a search that never ran writes no row) and hits are
  recorded surfaced. The source is ranked, so it counts toward pull-through.
  The bar is the prompt arm's setting until step 6 reads this source's near
  misses.
- Every other reply (option A): reporting-back gains "The operator's own
  shapes come first". Before a finding, decision, handoff or "where are we",
  search(content_type="rule") in the words of that moment and follow what
  comes back. Registered in the ownership guard with reporting-back as owner.
- Loading reply shapes at session start (option B) was rejected: it would be
  a small copy of the preloading milestone 394 retired.

Domain-neutral query (pinned); works on an install with no preferences.
Plugin version minted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/scribe/mcp/tools/tasks.py

@@ -31,6 +31,7 @@ from scribe.services.notes import minted_kind
 from scribe.services import placement as placement_svc
 from scribe.services import planning as planning_svc
 from scribe.services import record_batch as batch_svc
+from scribe.services import reply_preferences as reply_prefs_svc
 from scribe.services import rulebooks as rulebooks_svc
 from scribe.services import systems as systems_svc
 from scribe.services import task_logs as task_logs_svc
@@ -273,7 +274,13 @@ async def update_task(
     reads exactly like a real one when it is wrong.
 
     Closing a task (done or cancelled) also returns `report_back`: a one-line
-    reminder of what the reply to the operator should cover.
+    reminder of what the reply to the operator should cover. When the
+    operator has preferences for how a completion report is written, they
+    come back as `reply_preferences` ({id, title, statement, kind}) — found
+    by their `when_to_apply`, so a preference whose trigger is writing the
+    report after finishing a task is the one that arrives here. Where one
+    differs from the default shape, the preference is what the operator
+    asked for.
     """
     uid = current_user_id()
     fields: dict = {}
@@ -314,6 +321,14 @@ async def update_task(
     await placement_svc.attach_placement(uid, data, note)
     if status in _CLOSING_STATUSES:
         data["report_back"] = REPORT_BACK_CUE
+        # The operator's own adjustments to the completion report, retrieved
+        # at the one moment a server can see that report coming (milestone
+        # 409 step 4). Omitted rather than sent empty, like every decoration.
+        prefs = await reply_prefs_svc.completion_preferences(
+            uid, project_id=getattr(note, "project_id", None))
+        if prefs:
+            data["reply_preferences"] = prefs
+            data["report_back"] = REPORT_BACK_CUE + " " + REPLY_PREFERENCES_CUE
     return data
 
 
@@ -360,6 +375,12 @@ REPORT_BACK_CUE = (
     "Reporting this to the operator? Say where it sits (from `placement`), "
     "what now works, what needs them, and what comes next."
 )
+# Appended only when `reply_preferences` is present, so the key never arrives
+# unexplained and a session with no preferences reads exactly what it did.
+REPLY_PREFERENCES_CUE = (
+    "The operator has preferences for how this report is written — "
+    "follow `reply_preferences` over the default shape where they differ."
+)
 
 _ITEM_KEYS = {"title", "body", "type", "status", "priority", "kind", "tags", "system_ids"}
 

src/scribe/services/reply_preferences.py

@@ -0,0 +1,135 @@
+"""The operator's own preferences for a completion report, at the moment it is written.
+
+WHY THIS EXISTS (milestone 409 step 4)
+
+The reporting-back skill ships DEFAULT shapes. An operator will want some of
+them different ("my completion reports also say how it was tested", "decisions
+as a numbered list"), and those adjustments are `preference` records. The gap
+is the query: prompt-time retrieval matches the OPERATOR'S MESSAGE, and a shape
+preference is about the REPLY. "Fix the flaky test" never retrieves "completion
+reports should say how it was tested", so the preference is on file and never
+arrives.
+
+THE DECISION (operator, 2026-09-14, logged on the step): two deliveries, split
+by reply kind.
+
+  - A COMPLETION REPORT has a moment the server can see — a task closing — so
+    the server retrieves for it and hands the matches back beside
+    `report_back`. That is this module.
+  - EVERY OTHER REPLY KIND (a finding, a decision, a handoff…) has no tool call
+    in front of it, so the reporting-back skill asks: it tells the agent to
+    `search(content_type="rule")` for that kind before writing.
+
+Loading reply-shape preferences at session start was the rejected third option:
+it is a small copy of the preloading milestone 394 retired, and just as
+unmeasurable.
+
+HOW A PREFERENCE SAYS IT IS ABOUT COMPLETION REPORTS
+
+By its trigger, which is what it already has — no tag, no new column. A
+preference's `when_to_apply` dominates its embedded document, so one written
+for this moment ("writing the report after finishing a task") resembles
+COMPLETION_QUERY below, and one about anything else does not. That keeps
+delivery entirely in retrieval, as 394 decided, and leaves an operator nothing
+new to learn: a preference reaches the completion report the same way every
+other record reaches its moment.
+
+THE BAR IS THE PROMPT ARM'S, AND IT IS NOT YET EARNED HERE
+
+A fixed query against triggers is a different score distribution from an
+operator's message against the same documents. Starting at the prompt arm's
+setting is the value with evidence behind it, and an operator's tuning of that
+bar reaches this too. Every call logs under its own source,
+`report_preference`, so step 6 can read this surface's near misses apart from
+the prompt arm's before anyone moves the number.
+"""
+from __future__ import annotations
+
+import logging
+import time
+
+from scribe.services.embeddings import semantic_search_rules
+from scribe.services.plugin_context import (
+    PROMPTRULE_DEFAULT_THRESHOLD,
+    PROMPTRULE_THRESHOLD_KEY,
+)
+from scribe.services.retrieval_telemetry import record_retrieval
+from scribe.services.rule_usage import record_rule_surfaced
+from scribe.services.settings import get_setting
+
+logger = logging.getLogger(__name__)
+
+SOURCE = "report_preference"
+
+# Written in the vocabulary of the MOMENT, because that is what a trigger is
+# written in and what this query is scored against. Domain-neutral on purpose
+# (rule #115): a writing project or a home-infrastructure project closes tasks
+# too, and its operator's preferences must match as well as a developer's.
+COMPLETION_QUERY = (
+    "writing the completion report to the operator after finishing a task — "
+    "how that reply should be laid out and what it should include"
+)
+
+# A handful, not a menu. More than a few shape preferences for ONE kind of
+# reply would contradict each other before they helped; the limit is here to
+# keep one noisy corpus from turning a status change into a wall of text.
+LIMIT = 3
+
+
+async def _threshold(user_id: int) -> float:
+    try:
+        value = float(await get_setting(
+            user_id, PROMPTRULE_THRESHOLD_KEY, str(PROMPTRULE_DEFAULT_THRESHOLD)))
+    except (TypeError, ValueError):
+        value = PROMPTRULE_DEFAULT_THRESHOLD
+    return min(1.0, max(0.0, value))
+
+
+async def completion_preferences(user_id: int, *, project_id: int | None = None) -> list[dict]:
+    """The operator's preferences for a completion report, best match first.
+
+    KIND-FILTERED, for the reason `_reserve_slot_for_preference` gives: a
+    binding rule that happened to resemble the query would otherwise ride out
+    under a key that says "how the operator likes this written", which is a
+    claim about force the record does not make.
+
+    Every call is logged, the empty ones included: a surface that records only
+    the calls it liked reports a flawless clear-rate however badly its bar is
+    set. An install with no preferences at all logs nothing, because no search
+    ran (`searched` stays False) — that is not a decline.
+
+    Fails open to an empty list: this decorates a write that has already
+    happened, and a lookup that errors must not turn it into a failure.
+    """
+    try:
+        threshold = await _threshold(user_id)
+        report: dict = {}
+        t0 = time.perf_counter()
+        hits = await semantic_search_rules(
+            user_id, COMPLETION_QUERY, limit=LIMIT, threshold=threshold,
+            kind="preference", report=report,
+        )
+        hits = [(score, rule) for score, rule in hits if rule.kind == "preference"]
+        record_retrieval(
+            user_id=user_id, source=SOURCE, query=COMPLETION_QUERY,
+            threshold=threshold, limit=LIMIT, project_id=project_id,
+            is_task=None, results=hits,
+            best_available=report.get("best_available_score"),
+            best_available_id=report.get("best_available_id"),
+            searched=bool(report.get("searched", True)),
+            duration_ms=(time.perf_counter() - t0) * 1000.0,
+        )
+        if not hits:
+            return []
+        # RANKED: this surface chose what it showed, so the name is in
+        # rule_usage.RANKED_SOURCES and its hits count toward pull-through.
+        record_rule_surfaced(
+            user_id=user_id, rule_ids=[rule.id for _s, rule in hits], source=SOURCE,
+        )
+        return [
+            {"id": rule.id, "title": rule.title, "statement": rule.statement, "kind": "preference"}
+            for _score, rule in hits
+        ]
+    except Exception:  # noqa: BLE001 - a decoration never breaks its payload
+        logger.warning("completion preference lookup failed", exc_info=True)
+        return []

src/scribe/services/rule_usage.py

@@ -105,6 +105,9 @@ RANKED_SOURCES = (
     # surface built because a record class kept losing would be the one whose
     # hits nobody could confirm.
     "preference_slot",
+    # The completion-report lookup on update_task (milestone 409 step 4). It
+    # runs its own query and shows only what cleared the bar — a ranker.
+    "report_preference",
 )