Reply shapes reach the reply, a Stop hook checks completion reports, and hook pipes keep their output #156

Merged
bvandeusen merged 6 commits from dev into main 2026-09-14 18:57:07 -04:00
8 changed files with 294 additions and 8 deletions
Showing only changes of commit 921565696c - Show all commits
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "scribe",
"description": "Scribe for Claude Code: connects the scribe MCP server, adds the hooks that deliver live project state and relevant records at the right moment, ships the shared client-neutral Scribe skills (using-scribe, writing-plans, reporting-back, systematic-debugging, verification, brainstorming, reusing-code, shape-accounting), and syncs your saved Scribe Processes as skills (/scribe:sync).",
"version": "2026.09.14.2001",
"version": "2026.09.14.2143",
"author": {
"name": "Bryan Van Deusen"
},
+18
View File
@@ -41,6 +41,24 @@ it is wrong. So take placement from Scribe:
- Work with no task behind it: say so plainly — "this wasn't tracked as a
task" — and offer to record it. An honest "untracked" is a placement too.
## The operator's own shapes come first
The shapes below are defaults. An operator may have changed some of them — a
section they always want, an order they read faster, a kind of reply they want
shorter — and those changes are `preference` records. Where a preference and a
default differ, the preference is what they asked for.
- **A completion report brings its preferences with it.** Closing a task with
`update_task` returns them as **`reply_preferences`** when the operator has
any; the `report_back` line says so. Nothing to search for.
- **Every other reply, ask before writing it.** A finding, a decision, a
handoff, a "where are we" — no tool call comes before these, so nothing
hands their preferences over. Once you know which kind of reply you are
writing, `search(content_type="rule")` for it in the words of that moment —
"writing a decision for the operator", "handing off to the operator" — and
follow any preference that comes back. Nothing coming back means the default
shape stands.
## Reports — work happened
| Kind | Sections |
+22 -1
View File
@@ -31,6 +31,7 @@ from scribe.services.notes import minted_kind
from scribe.services import placement as placement_svc
from scribe.services import planning as planning_svc
from scribe.services import record_batch as batch_svc
from scribe.services import reply_preferences as reply_prefs_svc
from scribe.services import rulebooks as rulebooks_svc
from scribe.services import systems as systems_svc
from scribe.services import task_logs as task_logs_svc
@@ -273,7 +274,13 @@ async def update_task(
reads exactly like a real one when it is wrong.
Closing a task (done or cancelled) also returns `report_back`: a one-line
reminder of what the reply to the operator should cover.
reminder of what the reply to the operator should cover. When the
operator has preferences for how a completion report is written, they
come back as `reply_preferences` ({id, title, statement, kind}) — found
by their `when_to_apply`, so a preference whose trigger is writing the
report after finishing a task is the one that arrives here. Where one
differs from the default shape, the preference is what the operator
asked for.
"""
uid = current_user_id()
fields: dict = {}
@@ -314,6 +321,14 @@ async def update_task(
await placement_svc.attach_placement(uid, data, note)
if status in _CLOSING_STATUSES:
data["report_back"] = REPORT_BACK_CUE
# The operator's own adjustments to the completion report, retrieved
# at the one moment a server can see that report coming (milestone
# 409 step 4). Omitted rather than sent empty, like every decoration.
prefs = await reply_prefs_svc.completion_preferences(
uid, project_id=getattr(note, "project_id", None))
if prefs:
data["reply_preferences"] = prefs
data["report_back"] = REPORT_BACK_CUE + " " + REPLY_PREFERENCES_CUE
return data
@@ -360,6 +375,12 @@ REPORT_BACK_CUE = (
"Reporting this to the operator? Say where it sits (from `placement`), "
"what now works, what needs them, and what comes next."
)
# Appended only when `reply_preferences` is present, so the key never arrives
# unexplained and a session with no preferences reads exactly what it did.
REPLY_PREFERENCES_CUE = (
"The operator has preferences for how this report is written — "
"follow `reply_preferences` over the default shape where they differ."
)
_ITEM_KEYS = {"title", "body", "type", "status", "priority", "kind", "tags", "system_ids"}
+135
View File
@@ -0,0 +1,135 @@
"""The operator's own preferences for a completion report, at the moment it is written.
WHY THIS EXISTS (milestone 409 step 4)
The reporting-back skill ships DEFAULT shapes. An operator will want some of
them different ("my completion reports also say how it was tested", "decisions
as a numbered list"), and those adjustments are `preference` records. The gap
is the query: prompt-time retrieval matches the OPERATOR'S MESSAGE, and a shape
preference is about the REPLY. "Fix the flaky test" never retrieves "completion
reports should say how it was tested", so the preference is on file and never
arrives.
THE DECISION (operator, 2026-09-14, logged on the step): two deliveries, split
by reply kind.
- A COMPLETION REPORT has a moment the server can see — a task closing — so
the server retrieves for it and hands the matches back beside
`report_back`. That is this module.
- EVERY OTHER REPLY KIND (a finding, a decision, a handoff…) has no tool call
in front of it, so the reporting-back skill asks: it tells the agent to
`search(content_type="rule")` for that kind before writing.
Loading reply-shape preferences at session start was the rejected third option:
it is a small copy of the preloading milestone 394 retired, and just as
unmeasurable.
HOW A PREFERENCE SAYS IT IS ABOUT COMPLETION REPORTS
By its trigger, which is what it already has — no tag, no new column. A
preference's `when_to_apply` dominates its embedded document, so one written
for this moment ("writing the report after finishing a task") resembles
COMPLETION_QUERY below, and one about anything else does not. That keeps
delivery entirely in retrieval, as 394 decided, and leaves an operator nothing
new to learn: a preference reaches the completion report the same way every
other record reaches its moment.
THE BAR IS THE PROMPT ARM'S, AND IT IS NOT YET EARNED HERE
A fixed query against triggers is a different score distribution from an
operator's message against the same documents. Starting at the prompt arm's
setting is the value with evidence behind it, and an operator's tuning of that
bar reaches this too. Every call logs under its own source,
`report_preference`, so step 6 can read this surface's near misses apart from
the prompt arm's before anyone moves the number.
"""
from __future__ import annotations
import logging
import time
from scribe.services.embeddings import semantic_search_rules
from scribe.services.plugin_context import (
PROMPTRULE_DEFAULT_THRESHOLD,
PROMPTRULE_THRESHOLD_KEY,
)
from scribe.services.retrieval_telemetry import record_retrieval
from scribe.services.rule_usage import record_rule_surfaced
from scribe.services.settings import get_setting
logger = logging.getLogger(__name__)
SOURCE = "report_preference"
# Written in the vocabulary of the MOMENT, because that is what a trigger is
# written in and what this query is scored against. Domain-neutral on purpose
# (rule #115): a writing project or a home-infrastructure project closes tasks
# too, and its operator's preferences must match as well as a developer's.
COMPLETION_QUERY = (
"writing the completion report to the operator after finishing a task — "
"how that reply should be laid out and what it should include"
)
# A handful, not a menu. More than a few shape preferences for ONE kind of
# reply would contradict each other before they helped; the limit is here to
# keep one noisy corpus from turning a status change into a wall of text.
LIMIT = 3
async def _threshold(user_id: int) -> float:
try:
value = float(await get_setting(
user_id, PROMPTRULE_THRESHOLD_KEY, str(PROMPTRULE_DEFAULT_THRESHOLD)))
except (TypeError, ValueError):
value = PROMPTRULE_DEFAULT_THRESHOLD
return min(1.0, max(0.0, value))
async def completion_preferences(user_id: int, *, project_id: int | None = None) -> list[dict]:
"""The operator's preferences for a completion report, best match first.
KIND-FILTERED, for the reason `_reserve_slot_for_preference` gives: a
binding rule that happened to resemble the query would otherwise ride out
under a key that says "how the operator likes this written", which is a
claim about force the record does not make.
Every call is logged, the empty ones included: a surface that records only
the calls it liked reports a flawless clear-rate however badly its bar is
set. An install with no preferences at all logs nothing, because no search
ran (`searched` stays False) — that is not a decline.
Fails open to an empty list: this decorates a write that has already
happened, and a lookup that errors must not turn it into a failure.
"""
try:
threshold = await _threshold(user_id)
report: dict = {}
t0 = time.perf_counter()
hits = await semantic_search_rules(
user_id, COMPLETION_QUERY, limit=LIMIT, threshold=threshold,
kind="preference", report=report,
)
hits = [(score, rule) for score, rule in hits if rule.kind == "preference"]
record_retrieval(
user_id=user_id, source=SOURCE, query=COMPLETION_QUERY,
threshold=threshold, limit=LIMIT, project_id=project_id,
is_task=None, results=hits,
best_available=report.get("best_available_score"),
best_available_id=report.get("best_available_id"),
searched=bool(report.get("searched", True)),
duration_ms=(time.perf_counter() - t0) * 1000.0,
)
if not hits:
return []
# RANKED: this surface chose what it showed, so the name is in
# rule_usage.RANKED_SOURCES and its hits count toward pull-through.
record_rule_surfaced(
user_id=user_id, rule_ids=[rule.id for _s, rule in hits], source=SOURCE,
)
return [
{"id": rule.id, "title": rule.title, "statement": rule.statement, "kind": "preference"}
for _score, rule in hits
]
except Exception: # noqa: BLE001 - a decoration never breaks its payload
logger.warning("completion preference lookup failed", exc_info=True)
return []
+3
View File
@@ -105,6 +105,9 @@ RANKED_SOURCES = (
# surface built because a record class kept losing would be the one whose
# hits nobody could confirm.
"preference_slot",
# The completion-report lookup on update_task (milestone 409 step 4). It
# runs its own query and shows only what cleared the bar — a ranker.
"report_preference",
)
+3
View File
@@ -152,6 +152,9 @@ TOPICS: tuple[Topic, ...] = (
"prior art offered beside a write is not noise", index=("create_snippet",)),
Topic("report back where the work stands", "skill:reporting-back", ("reporting-back", "placement"),
"take the placement from the record", index=("placement",)),
Topic("the operator's own reply shapes come first", "skill:reporting-back",
("reply_preferences", 'content_type="rule"'),
"the operator's own shapes come first"),
# ── per-tool contracts and in-band behaviour — owned by the server ──
Topic("closing a task cues the report", "docstrings", ("report_back",), "reporting this to the operator?"),
Topic("a note that asserts a fact carries its check", "docstrings", ("verify_with", "expires_when"),
+31 -6
View File
@@ -3,6 +3,9 @@
The in-band half of milestone 409 step 3: the skill and static context only
exist in the Claude Code plugin, and a tool response reaches every MCP client
at the moment a piece of work closes. Pinned on the response, not the wording.
Step 4 adds the operator's own completion-report preferences beside the cue,
retrieved at that same moment — and only then, and only when there are any.
"""
from unittest.mock import AsyncMock, MagicMock, patch
@@ -10,24 +13,46 @@ import pytest
pytestmark = pytest.mark.usefixtures("_bind_user")
_PREF = {"id": 41, "title": "Say how it was checked", "statement": "", "kind": "preference"}
async def _update(**kwargs):
async def _update(prefs=None, **kwargs):
from scribe.mcp.tools.tasks import update_task
note = MagicMock(id=5, user_id=7, project_id=None)
note = MagicMock(id=5, user_id=7, project_id=3)
note.to_dict.return_value = {"id": 5}
lookup = AsyncMock(return_value=list(prefs or []))
with patch("scribe.mcp.tools.tasks.notes_svc.update_note", AsyncMock(return_value=note)), \
patch("scribe.mcp.tools.tasks.systems_tools.attach_systems", AsyncMock()), \
patch("scribe.mcp.tools.tasks.placement_svc.attach_placement", AsyncMock()):
return await update_task(task_id=5, **kwargs)
patch("scribe.mcp.tools.tasks.placement_svc.attach_placement", AsyncMock()), \
patch("scribe.mcp.tools.tasks.reply_prefs_svc.completion_preferences", lookup):
return await update_task(task_id=5, **kwargs), lookup
@pytest.mark.parametrize("status", ["done", "cancelled"])
async def test_closing_a_task_carries_the_cue(status):
out = await _update(status=status)
out, _ = await _update(status=status)
assert "placement" in out["report_back"] and "needs them" in out["report_back"]
@pytest.mark.parametrize("kwargs", [{"status": "in_progress"}, {"status": "todo"}, {"body": "more notes"}])
async def test_other_updates_do_not(kwargs):
assert "report_back" not in await _update(**kwargs)
out, lookup = await _update(prefs=[_PREF], **kwargs)
assert "report_back" not in out and "reply_preferences" not in out
lookup.assert_not_awaited()
async def test_closing_hands_back_the_operators_report_preferences():
out, lookup = await _update(prefs=[_PREF], status="done")
assert out["reply_preferences"] == [_PREF]
# The cue says the key is there, so it never arrives unexplained.
assert "reply_preferences" in out["report_back"]
lookup.assert_awaited_once_with(7, project_id=3)
async def test_no_preferences_means_no_key_and_the_plain_cue():
from scribe.mcp.tools.tasks import REPORT_BACK_CUE
out, _ = await _update(prefs=[], status="done")
assert "reply_preferences" not in out
assert out["report_back"] == REPORT_BACK_CUE
+81
View File
@@ -0,0 +1,81 @@
"""The completion-report preference lookup (milestone 409 step 4).
What it pins: the lookup asks for PREFERENCES only, logs every call under its
own source (the empty ones too), counts only what it showed as surfaced, and
fails open. The query stays domain-neutral, because every kind of project
closes tasks.
"""
import re
from unittest.mock import AsyncMock, MagicMock, patch
M = "scribe.services.reply_preferences"
def _rule(rid, kind="preference"):
return MagicMock(id=rid, kind=kind, title=f"t{rid}", statement=f"s{rid}")
async def _run(hits, *, searched=True, raises=None):
from scribe.services.reply_preferences import completion_preferences
async def search(user_id, query, **kw):
if raises:
raise raises
kw["report"].update({"searched": searched, "best_available_score": 0.7})
return hits
search_mock = AsyncMock(side_effect=search)
with patch(f"{M}.semantic_search_rules", search_mock), \
patch(f"{M}.get_setting", AsyncMock(return_value="0.72")), \
patch(f"{M}.record_retrieval") as logged, \
patch(f"{M}.record_rule_surfaced") as surfaced:
out = await completion_preferences(7, project_id=3)
return out, search_mock, logged, surfaced
async def test_asks_for_preferences_only_and_returns_them_best_first():
out, search, logged, surfaced = await _run([(0.9, _rule(1)), (0.8, _rule(2))])
assert search.await_args.kwargs["kind"] == "preference"
assert [p["id"] for p in out] == [1, 2]
assert all(p["kind"] == "preference" for p in out)
assert logged.call_args.kwargs["source"] == "report_preference"
assert surfaced.call_args.kwargs == {"user_id": 7, "rule_ids": [1, 2], "source": "report_preference"}
async def test_a_rule_that_slips_through_is_not_handed_back_as_a_preference():
out, _, _, surfaced = await _run([(0.9, _rule(1, kind="rule")), (0.8, _rule(2))])
assert [p["id"] for p in out] == [2]
assert surfaced.call_args.kwargs["rule_ids"] == [2]
async def test_an_empty_call_is_still_logged_and_surfaces_nothing():
out, _, logged, surfaced = await _run([])
assert out == []
logged.assert_called_once()
assert logged.call_args.kwargs["results"] == []
surfaced.assert_not_called()
async def test_a_search_that_never_ran_says_so_to_the_log():
_, _, logged, _ = await _run([], searched=False)
assert logged.call_args.kwargs["searched"] is False
async def test_fails_open():
out, _, _, surfaced = await _run([], raises=RuntimeError("embedder down"))
assert out == []
surfaced.assert_not_called()
def test_it_is_a_ranked_source():
from scribe.services.rule_usage import is_ambient
assert not is_ambient("report_preference")
def test_the_query_assumes_no_particular_domain():
from scribe.services.reply_preferences import COMPLETION_QUERY
dev_only = [w for w in (r"\bCI\b", r"\bcommit", r"\bpull request", r"\bcode\b", r"\btest")
if re.search(w, COMPLETION_QUERY, re.IGNORECASE)]
assert not dev_only, f"software-only vocabulary in a query every project runs: {dev_only}"