feat(500): one end-of-turn request - the completion-section check folds into the reply check
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m7s
CI & Build / Python tests (push) Failing after 1m25s
CI & Build / Build & push image (push) Skipped

The Stop hook sent the finished reply twice: scribe_report_check.sh checked a
task-closing reply for the completion sections in shell and reported to
/report-check, and scribe_reply_check.sh sent the same reply to /reply-rules
for the rule hold. Now the reply goes once. When the turn closed a task the
hook adds the close count and ids, and the server runs the section check
(services/report_check, the same three patterns) beside the reply hold,
folding both into one reason.

The report_check adherence log is still written for every checked reply
(milestone 409's number). A section hold marks the session, so its rewrite is
sent back once with rewrite:true to record how it came out, and is never held.
The block reason carries the completion shape's one line and points at
list_reply_shapes rather than at the skill. #5496 (step 4 of milestone 500).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 15:38:17 -04:00
co-authored by Claude Opus 5.5
parent ae66d83213
commit 6071007fe2
15 changed files with 348 additions and 433 deletions
+40 -49
View File
@@ -367,28 +367,58 @@ async def reply_rules():
the reply asks something), and the reply text against every rule's
trigger — the backstop for whatever the earlier arms missed.
Body: `{"reply": "<text>"}`. Query: `repo` / `project_id` for the scope;
ONE END-OF-TURN REQUEST (milestone 500 step 4): a reply that closed tasks
is also checked here for the completion sections (services/report_check),
which used to be a Stop hook and an endpoint of its own. Both holds fold
into one `reason`; `report_check` says what the section check recorded, so
the hook knows the rewrite is one to report.
Body: `{"reply": "<text>", "closed": 1, "closed_task_ids": [41],
"rewrite": false}` — the last three only when the turn closed a task.
`closed` is the count and decides whether the check runs (a task created
already done has no id to list); `rewrite` marks the reply written after a
section hold, which is recorded and never held.
Query: `repo` / `project_id` for the scope;
`held_rule_ids` (opened this session — exempt, as at the act checkpoint);
`exclude_rule_ids` (named this session — not re-counted as surfaced);
`stopped_rule_ids` (already held a reply or an act this session — a rule
holds once, and the per-session cap counts these).
Returns `reason` — the words the hook blocks with, empty when nothing
holds — plus `rule_ids` for the hook's ledger and `moments` reached.
holds — plus `rule_ids` for the hook's ledger, `moments` reached and
`report_check` (the recorded outcome, or "" when nothing was checked).
"""
data = await request.get_json(silent=True) or {}
if not isinstance(data, dict):
data = {}
reply = str(data.get("reply") or "")
# `type(...) is int`, not isinstance: JSON `true` would otherwise read as 1.
closed = data.get("closed") if type(data.get("closed")) is int else 0
ids = data.get("closed_task_ids")
task_ids = [i for i in ids if type(i) is int and i > 0][:20] if isinstance(ids, list) else []
rewrite = data.get("rewrite") is True
project_id, _repo, _unbound = await _project_scope()
held = await moment_delivery_svc.reply_hold(
g.user.id, reply, project_id=project_id,
exclude=frozenset(_int_list(request.args.get("exclude_rule_ids"))),
held=frozenset(_int_list(request.args.get("held_rule_ids"))),
stopped=frozenset(_int_list(request.args.get("stopped_rule_ids"))),
)
checked: dict = {}
if closed > 0:
checked = await report_check_svc.check_reply(
g.user.id, reply, task_ids=task_ids, rewrite=rewrite, project_id=project_id or None,
)
# The rewrite after a hold goes out as written: recorded above, held by nothing.
held: dict = {}
if not rewrite:
held = await moment_delivery_svc.reply_hold(
g.user.id, reply, project_id=project_id,
exclude=frozenset(_int_list(request.args.get("exclude_rule_ids"))),
held=frozenset(_int_list(request.args.get("held_rule_ids"))),
stopped=frozenset(_int_list(request.args.get("stopped_rule_ids"))),
)
reasons = [r for r in (checked.get("reason", ""), held.get("reason", "")) if r]
return jsonify({
"reason": held.get("reason", ""),
"reason": "\n\n".join(reasons),
"rule_ids": held.get("rule_ids", []),
"moments": held.get("moments", []),
"report_check": checked.get("outcome", ""),
})
@@ -504,45 +534,6 @@ def _parse_shapes(raw: str) -> list[tuple[str, str]]:
return out
@plugin_bp.get("/report-check")
@login_required
async def report_check():
"""Record what a Stop hook found in a reply that closed a task (milestone 409 step 5).
The hook decides which completion sections the reply lacks — a local check
of text it can read — and reports the outcome here. For `blocked` the
response carries the `reason` to send the agent back with: the words are
the server's, so every client's hook says the same thing (plugin/PACKAGING.md).
A hook blocks only on a `reason` it received, which means only on a block
that was recorded.
A GET for the reason every plugin endpoint is one: a read-scoped key must
be enough to run the plugin, and this records telemetry the way /retrieve
records a retrieval log.
Query:
outcome (str) — passed | blocked | passed_after_rewrite |
missing_after_rewrite. Anything else is a 400.
missing (opt) — comma-separated sections the reply lacked:
"where it sits", "needs you", "next".
task_ids (opt) — comma-separated ids of the tasks the turn closed.
repo (opt) — working repo remote, resolved like the other arms.
"""
outcome = (request.args.get("outcome") or "").strip()
if outcome not in report_check_svc.OUTCOMES:
return jsonify({"error": f"outcome must be one of {list(report_check_svc.OUTCOMES)}"}), 400
missing = [m for m in (request.args.get("missing") or "").split(",") if m.strip()]
task_ids = _int_list(request.args.get("task_ids"))[:20]
project_id, _repo, _unbound = await _project_scope()
await report_check_svc.record_report_check(
g.user.id, outcome, missing=missing, task_ids=task_ids, project_id=project_id or None,
)
body: dict = {"status": "ok"}
if outcome == "blocked":
body["reason"] = report_check_svc.block_reason(missing)
return jsonify(body)
@plugin_bp.get("/shape-check")
@login_required
async def shape_check():
@@ -557,7 +548,7 @@ async def shape_check():
recorded, and never on the stop that follows one (`phase=after`).
A GET like every plugin endpoint: a read-scoped key runs the plugin, and
this records telemetry the way /report-check does. It changes no ledger
this records telemetry the way /reply-rules records its section check. It changes no ledger
row — the agent's own classify_shapes call does that.
Query:
+74 -22
View File
@@ -1,38 +1,60 @@
"""The report-shape check: what the plugin's Stop hook found, and what it says.
"""The report-shape check: does a reply that closed a task carry the completion
sections, and what is it sent back with when it does not.
WHY THIS EXISTS (milestone 409 step 5)
Everything that helps an agent write a readable completion report arrives
BEFORE the reply is written. A client's Stop hook is the one moment the
finished reply exists, so it checks that a reply closing a task carries the
completion sections (where the work sits, what needs the operator, what comes
next), and reports what it found here. Two jobs live on this side:
BEFORE the reply is written. The Stop hook is the one moment the finished reply
exists, so a reply that closed a task is checked for the completion sections
(where the work sits, what needs the operator, what comes next). Two jobs:
- RECORDING the outcome, so the rate of `blocked` among checked replies is a
number milestone 409's last step can read rather than an impression.
- OWNING THE WORDS the agent is sent back with. A hook carries timing and
transport only (plugin/PACKAGING.md); guidance text comes from the server,
so a second client's hook gets the same instruction by calling the same
endpoint, and the wording changes in one place.
number rather than an impression (`category=plugin, action=report_check`).
- OWNING THE WORDS the agent is sent back with, so every client's hook says
the same thing (plugin/PACKAGING.md: hooks carry timing and transport).
ONE END-OF-TURN REQUEST (milestone 500 step 4). This used to be a Stop hook of
its own that checked the sections in shell and reported here; the reply-moment
check sent the same reply to `/reply-rules` a moment later. The hook now sends
the reply once, with the ids of the tasks the turn closed, and the section
check runs here beside the reply hold. The patterns are the ones the shell
used, matched on the words that carry the meaning rather than exact headings,
so a shape's wording can change without breaking this.
app_logs rather than a table of its own: one small event with a JSON detail is
what that table holds, it already has retention and an admin viewer, and
nothing here needs a join. If the numbers earn a readout, that is the moment to
decide whether they earn a table.
nothing here needs a join.
"""
from __future__ import annotations
import json
import logging
import re
from scribe.models import async_session
from scribe.models.app_log import AppLog
from scribe.services import reply_shapes
logger = logging.getLogger(__name__)
OUTCOMES = ("passed", "blocked", "passed_after_rewrite", "missing_after_rewrite")
# The sections a hook may name as missing, in the order the reason lists them.
# Anything else a client sends is dropped rather than echoed into an
# instruction the agent will follow.
SECTIONS = ("where it sits", "needs you", "next")
# The sections, in the order the reason lists them, each with what counts as
# having it. A bare id ("closed #41") does not place the work: naming a record
# by id AND title, or a step position, is the shape — the title is what spares
# the reader a lookup. "Needs you: nothing" counts; it is an answer.
SECTION_PATTERNS: dict[str, re.Pattern] = {
"where it sits": re.compile(
r'(#[0-9]+|milestone [0-9]+|task [0-9]+)[*_`]*\s*[*_`]*["“]|step [0-9]+ of [0-9]+', re.I),
"needs you": re.compile(
r"needs? (from )?you|nothing (is )?needed from you|your (call|decision)", re.I),
"next": re.compile(r"\bnext\b", re.I),
}
SECTIONS = tuple(SECTION_PATTERNS)
def missing_sections(reply: str) -> list[str]:
return [name for name, pat in SECTION_PATTERNS.items() if not pat.search(reply or "")]
def known_sections(missing: list[str]) -> list[str]:
@@ -43,16 +65,17 @@ def known_sections(missing: list[str]) -> list[str]:
def block_reason(missing: list[str]) -> str:
"""What the agent is told when its completion report is sent back.
Names what is missing and points at the reporting-back skill for the shape
rather than restating it — the skill owns the shape (decision #4027).
Names what is missing and carries the completion shape's one line rather
than the whole shape: the shape itself was delivered when the task closed,
and `list_reply_shapes` has it in full.
"""
listed = ", ".join(known_sections(missing)) or "the completion sections"
shape = reply_shapes.SHAPES["completion"]
return (
f"This turn closed a Scribe task, and the reply that ends it is missing: {listed}. "
"The operator reads this reply to find out where the work stands. Rewrite it as a "
"completion report (the reporting-back skill has the shape): where it sits — the task "
"or milestone by id and title, from `placement` — what now works, what needs them "
"(or \"nothing\"), and what comes next."
"The person the work is for reads this reply to find out where it stands. Rewrite it "
f"as a completion report — {shape.reminder} Take where it sits from `placement`, the "
"task or plan by id and title (`list_reply_shapes` has the shape in full)."
)
@@ -78,3 +101,32 @@ async def record_report_check(
details=json.dumps(details),
))
await session.commit()
async def check_reply(
user_id: int, reply: str, *, task_ids: list[int], rewrite: bool,
project_id: int | None = None,
) -> dict:
"""Check a reply that closed tasks, record the outcome, and say whether it is held.
`rewrite` is the reply written after an earlier hold: it is recorded and
never held again, so a hold costs one turn at most.
Returns {"outcome", "reason"} — `reason` empty unless the reply is held.
A hold happens only when its record was written: an outcome the numbers
cannot see is not one this check may act on, so a recording failure
degrades to no hold rather than to an unmeasured one.
"""
missing = missing_sections(reply)
if rewrite:
outcome = "missing_after_rewrite" if missing else "passed_after_rewrite"
else:
outcome = "blocked" if missing else "passed"
try:
await record_report_check(user_id, outcome, missing=missing, task_ids=task_ids,
project_id=project_id)
except Exception: # noqa: BLE001 - an unrecorded check never holds a reply
logger.warning("report check not recorded", exc_info=True)
return {"outcome": "", "reason": ""}
return {"outcome": outcome,
"reason": block_reason(missing) if outcome == "blocked" else ""}