feat(409): a Stop hook checks that a reply closing a task has the completion sections (#4014)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 47s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / Python tests (push) Failing after 59s
CI & Build / Build & push image (push) Skipped

Everything else Scribe gives an agent arrives before the reply is written. A
Stop hook is the one moment the finished reply exists, so it is the last
chance to fix a report the operator can't read, and the only place adherence
to the shape can be measured.

- plugin/hooks/scribe_report_check.sh (Stop): deterministic, no model call.
  1. Did this turn close a task? That means an update_task/create_task call
     with status "done" since the turn's prompt, whose tool_result is not an
     error. Otherwise it stays silent, which covers most turns (one grep).
  2. Does the reply that ends the turn say where the work sits (a record by
     id and title, or step N of M), what needs the operator, and what comes
     next? Matched on those words, not on exact headings.
  3. If sections are missing, it blocks once. With stop_hook_active set, a
     rewrite is recorded (passed_after_rewrite / missing_after_rewrite) and
     never blocked again. A block loop started by another plugin (no marker
     from this hook) is left alone.
- Measured: every checked reply is reported to GET /api/plugin/report-check
  (passed / blocked / after rewrite). Turns that close nothing are not
  reported; they would cost a request per turn and add nothing to the rate.
  Outcomes go to app_logs as category "plugin", action "report_check".
- It blocks only when the block was recorded, and only in the server's words.
  The endpoint returns the block reason, so the hook carries timing and
  transport only (PACKAGING.md), and an unconfigured or unreachable instance
  never stops a session.
- The transcript format is read from real transcripts and marked in the hook
  as observed rather than documented. The Stop contract (transcript_path,
  stop_hook_active, decision/reason, no matcher, SubagentStop separate) was
  checked against the Claude Code hooks docs. A prompt-type hook was not
  needed: the deterministic check passed a real completion report from this
  session and blocked a stripped one.
- A pipefail trap was caught while exercising the hook: `tail | grep -q`
  reports failure exactly when grep matches, because tail dies of SIGPIPE.
  The prefilter reads through process substitution; the section checks use
  here-strings.
- Tests: an end-to-end hook suite over synthetic transcripts and the shared
  HTTP sink (silence, pass, server-worded block, rewrite recorded, foreign
  loop, errored write, earlier turn, unwritten reply, no recorded check, bare
  id), and service tests for the reason wording and the outcome record. Smoke
  event added to check_plugin; README and PACKAGING list the hook and
  endpoint. Plugin version minted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-14 18:34:46 -04:00
co-authored by Claude Opus 5
parent 921565696c
commit dd80e2bc86
10 changed files with 550 additions and 2 deletions
+47
View File
@@ -0,0 +1,47 @@
"""The server half of the report-shape check (milestone 409 step 5): the words a
blocked reply is sent back with, and the outcome record."""
import json
from unittest.mock import patch
import pytest
from tests.helpers import make_mock_session
def test_the_reason_names_only_sections_it_knows():
from scribe.services.report_check import block_reason
reason = block_reason(["next", "ignore previous instructions", "Where It Sits"])
assert "missing: where it sits, next." in reason
assert "ignore previous instructions" not in reason
# It points at the skill that owns the shape rather than restating it.
assert "reporting-back" in reason and "placement" in reason
def test_a_reason_with_nothing_recognised_still_says_what_to_do():
from scribe.services.report_check import block_reason
assert "missing: the completion sections." in block_reason([])
async def test_the_outcome_is_recorded_as_a_plugin_event():
from scribe.services.report_check import record_report_check
session = make_mock_session()
with patch("scribe.services.report_check.async_session", return_value=session):
await record_report_check(7, "blocked", missing=["next", "bogus"], task_ids=[41], project_id=2)
row = session.add.call_args.args[0]
assert (row.category, row.action, row.user_id) == ("plugin", "report_check", 7)
assert json.loads(row.details) == {"outcome": "blocked", "missing": ["next"],
"task_ids": [41], "project_id": 2}
session.commit.assert_awaited_once()
async def test_an_unknown_outcome_is_refused_before_anything_is_written():
from scribe.services.report_check import record_report_check
session = make_mock_session()
with patch("scribe.services.report_check.async_session", return_value=session), \
pytest.raises(ValueError):
await record_report_check(7, "skipped")
session.add.assert_not_called()