feat(409): a Stop hook checks that a reply closing a task has the completion sections (#4014)

Everything else Scribe gives an agent arrives before the reply is written. A
Stop hook is the one moment the finished reply exists, so it is the last
chance to fix a report the operator can't read, and the only place adherence
to the shape can be measured.

- plugin/hooks/scribe_report_check.sh (Stop): deterministic, no model call.
  1. Did this turn close a task? That means an update_task/create_task call
     with status "done" since the turn's prompt, whose tool_result is not an
     error. Otherwise it stays silent, which covers most turns (one grep).
  2. Does the reply that ends the turn say where the work sits (a record by
     id and title, or step N of M), what needs the operator, and what comes
     next? Matched on those words, not on exact headings.
  3. If sections are missing, it blocks once. With stop_hook_active set, a
     rewrite is recorded (passed_after_rewrite / missing_after_rewrite) and
     never blocked again. A block loop started by another plugin (no marker
     from this hook) is left alone.
- Measured: every checked reply is reported to GET /api/plugin/report-check
  (passed / blocked / after rewrite). Turns that close nothing are not
  reported; they would cost a request per turn and add nothing to the rate.
  Outcomes go to app_logs as category "plugin", action "report_check".
- It blocks only when the block was recorded, and only in the server's words.
  The endpoint returns the block reason, so the hook carries timing and
  transport only (PACKAGING.md), and an unconfigured or unreachable instance
  never stops a session.
- The transcript format is read from real transcripts and marked in the hook
  as observed rather than documented. The Stop contract (transcript_path,
  stop_hook_active, decision/reason, no matcher, SubagentStop separate) was
  checked against the Claude Code hooks docs. A prompt-type hook was not
  needed: the deterministic check passed a real completion report from this
  session and blocked a stripped one.
- A pipefail trap was caught while exercising the hook: `tail | grep -q`
  reports failure exactly when grep matches, because tail dies of SIGPIPE.
  The prefilter reads through process substitution; the section checks use
  here-strings.
- Tests: an end-to-end hook suite over synthetic transcripts and the shared
  HTTP sink (silence, pass, server-worded block, rewrite recorded, foreign
  loop, errored write, earlier turn, unwritten reply, no recorded check, bare
  id), and service tests for the reason wording and the outcome record. Smoke
  event added to check_plugin; README and PACKAGING list the hook and
  endpoint. Plugin version minted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

plugin/hooks/hooks.json

@@ -54,6 +54,16 @@
           }
         ]
       }
+    ],
+    "Stop": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "bash \"${CLAUDE_PLUGIN_ROOT}/hooks/scribe_report_check.sh\""
+          }
+        ]
+      }
     ]
   }
 }

plugin/hooks/scribe_report_check.sh

@@ -0,0 +1,186 @@
+#!/usr/bin/env bash
+# Scribe plugin — Stop hook: a reply that closes a task carries the completion
+# sections (milestone 409 step 5).
+#
+# Everything else the plugin does happens BEFORE the agent writes: context,
+# retrieval, the reporting-back skill. This is the one moment the finished
+# reply exists, so it is both the last chance to fix a report the operator
+# cannot read and the only place adherence to the shape can be measured.
+#
+# DETERMINISTIC, NO MODEL CALL. Three questions, cheapest first:
+#
+#   1. Did this turn close a Scribe task? An `update_task` / `create_task` tool
+#      call with status "done" since the turn's prompt, whose result was not an
+#      error. Most turns stop here, silently.
+#   2. Does the reply that ends the turn have the completion sections? Loosely:
+#      where the work sits (a record named by id and title, or step N of M),
+#      what needs the operator, and what comes next. Matched on the words that
+#      carry the meaning rather than exact headings, so the skill's wording can
+#      change without breaking this.
+#   3. If sections are missing, block once with a reason naming them. The agent
+#      rewrites; the rewrite is checked and recorded, and never blocked again.
+#
+# MEASURED FROM THE FIRST CALL. Each checked reply is reported to the instance
+# (`/api/plugin/report-check`): passed, blocked, and after a rewrite either
+# passed_after_rewrite or missing_after_rewrite. Turns that closed no task are
+# not reported — they would cost a request on every turn and add nothing to
+# the rate step 6 reads (blocked among checked replies).
+#
+# IT BLOCKS ONLY WHEN THE BLOCK IS RECORDED, AND ONLY IN THE SERVER'S WORDS.
+# The report goes out first; the instance answers a recorded `blocked` with
+# the reason to send the agent back with, and the hook blocks only on that
+# reason. An unconfigured or unreachable instance therefore never stops a
+# session, every intervention is one the numbers can see, and the guidance
+# text lives on the server (plugin/PACKAGING.md: hooks carry timing and
+# transport).
+#
+# THE TRANSCRIPT FORMAT IS OBSERVED, NOT DOCUMENTED. Claude Code documents
+# `transcript_path` and `stop_hook_active` for Stop, not the JSONL inside. As
+# read from real transcripts (2026-09-14): one content block per line;
+# `type: "assistant"` lines carry `message.content[]` blocks of `text` /
+# `tool_use` ({id, name, input}); tool results arrive as `type: "user"` lines
+# whose content is a `tool_result` array ({tool_use_id, is_error}); a turn's
+# prompt — typed, or a background-task notification — is a `user` line whose
+# content is a plain string and which is not `isMeta`. Anything that does not
+# parse that way makes the hook stay out of the way rather than guess.
+#
+# Config (same as the other hooks):
+#   CLAUDE_PLUGIN_OPTION_API_ENDPOINT  base URL, no trailing slash
+#   CLAUDE_PLUGIN_OPTION_API_TOKEN     fmcp_ API key (sensitive)
+#   SCRIBE_URL / SCRIBE_TOKEN override for the settings.json dogfooding path.
+set -uo pipefail
+
+command -v jq   >/dev/null 2>&1 || exit 0
+command -v curl >/dev/null 2>&1 || exit 0
+
+# shellcheck source=plugin/hooks/scribe_defs.sh
+. "$(dirname "${BASH_SOURCE[0]}")/scribe_defs.sh"
+
+# Stop delivers { session_id, transcript_path, cwd, hook_event_name, stop_hook_active }.
+event=$(cat 2>/dev/null || true)
+transcript=$(printf '%s' "$event" | jq -r '.transcript_path // empty' 2>/dev/null) || exit 0
+[ -n "$transcript" ] && [ -f "$transcript" ] || exit 0
+session_id=$(printf '%s' "$event" | jq -r '.session_id // empty' 2>/dev/null) || session_id=""
+active=$(printf '%s' "$event" | jq -r '.stop_hook_active // false' 2>/dev/null) || active="false"
+event_cwd=$(printf '%s' "$event" | jq -r '.cwd // empty' 2>/dev/null) || event_cwd=""
+
+safe_sid=$(printf '%s' "${session_id:-nosession}" | tr -c 'A-Za-z0-9._-' '_')
+state_dir="${TMPDIR:-/tmp}/scribe-reportcheck"
+mkdir -p "$state_dir" 2>/dev/null || true
+marker="$state_dir/${safe_sid}.blocked"
+
+# Cheap prefilter: no task tool anywhere in the recent transcript → nothing to
+# check. Keeps the ordinary turn at one grep. Process substitution, NOT a pipe:
+# under `pipefail`, `grep -q` exiting on the first match kills `tail` with
+# SIGPIPE, and the pipeline then reports failure precisely when it matched.
+grep -q -E '"name":"([^"]*__)?(update|create)_task"' < <(tail -c 2000000 "$transcript" 2>/dev/null) || {
+  rm -f "$marker" 2>/dev/null || true
+  exit 0
+}
+
+# The turn, parsed once. A window of recent lines, slurped raw and split inside
+# jq (a line-by-line `-R` read is the #2198 trap). A first line cut mid-record
+# fails to parse and is dropped. If the window holds no prompt, the turn cannot
+# be bounded, so the hook reports nothing and stays out of the way.
+facts=$(tail -n 3000 "$transcript" 2>/dev/null | jq -sRc '
+  split("\n") | map(try fromjson catch empty)
+  | map(select((.isSidechain // false) | not))
+  | . as $lines
+  | [range(0; length) | select(
+      $lines[.].type == "user" and ($lines[.].isMeta // false | not)
+      and ($lines[.].message.content | type) == "string")] as $prompts
+  | if ($prompts | length) == 0 then {bounded: false} else
+    $lines[($prompts | last) + 1:] as $turn
+    | [ $turn[] | select(.type == "assistant") | .message.content[]?
+        | select(.type == "tool_use"
+                 and ((.name // "") | test("(^|__)(update|create)_task$"))
+                 and (.input.status? == "done"))
+        | {id, task: (.input.task_id? // null)} ] as $closes
+    | [ $turn[] | select(.type == "user") | .message.content[]?
+        | select(type == "object" and .type == "tool_result" and .is_error == true)
+        | .tool_use_id ] as $errors
+    | [ $closes[] | select(.id as $i | ($errors | index($i)) | not) ] as $closed
+    | ([range(0; $turn | length) | select(
+          $turn[.].type == "user"
+          or ($turn[.].type == "assistant"
+              and ([$turn[.].message.content[]?.type] | index("tool_use"))))]
+        | last // -1) as $last_act
+    | {bounded: true,
+       closed: ($closed | length),
+       task_ids: [$closed[].task | select(. != null)],
+       reply: ([ $turn[$last_act + 1:][] | select(.type == "assistant")
+                 | .message.content[]? | select(.type == "text") | .text ] | join("\n"))}
+    end' 2>/dev/null) || exit 0
+
+[ "$(printf '%s' "$facts" | jq -r '.bounded // false')" = "true" ] || exit 0
+closed=$(printf '%s' "$facts" | jq -r '.closed // 0')
+if [ "${closed:-0}" = "0" ]; then
+  rm -f "$marker" 2>/dev/null || true
+  exit 0
+fi
+reply=$(printf '%s' "$facts" | jq -r '.reply // ""')
+task_ids=$(printf '%s' "$facts" | jq -r '.task_ids | map(tostring) | join(",")')
+
+# The reply may not be written to the transcript yet when the hook fires. An
+# empty reply is "cannot tell", not "missing everything" — stay out of the way.
+[ -n "$(printf '%s' "$reply" | tr -d '[:space:]')" ] || exit 0
+
+missing=()
+# Where the work sits: a record named by id AND title (#12 "…", milestone 3 "…"),
+# or a step position. A bare id is exactly the homework this shape removes.
+# shellcheck disable=SC2016 # backticks here are literal markdown, not an expansion
+grep -q -i -E '(#[0-9]+|milestone [0-9]+|task [0-9]+)[*_`]*[[:space:]]*[*_`]*["“]|step [0-9]+ of [0-9]+' <<< "$reply" \
+  || missing+=("where it sits")
+# What needs the operator — "needs you: nothing" counts; it is an answer.
+grep -q -i -E 'needs? (from )?you|nothing (is )?needed from you|your (call|decision)' <<< "$reply" \
+  || missing+=("needs you")
+# What comes next.
+grep -q -i -E '\bnext\b' <<< "$reply" \
+  || missing+=("next")
+
+# Reports the outcome; prints the instance's reply and returns 0 only if the
+# instance recorded it.
+report() {
+  scribe_config || return 1
+  local q repo enc m
+  q="outcome=$1&task_ids=${task_ids}"
+  m=$(IFS=,; printf '%s' "${missing[*]:-}")
+  if [ -n "$m" ]; then
+    enc=$(printf '%s' "$m" | jq -sRr '@uri' 2>/dev/null) || enc=""
+    q="${q}&missing=${enc}"
+  fi
+  repo=$(git -C "${event_cwd:-${CLAUDE_PROJECT_DIR:-$PWD}}" remote get-url origin 2>/dev/null || true)
+  if [ -n "$repo" ]; then
+    enc=$(printf '%s' "$repo" | jq -sRr '@uri' 2>/dev/null) || enc=""
+    [ -n "$enc" ] && q="${q}&repo=${enc}"
+  fi
+  curl -fsS --max-time 4 \
+    -H "Authorization: Bearer ${token}" \
+    "${url%/}/api/plugin/report-check?${q}" 2>/dev/null
+}
+
+if [ "$active" = "true" ]; then
+  # A Stop hook already blocked this stop. If it was this one, the reply is
+  # the rewrite: record how it came out, and let the session stop whatever
+  # the answer. If it was another plugin's block, this hook has nothing to add.
+  [ -f "$marker" ] || exit 0
+  rm -f "$marker" 2>/dev/null || true
+  if [ ${#missing[@]} -eq 0 ]; then report passed_after_rewrite >/dev/null; else report missing_after_rewrite >/dev/null; fi
+  exit 0
+fi
+rm -f "$marker" 2>/dev/null || true
+
+if [ ${#missing[@]} -eq 0 ]; then
+  report passed >/dev/null
+  exit 0
+fi
+
+# The words the agent is sent back with are the server's (plugin/PACKAGING.md:
+# a hook carries timing and transport). No reason back → nothing recorded →
+# no block.
+answer=$(report blocked) || exit 0
+reason=$(printf '%s' "$answer" | jq -r '.reason // empty' 2>/dev/null) || reason=""
+[ -n "$reason" ] || exit 0
+: > "$marker" 2>/dev/null || true
+jq -n --arg r "$reason" '{decision: "block", reason: $r}'
+exit 0