CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / Python tests (push) Failing after 1m7s
CI & Build / Build & push image (push) Skipped
Every hook opened `command -v jq >/dev/null 2>&1 || exit 0`, so on a machine without jq the operator got no session context, no rules, no prior art and no process sync — and not one word saying why, because `exit 0` is indistinguishable from "ran fine, nothing to say". jq is absent by default on macOS, on the Debian/Ubuntu slim images, on Alpine and in most CI containers. That is not a prerequisite to document; it is the plugin handing its own packaging problem to whoever installs it. `tac` was worse: GNU-only, so the prior-art hook's enclosing-definition arm did nothing at all on every Mac, silently, from the day it shipped. It is not replaced but removed — scribe_defs judges each line independently, so extracting forward and taking `tail -1` is the same answer as reversing and taking the head, and it drops the early-exit `head` that #4042 was filed for. No server contract changed, so a lagging plugin cache keeps working. scribe_json.awk JSON -> IDX<TAB>PATH<TAB>VALUE. Two modes: `whole` for an event or a response body, `lines` for a transcript, where an unparseable record is dropped and the rest still read — the `map(try fromjson catch empty)` the jq program opened with. Arrays also report their LENGTH at `[#]`, which is what keeps "zero notes" distinct from "no answer" (#2932). scribe_turn.awk the turn-bounding program, replacing the thirty lines of jq in the Stop hook. scribe_defs.sh scribe_json_flat / _pick / _list / _len / _list_minus read, scribe_json_out writes the envelope (five copies of one shape, gone), scribe_urlenc replaces `jq -sRr '@uri'`. Percent-encoding goes through `od -tu1` rather than an awk character loop on purpose: awk's idea of a character follows the locale, so gawk reads an accented letter as one and mawk as two, and an encoder built on substr() would emit a different URL depending on which awk is installed. Encoding is defined on bytes. Verified byte-identical to `jq -sRr '@uri'`. Measured, not assumed. The per-event path costs 8ms against jq's 3ms. The transcript path was 70x slower until two fixes: the Stop hook now finds where the turn starts with a fixed-string grep before parsing (a needle carrying unescaped quotes cannot occur inside a JSON string, so it matches only at a record's top level — checked against a full JSON parse of a 27MB transcript: 152 prompt records, 152 matches, no misses, no extras), and the parser reads each token out of a 1024-byte window instead of copying the rest of the buffer per token, which was quadratic in line length on the 400KB tool results a transcript carries. Differential-tested against the jq program it replaces over 724 windows cut from three real transcripts — 724 identical, 0 mismatched, 45 of them exercising a real task close and a real reply. That sweep is what caught `scribe_turn.awk` never setting FS, which truncated every multi-word reply at its first space and was invisible to a test whose replies were all empty. check_plugin.py's `jq -R` lint becomes a guard against either binary coming back, and three smoke checks lose their `shutil.which("jq")` skip. jq is not in `ci-python` either, so those three announced a skip on every CI run and had never once run there: removing the dependency from the product also closed a permanent hole in its verification. They pass now across all ten hooks. tests/test_hook_json_reader.py is a differential against Python's `json` over nested objects, arrays, unicode, escapes, control characters, empty cases and a value longer than the token window, plus the envelope, the encoder and the turn analyzer. 139 cases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
155 lines
6.6 KiB
Python
155 lines
6.6 KiB
Python
"""Named is not read: the ledger's third state (#4100).
|
|
|
|
WHY THIS EXISTS
|
|
|
|
Milestone 386 made a repeat REFERENCED rather than withheld, and the line it
|
|
chose says "You saw it earlier this session". That claim was never checked.
|
|
The arms emit a TEASER — title, trigger, `get_rule(N)` — so a session can be
|
|
shown a rule twenty times and never read a word of it, and a compaction
|
|
summarises the teaser away leaving nothing at all. The server was asserting
|
|
something about the reader's context it had no way to know.
|
|
|
|
`.opened.ids` is the observable half, written by a PostToolUse hook from the
|
|
`get_rule` call itself. That is why this is not the self-report 386 rejected:
|
|
the objection was to ASKING a model about its own context, and a tool call is
|
|
an event the harness reports whether anyone asks or not.
|
|
|
|
WHAT THIS PINS
|
|
|
|
The three states and their three lines, the hook that records an open, and the
|
|
fact that BOTH ledgers die together on a compaction — a session told "you
|
|
opened it earlier" about a rule that was just summarised out of its context
|
|
would be a more confident version of the bug this removes.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
import re
|
|
import shutil
|
|
import subprocess
|
|
from pathlib import Path
|
|
from unittest.mock import MagicMock
|
|
|
|
import pytest
|
|
|
|
from scribe.services.plugin_context import _rule_hint_line
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
HOOKS = ROOT / "plugin" / "hooks"
|
|
RECORDER = HOOKS / "scribe_record_opened.sh"
|
|
|
|
|
|
def _rule(rid=156, kind="rule"):
|
|
return MagicMock(id=rid, kind=kind, title="A wait with no deadline is a bug",
|
|
when_to_apply="crossing a process boundary")
|
|
|
|
|
|
# ── the three lines ────────────────────────────────────────────────────────
|
|
|
|
def test_a_rule_never_surfaced_is_offered_as_new():
|
|
line = _rule_hint_line(_rule(), where="here", seen=False, held=False)
|
|
assert "not in this session's loaded set" in line
|
|
|
|
|
|
def test_a_rule_named_but_not_opened_says_so_and_still_invites():
|
|
"""The state that did not exist. It must NOT claim the reader saw it, and
|
|
must still carry the pull pointer — a session that skipped the teaser is
|
|
in nearly the position of one that was never shown it."""
|
|
line = _rule_hint_line(_rule(), where="here", seen=True, held=False)
|
|
assert "not opened" in line
|
|
assert "get_rule(156)" in line
|
|
assert "You saw it earlier" not in line, (
|
|
"the middle state is claiming the reader read something they did not"
|
|
)
|
|
|
|
|
|
def test_a_rule_the_session_opened_is_described_as_opened():
|
|
line = _rule_hint_line(_rule(), where="here", seen=True, held=True)
|
|
assert "opened it earlier" in line
|
|
assert "get_rule(156)" in line
|
|
|
|
|
|
def test_the_three_states_produce_three_different_lines():
|
|
"""Guard against a refactor collapsing two branches: each state has to be
|
|
distinguishable, or the distinction this milestone bought is gone while
|
|
every individual assertion above still passes."""
|
|
lines = {
|
|
_rule_hint_line(_rule(), where="here", seen=s, held=h)
|
|
for s, h in ((False, False), (True, False), (True, True))
|
|
}
|
|
assert len(lines) == 3
|
|
|
|
|
|
def test_held_outranks_seen_regardless_of_kind():
|
|
"""`kind` moves the head and the ledger moves the tail; #3497's history is
|
|
the two being reasoned about together and one of them being forgotten."""
|
|
for kind in ("rule", "preference"):
|
|
line = _rule_hint_line(_rule(kind=kind), where="here", seen=True, held=True)
|
|
assert "opened it earlier" in line
|
|
|
|
|
|
# ── the recorder ───────────────────────────────────────────────────────────
|
|
|
|
def _run_recorder(event: dict, tmp: Path) -> Path:
|
|
for tool in ("bash",):
|
|
if shutil.which(tool) is None:
|
|
pytest.skip(f"hook runtime tool {tool!r} not installed")
|
|
env = {"PATH": os.environ["PATH"], "HOME": str(tmp), "TMPDIR": str(tmp)}
|
|
out = subprocess.run(["bash", str(RECORDER)], input=json.dumps(event),
|
|
capture_output=True, text=True, env=env, timeout=30)
|
|
assert out.returncode == 0, out.stderr
|
|
return tmp / "scribe-priorart" / "s1.opened.ids"
|
|
|
|
|
|
def test_opening_a_rule_is_recorded(tmp_path):
|
|
led = _run_recorder(
|
|
{"session_id": "s1", "tool_name": "mcp__scribe__get_rule",
|
|
"tool_input": {"rule_id": 156}}, tmp_path)
|
|
assert led.exists()
|
|
assert led.read_text().split("\t")[0] == "156"
|
|
|
|
|
|
def test_the_entry_is_stamped_so_it_ages_like_the_naming_ledger(tmp_path):
|
|
"""Both ledgers are read by `scribe_rules_live`, which ages on that stamp.
|
|
An unstamped entry never expires — bounded, but it would mean an opened
|
|
rule stays 'opened' for a session's whole life."""
|
|
led = _run_recorder(
|
|
{"session_id": "s1", "tool_name": "mcp__scribe__get_rule",
|
|
"tool_input": {"rule_id": 9}}, tmp_path)
|
|
parts = led.read_text().strip().split("\t")
|
|
assert len(parts) == 2 and parts[1].isdigit()
|
|
|
|
|
|
@pytest.mark.parametrize("event", [
|
|
{"session_id": "s1", "tool_name": "mcp__scribe__get_rule", "tool_input": {}},
|
|
{"session_id": "s1", "tool_name": "mcp__scribe__get_rule",
|
|
"tool_input": {"rule_id": "../../etc"}},
|
|
{"tool_name": "mcp__scribe__get_rule", "tool_input": {"rule_id": 5}},
|
|
])
|
|
def test_an_event_with_nothing_usable_records_nothing_and_still_exits_zero(event, tmp_path):
|
|
"""A hook that fails a tool call over bookkeeping is worse than one that
|
|
misses an id: the cost of a miss is one extra line, in the direction that
|
|
shows more rather than less."""
|
|
led = _run_recorder(event, tmp_path)
|
|
assert not led.exists()
|
|
|
|
|
|
# What a compaction clears — including `.opened.ids`, whose claim is the one
|
|
# that would be worst to get wrong — is asserted against the running hook in
|
|
# tests/test_session_ledger_clear.py, so there is one home for it.
|
|
|
|
|
|
def test_the_recorder_is_registered_on_the_get_rule_tool():
|
|
hooks = json.loads((HOOKS / "hooks.json").read_text())["hooks"]
|
|
posts = hooks["PostToolUse"]
|
|
mine = [b for b in posts
|
|
if any("scribe_record_opened.sh" in h["command"] for h in b["hooks"])]
|
|
assert len(mine) == 1, "the opened-recorder is not registered exactly once"
|
|
# An MCP tool's server segment varies with how the plugin was installed, so
|
|
# the matcher must not pin one spelling of it.
|
|
matcher = mine[0]["matcher"]
|
|
assert re.fullmatch(matcher, "mcp__plugin_scribe_scribe__get_rule"), matcher
|
|
assert re.fullmatch(matcher, "mcp__scribe__get_rule"), matcher
|
|
assert not re.fullmatch(matcher, "Bash"), matcher
|