fix(rules): the staleness signal must not wait for prior art to match (#3244)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Failing after 52s
CI & Build / Build & push image (push) Skipped

CI caught two things, and the first is the feature not working rather than a
test being wrong.

THE SIGNAL WAS GATED ON A COINCIDENCE. build_write_path_hint returns early
when no prior art, stamp, divergence or derive matched, and that guard sat
ABOVE the new arm — so a session whose rules had changed was told only if the
file it happened to be editing also matched something else. A staleness
signal that fires on that coincidence is not a staleness signal. The arm now
runs above the guard, collecting into its own list that `lines` is seeded
from, and the guard accounts for it.

The standing-rule arm (milestone 307) is deliberately LEFT below that guard,
and this is a finding rather than a fix: it has the same gating and probably
should not, but it runs a SEMANTIC search, so lifting it would put an
embedding query on every write in every session. That is a cost decision, not
a bug fix, and not this task's to make.

THE MARKER MUST NOT BREAK THE PAYLOAD IT DECORATES. rules_etag is computed on
the SessionStart path, where `max()` raising costs the whole context payload
— every rule title, the project, all of it — to save a hint. A row with no
usable timestamp is now skipped and a set with none degrades to a count-only
marker, which still catches a rule added or deleted and only loses edits.
That is the right way round to lose information. CI found it because
build_session_context's tests pass MagicMock rules and `max()` over those
raises TypeError.

Also: list_always_on_rules on an install with no always-on rulebooks returns
`rules_etag: "empty|0"`. Its exact-dict test is updated rather than loosened
— the key being present on an empty install is the behaviour, not noise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-30 13:10:04 -04:00
co-authored by Claude Opus 5
parent 5c9bb40777
commit efabba58dd
11 changed files with 296 additions and 77 deletions
+4 -1
View File
@@ -260,7 +260,10 @@ async def test_list_always_on_rules_returns_empty_when_no_always_on_rulebooks():
):
from scribe.mcp.tools.rulebooks import list_always_on_rules
out = await list_always_on_rules()
assert out == {"rules": [], "total": 0}
# An install with no always-on rulebooks still gets a marker (milestone
# 323): "no rules" is a STATE, and a payload that omitted the key would
# make the write path read every session on a fresh install as a change.
assert out == {"rules": [], "total": 0, "rules_etag": "empty|0"}
@pytest.mark.asyncio
@@ -0,0 +1,106 @@
"""A record is cited by id AND title — never by number alone.
THE PROBLEM THIS IS ABOUT. The agent has the record open; the operator does
not. `#3244` reads as complete to the writer and as homework to the reader,
who has to look it up to know what their own conversation is about. The
operator's words: *"I don't know what a note, task, or milestone is by its ID
number."*
PRODUCT, NOT A RULE (rule 119). Every Scribe user hits this, so the fix is in
the surfaces the product ships — the skill that shapes how an agent writes,
and the tool responses that hand a record back. A per-instance rule would fix
it for one operator and leave the behaviour wrong for everyone else.
Scribe's duplicate gate already had the right shape — `id 412: "debounce
helper"` — which is why these assert the CONVENTION reaches the other
surfaces rather than inventing a new one.
"""
import pathlib
import pytest
from unittest.mock import AsyncMock, MagicMock, patch
SKILL = (
pathlib.Path(__file__).resolve().parents[1]
/ "plugin/skills/using-scribe/SKILL.md"
)
def test_the_skill_carries_the_convention():
"""The skill is read while deciding HOW to write, which is the only moment
this can be applied. A tool response can name one record; only the skill
can govern the prose around it."""
text = " ".join(SKILL.read_text().split())
assert "Name the record, never just its number" in text, (
"the using-scribe skill no longer tells an agent to write the title "
"alongside the id. Nothing else governs how records are cited in "
"prose, commit messages, or task bodies."
)
def test_the_skill_says_where_it_matters_most():
"""A convention stated only for chat messages gets applied only there —
and the places read LATER, by someone with even less context, are where a
bare id costs most."""
text = " ".join(SKILL.read_text().split()).lower()
assert "commit message" in text and "task bod" in text
def test_the_skill_says_to_look_up_a_title_it_does_not_know():
"""The escape hatch that would otherwise swallow the convention whole: an
agent that does not know the title will emit the number and move on."""
text = " ".join(SKILL.read_text().split()).lower()
assert "an id you can't name is one you haven't checked" in text
# ── The tool responses (the other half) ────────────────────────────────
#
# A deletion is the sharpest case: afterwards the row is trashed, so if the
# confirmation did not name it, nothing can. An operator who cannot recognise
# what was deleted cannot tell it was the wrong thing.
@pytest.mark.parametrize("module,fn,kind,rid", [
("tasks", "delete_task", "task", 3244),
("notes", "delete_note", "note", 2109),
("milestones", "delete_milestone", "milestone", 323),
])
@pytest.mark.asyncio
async def test_a_delete_confirmation_names_what_it_deleted(
module, fn, kind, rid, monkeypatch,
):
import importlib
mod = importlib.import_module(f"scribe.mcp.tools.{module}")
title = "the staleness signal"
row = MagicMock()
row.title = title
patches = [
patch.object(mod, "current_user_id", MagicMock(return_value=1)),
patch.object(mod.trash_svc, "delete", AsyncMock(return_value="batch-1")),
]
if module == "milestones":
patches.append(
patch.object(mod.milestones_svc, "get_milestone",
AsyncMock(return_value=row)))
else:
patches.append(
patch.object(mod.notes_svc, "get_note_for_user",
AsyncMock(return_value=(row, "owner"))))
for p in patches:
p.start()
try:
out = await getattr(mod, fn)(rid)
finally:
for p in patches:
p.stop()
assert out["title"] == title, f"{fn} returns no title for the record"
assert title in out["message"], (
f"{fn}'s message names only the id. After the delete the row is "
f"trashed, so this line is the last chance to say WHAT went."
)
assert str(rid) in out["message"], (
f"{fn} dropped the id — the title alone is not addressable, and the "
f"convention is id AND title, not one or the other."
)
+41
View File
@@ -224,3 +224,44 @@ async def test_the_arm_fails_open():
for p in patches:
p.stop()
assert "changed since this session started" not in out["context"]
def test_the_marker_cannot_break_the_payload_it_decorates():
"""It is computed on the SessionStart path. Raising there would cost the
whole context payload — every rule title, the project, the lot — to save
a hint, which is the wrong trade in every case.
A row with no usable timestamp is skipped; a set with none degrades to a
count-only marker. Count-only still catches a rule ADDED or DELETED and
only loses edits, which is the right way round to lose information.
Found by CI: `build_session_context` tests hand it MagicMock rules, and
`max()` over those raises TypeError rather than returning anything.
"""
from unittest.mock import MagicMock
assert svc.rules_etag([MagicMock(), MagicMock()]) == "unknown|2"
assert svc.rules_etag([SimpleNamespace()]) == "unknown|1"
# A count-only marker still moves when the set does.
assert svc.rules_etag([MagicMock()]) != svc.rules_etag([MagicMock(), MagicMock()])
# One usable stamp is enough to keep the real thing.
assert svc.rules_etag([_rule(), MagicMock()]).startswith("2026-")
@pytest.mark.asyncio
async def test_the_signal_arrives_even_when_nothing_else_matched():
"""THE BUG CI CAUGHT, and the one the task's acceptance criterion was
written to catch.
`build_write_path_hint` returns early when no prior art, stamp, divergence
or derive matched — which sat ABOVE this arm, so a session whose rules had
changed was told only if the file it happened to be editing also matched
something else. A staleness signal that fires on that coincidence is not a
staleness signal.
"""
held = svc.rules_etag([_rule(rid=1), _rule(rid=2, title="dev is home")])
current = [_rule(rid=1), _rule(rid=2, title="dev is home", updated_at=LATER)]
# Every other arm silent — which is exactly the case that used to return "".
ctx = await _hint(current, held)
assert "changed since this session started" in ctx