fix(rules): the staleness signal must not wait for prior art to match (#3244)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Failing after 52s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 24s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Failing after 52s
CI & Build / Build & push image (push) Skipped
CI caught two things, and the first is the feature not working rather than a test being wrong. THE SIGNAL WAS GATED ON A COINCIDENCE. build_write_path_hint returns early when no prior art, stamp, divergence or derive matched, and that guard sat ABOVE the new arm — so a session whose rules had changed was told only if the file it happened to be editing also matched something else. A staleness signal that fires on that coincidence is not a staleness signal. The arm now runs above the guard, collecting into its own list that `lines` is seeded from, and the guard accounts for it. The standing-rule arm (milestone 307) is deliberately LEFT below that guard, and this is a finding rather than a fix: it has the same gating and probably should not, but it runs a SEMANTIC search, so lifting it would put an embedding query on every write in every session. That is a cost decision, not a bug fix, and not this task's to make. THE MARKER MUST NOT BREAK THE PAYLOAD IT DECORATES. rules_etag is computed on the SessionStart path, where `max()` raising costs the whole context payload — every rule title, the project, all of it — to save a hint. A row with no usable timestamp is now skipped and a set with none degrades to a count-only marker, which still catches a rule added or deleted and only loses edits. That is the right way round to lose information. CI found it because build_session_context's tests pass MagicMock rules and `max()` over those raises TypeError. Also: list_always_on_rules on an install with no always-on rulebooks returns `rules_etag: "empty|0"`. Its exact-dict test is updated rather than loosened — the key being present on an empty install is the behaviour, not noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -260,7 +260,10 @@ async def test_list_always_on_rules_returns_empty_when_no_always_on_rulebooks():
|
||||
):
|
||||
from scribe.mcp.tools.rulebooks import list_always_on_rules
|
||||
out = await list_always_on_rules()
|
||||
assert out == {"rules": [], "total": 0}
|
||||
# An install with no always-on rulebooks still gets a marker (milestone
|
||||
# 323): "no rules" is a STATE, and a payload that omitted the key would
|
||||
# make the write path read every session on a fresh install as a change.
|
||||
assert out == {"rules": [], "total": 0, "rules_etag": "empty|0"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""A record is cited by id AND title — never by number alone.
|
||||
|
||||
THE PROBLEM THIS IS ABOUT. The agent has the record open; the operator does
|
||||
not. `#3244` reads as complete to the writer and as homework to the reader,
|
||||
who has to look it up to know what their own conversation is about. The
|
||||
operator's words: *"I don't know what a note, task, or milestone is by its ID
|
||||
number."*
|
||||
|
||||
PRODUCT, NOT A RULE (rule 119). Every Scribe user hits this, so the fix is in
|
||||
the surfaces the product ships — the skill that shapes how an agent writes,
|
||||
and the tool responses that hand a record back. A per-instance rule would fix
|
||||
it for one operator and leave the behaviour wrong for everyone else.
|
||||
|
||||
Scribe's duplicate gate already had the right shape — `id 412: "debounce
|
||||
helper"` — which is why these assert the CONVENTION reaches the other
|
||||
surfaces rather than inventing a new one.
|
||||
"""
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
SKILL = (
|
||||
pathlib.Path(__file__).resolve().parents[1]
|
||||
/ "plugin/skills/using-scribe/SKILL.md"
|
||||
)
|
||||
|
||||
|
||||
def test_the_skill_carries_the_convention():
|
||||
"""The skill is read while deciding HOW to write, which is the only moment
|
||||
this can be applied. A tool response can name one record; only the skill
|
||||
can govern the prose around it."""
|
||||
text = " ".join(SKILL.read_text().split())
|
||||
assert "Name the record, never just its number" in text, (
|
||||
"the using-scribe skill no longer tells an agent to write the title "
|
||||
"alongside the id. Nothing else governs how records are cited in "
|
||||
"prose, commit messages, or task bodies."
|
||||
)
|
||||
|
||||
|
||||
def test_the_skill_says_where_it_matters_most():
|
||||
"""A convention stated only for chat messages gets applied only there —
|
||||
and the places read LATER, by someone with even less context, are where a
|
||||
bare id costs most."""
|
||||
text = " ".join(SKILL.read_text().split()).lower()
|
||||
assert "commit message" in text and "task bod" in text
|
||||
|
||||
|
||||
def test_the_skill_says_to_look_up_a_title_it_does_not_know():
|
||||
"""The escape hatch that would otherwise swallow the convention whole: an
|
||||
agent that does not know the title will emit the number and move on."""
|
||||
text = " ".join(SKILL.read_text().split()).lower()
|
||||
assert "an id you can't name is one you haven't checked" in text
|
||||
|
||||
|
||||
# ── The tool responses (the other half) ────────────────────────────────
|
||||
#
|
||||
# A deletion is the sharpest case: afterwards the row is trashed, so if the
|
||||
# confirmation did not name it, nothing can. An operator who cannot recognise
|
||||
# what was deleted cannot tell it was the wrong thing.
|
||||
|
||||
@pytest.mark.parametrize("module,fn,kind,rid", [
|
||||
("tasks", "delete_task", "task", 3244),
|
||||
("notes", "delete_note", "note", 2109),
|
||||
("milestones", "delete_milestone", "milestone", 323),
|
||||
])
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_delete_confirmation_names_what_it_deleted(
|
||||
module, fn, kind, rid, monkeypatch,
|
||||
):
|
||||
import importlib
|
||||
|
||||
mod = importlib.import_module(f"scribe.mcp.tools.{module}")
|
||||
title = "the staleness signal"
|
||||
row = MagicMock()
|
||||
row.title = title
|
||||
|
||||
patches = [
|
||||
patch.object(mod, "current_user_id", MagicMock(return_value=1)),
|
||||
patch.object(mod.trash_svc, "delete", AsyncMock(return_value="batch-1")),
|
||||
]
|
||||
if module == "milestones":
|
||||
patches.append(
|
||||
patch.object(mod.milestones_svc, "get_milestone",
|
||||
AsyncMock(return_value=row)))
|
||||
else:
|
||||
patches.append(
|
||||
patch.object(mod.notes_svc, "get_note_for_user",
|
||||
AsyncMock(return_value=(row, "owner"))))
|
||||
for p in patches:
|
||||
p.start()
|
||||
try:
|
||||
out = await getattr(mod, fn)(rid)
|
||||
finally:
|
||||
for p in patches:
|
||||
p.stop()
|
||||
|
||||
assert out["title"] == title, f"{fn} returns no title for the record"
|
||||
assert title in out["message"], (
|
||||
f"{fn}'s message names only the id. After the delete the row is "
|
||||
f"trashed, so this line is the last chance to say WHAT went."
|
||||
)
|
||||
assert str(rid) in out["message"], (
|
||||
f"{fn} dropped the id — the title alone is not addressable, and the "
|
||||
f"convention is id AND title, not one or the other."
|
||||
)
|
||||
@@ -224,3 +224,44 @@ async def test_the_arm_fails_open():
|
||||
for p in patches:
|
||||
p.stop()
|
||||
assert "changed since this session started" not in out["context"]
|
||||
|
||||
|
||||
def test_the_marker_cannot_break_the_payload_it_decorates():
|
||||
"""It is computed on the SessionStart path. Raising there would cost the
|
||||
whole context payload — every rule title, the project, the lot — to save
|
||||
a hint, which is the wrong trade in every case.
|
||||
|
||||
A row with no usable timestamp is skipped; a set with none degrades to a
|
||||
count-only marker. Count-only still catches a rule ADDED or DELETED and
|
||||
only loses edits, which is the right way round to lose information.
|
||||
|
||||
Found by CI: `build_session_context` tests hand it MagicMock rules, and
|
||||
`max()` over those raises TypeError rather than returning anything.
|
||||
"""
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
assert svc.rules_etag([MagicMock(), MagicMock()]) == "unknown|2"
|
||||
assert svc.rules_etag([SimpleNamespace()]) == "unknown|1"
|
||||
# A count-only marker still moves when the set does.
|
||||
assert svc.rules_etag([MagicMock()]) != svc.rules_etag([MagicMock(), MagicMock()])
|
||||
# One usable stamp is enough to keep the real thing.
|
||||
assert svc.rules_etag([_rule(), MagicMock()]).startswith("2026-")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_signal_arrives_even_when_nothing_else_matched():
|
||||
"""THE BUG CI CAUGHT, and the one the task's acceptance criterion was
|
||||
written to catch.
|
||||
|
||||
`build_write_path_hint` returns early when no prior art, stamp, divergence
|
||||
or derive matched — which sat ABOVE this arm, so a session whose rules had
|
||||
changed was told only if the file it happened to be editing also matched
|
||||
something else. A staleness signal that fires on that coincidence is not a
|
||||
staleness signal.
|
||||
"""
|
||||
held = svc.rules_etag([_rule(rid=1), _rule(rid=2, title="dev is home")])
|
||||
current = [_rule(rid=1), _rule(rid=2, title="dev is home", updated_at=LATER)]
|
||||
|
||||
# Every other arm silent — which is exactly the case that used to return "".
|
||||
ctx = await _hint(current, held)
|
||||
assert "changed since this session started" in ctx
|
||||
|
||||
Reference in New Issue
Block a user