CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Failing after 33s
CI & Build / Python tests (push) Failing after 37s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / Build & push image (push) Skipped
`when_to_apply` is not metadata. `rule_document` embeds a rule as
`{title} — {trigger}` / `When to apply: {trigger}\n\n{statement}`, the trigger
appearing twice so purpose dominates a short vector — the shape note 2485
measured on snippets (a 0.153 top-to-second gap against 0.010–0.023 for
everything else). Without one the document silently becomes title + statement:
a DIFFERENT shape, ranked against a corpus it does not match, with nothing to
report it. Every bar and every rank in the system assumes one shape.
`create_preference` has refused an empty trigger since it shipped. The two rule
creators defaulted it to "" — so the shape was enforced for the record kind
that guides and optional for the kind that binds.
The guard lives in the SERVICE, because both doors reach it: the MCP tools and
the frontend's fast path in routes/rulebooks.py. Written in either alone, the
other could still create a rule that never fires. The route keeps a matching
check for the STATUS CODE only (400, not the 404 it maps ValueError to).
update_rule refuses to EMPTY an existing trigger, checked after the mutation so
it covers `clear=[...]`, an emptied form input, and any route added later.
Deliberately asked as "did this edit remove one" rather than "does one exist":
a rule predating the guard has none, and refusing to save it would freeze
precisely the unreachable records that most need fixing.
Deliberately not following arose_from_id, which the human door exempts itself
from because provenance is about auditing what the AGENT changed. That reasoning
does not reach this field — a missing trigger is not a missing explanation, it
is a rule that does not work, and it fails an operator as badly as a session.
15 test fixtures across 6 files were creating rules with no trigger. They now
pass one; that they did not is the point — curation is not a guarantee.
Step 1 of milestone 416 "Retrieval stops guessing a bar". First because every
later step assumes one document shape, and it is much cheaper to guarantee
before a corpus grows than to backfill after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
227 lines
9.7 KiB
Python
227 lines
9.7 KiB
Python
"""The preference write path, and how it differs from a rule's (milestone 399).
|
|
|
|
WHY TWO DOORS AT ALL
|
|
|
|
`create_rule`'s docstring IS the approval gate (#3557): it tells its caller to
|
|
propose, offer three answers, and wait. That is right for a rule — the person
|
|
a rule binds should have agreed to be bound.
|
|
|
|
A preference inverts it. The operator's framing: *"preferences are rules that
|
|
scribe can and should update during use."* A preference that asks every time
|
|
never drifts, and drifting is the whole feature. Reaching one through
|
|
`create_rule(kind=...)` would mean reading it through the gate's prose, and
|
|
when_to_apply="when the moment this fixture stands in for arises",
|
|
the caller would hesitate over exactly the act this kind exists to make
|
|
routine.
|
|
|
|
So the asymmetry is the product, and these tests pin it.
|
|
|
|
TWO PRESENCE CHECKS, NEVER AN ABSENCE
|
|
|
|
The tempting guard is "create_preference's docstring does NOT run the approval
|
|
loop". That is the shape snippet #3352 warns against: an absence check passes
|
|
against a docstring that has been deleted, emptied, or rewritten into
|
|
something else entirely, and it reads as coverage while proving nothing.
|
|
|
|
So the asymmetry is asserted as two PRESENCE facts — the rule door still asks,
|
|
the preference door still says write it — and each fails if its own side is
|
|
tidied away. Synonym families, structure not wording, the same bargain
|
|
test_rule_creation_asks_first strikes.
|
|
"""
|
|
from unittest.mock import AsyncMock, patch
|
|
|
|
import pytest
|
|
|
|
from tests.helpers import fake_rule, plain_rule_detail as _plain_detail
|
|
from tests.helpers import tool_doc as _doc
|
|
|
|
# The tool layer reads its caller from a ContextVar the HTTP transport sets.
|
|
# With no request in flight, the module binds it itself (snippet #2836).
|
|
pytestmark = pytest.mark.usefixtures("_bind_user")
|
|
|
|
MODULE = "scribe.mcp.tools.rulebooks"
|
|
|
|
|
|
# ── the required fields, and why each is required ───────────────────────
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_preference_without_a_trigger_is_refused():
|
|
"""A preference with no `when_to_apply` is inert, not merely incomplete.
|
|
|
|
The trigger is two-thirds of the embedded document, so a record without
|
|
one never surfaces at the moment it applies. Refusing at the tool is the
|
|
difference between an error the writer can fix and a preference that is
|
|
written, stored, and silently never delivered — which looks identical to
|
|
one nobody wrote.
|
|
"""
|
|
create_mock = AsyncMock()
|
|
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
|
|
from scribe.mcp.tools.rulebooks import create_preference
|
|
with pytest.raises(ValueError, match="when_to_apply is required"):
|
|
await create_preference(
|
|
topic_id=10, title="t", statement="s",
|
|
when_to_apply=" ", arose_from_id=42,
|
|
)
|
|
create_mock.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_preference_without_provenance_is_refused():
|
|
"""Provenance is the price of the ungated write.
|
|
|
|
A preference is expected to change as the work teaches it. A corpus that
|
|
drifts with no record of what taught each change is one nobody can audit —
|
|
and the operator's veto over drift depends entirely on being able to read
|
|
why it happened.
|
|
"""
|
|
create_mock = AsyncMock()
|
|
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
|
|
from scribe.mcp.tools.rulebooks import create_preference
|
|
with pytest.raises(ValueError, match="arose_from_id is required"):
|
|
await create_preference(
|
|
topic_id=10, title="t", statement="s",
|
|
when_to_apply="when x", arose_from_id=0,
|
|
)
|
|
create_mock.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_create_preference_stores_the_preference_kind():
|
|
"""The tool's one irreducible job.
|
|
|
|
Asserted on the kwarg reaching the service rather than on the returned
|
|
payload: a tool that accepted the call and wrote a plain rule would
|
|
return something that reads correctly, and the force would be wrong.
|
|
"""
|
|
rule = fake_rule(id=100, kind="preference")
|
|
create_mock = AsyncMock(return_value=rule)
|
|
with patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock), _plain_detail():
|
|
from scribe.mcp.tools.rulebooks import create_preference
|
|
await create_preference(
|
|
topic_id=10, title="Pace hard debugging",
|
|
statement="One step per turn.",
|
|
when_to_apply="during hard debugging",
|
|
arose_from_id=42,
|
|
)
|
|
kwargs = create_mock.call_args.kwargs
|
|
assert kwargs["kind"] == "preference"
|
|
assert kwargs["arose_from_id"] == 42
|
|
assert kwargs["when_to_apply"] == "during hard debugging"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_a_near_duplicate_preference_blocks():
|
|
"""The gate is what lets this corpus be written freely and stay small.
|
|
|
|
The second preference about a thing must UPDATE the first. Two that
|
|
quietly disagree are worse than none: retrieval surfaces whichever scores
|
|
higher, and nobody learns the other exists.
|
|
"""
|
|
from scribe.services.dedup import DuplicateMatch
|
|
dup = DuplicateMatch(id=47, title="Pace hard debugging", similarity=1.0, reason="title")
|
|
create_mock = AsyncMock()
|
|
with patch(f"{MODULE}.dedup_svc.find_duplicate_rule", AsyncMock(return_value=dup)), \
|
|
patch(f"{MODULE}.rulebooks_svc.create_rule", create_mock):
|
|
from scribe.mcp.tools.rulebooks import create_preference
|
|
out = await create_preference(
|
|
topic_id=10, title="Pace hard debugging", statement="s",
|
|
when_to_apply="when", arose_from_id=42,
|
|
)
|
|
assert out["duplicate"] is True
|
|
assert out["existing_id"] == 47
|
|
create_mock.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_updating_a_preference_without_provenance_is_refused():
|
|
update_mock = AsyncMock()
|
|
with patch(f"{MODULE}.rulebooks_svc.update_rule", update_mock):
|
|
from scribe.mcp.tools.rulebooks import update_preference
|
|
with pytest.raises(ValueError, match="arose_from_id is required"):
|
|
await update_preference(rule_id=5, arose_from_id=0, statement="new")
|
|
update_mock.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_update_preference_forwards_what_taught_the_change():
|
|
rule = fake_rule(id=5, kind="preference")
|
|
update_mock = AsyncMock(return_value=rule)
|
|
with patch(f"{MODULE}.rulebooks_svc.update_rule", update_mock), _plain_detail():
|
|
from scribe.mcp.tools.rulebooks import update_preference
|
|
await update_preference(
|
|
rule_id=5, arose_from_id=99, statement="the new way",
|
|
)
|
|
kwargs = update_mock.call_args.kwargs
|
|
assert kwargs["arose_from_id"] == 99
|
|
assert kwargs["statement"] == "the new way"
|
|
|
|
|
|
# ── the asymmetry, as two presence facts ────────────────────────────────
|
|
|
|
|
|
def test_the_rule_door_still_asks_before_writing():
|
|
"""Half one of the asymmetry. If this fails, the gate was tidied away and
|
|
preferences are no longer the exception — they are just the same thing.
|
|
"""
|
|
doc = _doc(MODULE, "create_rule").lower()
|
|
asks = ("approve", "propose", "ask", "question")
|
|
assert any(w in doc for w in asks), (
|
|
"create_rule's docstring no longer runs the propose-then-approve loop. "
|
|
"The preference path's whole justification is that it is the exception "
|
|
"to this; with the gate gone there is no asymmetry left to justify."
|
|
)
|
|
|
|
|
|
def test_the_preference_door_says_to_write_it():
|
|
"""Half two. The inverting instruction has to be PRESENT, not merely
|
|
unaccompanied by a gate.
|
|
|
|
An agent that has internalised #3557 will hesitate to write or rewrite a
|
|
preference unless told plainly that this door is different. Silence here
|
|
does not read as permission — it reads as an omission, and the caller
|
|
falls back on the behaviour it already knows.
|
|
"""
|
|
for tool in ("create_preference", "update_preference"):
|
|
doc = _doc(MODULE, tool).lower()
|
|
permits = ("expected", "no approval", "without asking", "ordinary work",
|
|
"write it", "not a liberty", "no proposal")
|
|
assert any(w in doc for w in permits), (
|
|
f"{tool}'s docstring no longer tells its caller that writing "
|
|
"without an approval loop is expected. A caller carrying "
|
|
"create_rule's gate will default to asking, and a preference "
|
|
"nothing ever updates is a rule nobody enforces."
|
|
)
|
|
|
|
|
|
def test_the_preference_door_names_the_force_distinction():
|
|
"""The routing test, stated positively (rule 165).
|
|
|
|
The confusion this milestone exists to fix is that a session cannot tell
|
|
which kind it is holding. If the docstring stops drawing the line, the
|
|
tool becomes a second way to write rules.
|
|
"""
|
|
doc = _doc(MODULE, "create_preference").lower()
|
|
assert "rule" in doc and any(
|
|
w in doc for w in ("followed", "binds", "breaks", "consistency")
|
|
), (
|
|
"create_preference's docstring no longer distinguishes a preference "
|
|
"from a rule by force. Without that line the tool is a second door "
|
|
"onto the rulebook with a weaker gate."
|
|
)
|
|
|
|
|
|
def test_the_preference_door_keeps_the_record_out_of_scope():
|
|
"""Preferences shape HOW work is done, never WHAT is recorded.
|
|
|
|
Worth pinning because "record it the way I like it" is the natural next
|
|
reach, and it would make dev-logs and issues idiosyncratic per author —
|
|
while the record is the one thing that has to outlive the person.
|
|
"""
|
|
doc = _doc(MODULE, "create_preference").lower()
|
|
assert "record" in doc, (
|
|
"create_preference's docstring no longer says that a preference does "
|
|
"not change what gets recorded. That boundary is the one a reader "
|
|
"would cross without noticing."
|
|
)
|