bvandeusen/FabledScribe
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases 53 Wiki Activity

test(410): an exactly-one-owner guard replaces the tests that required every surface to repeat itself (#4033)
CI & Build / Python lint (push) Successful in 3s
Details
CI & Build / Plugin hooks (push) Successful in 12s
Details
CI & Build / TypeScript typecheck (push) Successful in 54s
Details
CI & Build / integration (push) Successful in 56s
Details
CI & Build / Python tests (push) Successful in 1m30s
Details
CI & Build / Build & push image (push) Successful in 15s
Details

Browse Source 
Step 6 of milestone 410 "One owner per piece of guidance". CI now keeps the
shape decision #4027 set, so the next feature cannot quietly add a copy.

tests/test_guidance_ownership.py, 33 topics, each with an owner, markers, a
statement distinctive to the owner's full wording, and optional index markers:
- test_every_topic_is_stated_by_its_owner: markers and statement on the owner
- test_no_topic_is_stated_in_full_off_its_owner: the statement appears on no
  other session surface (index, adapter static text and commands, live
  context, other skills). Tool docstrings are not scanned; a contract may
  elaborate the reflex that calls it.
- test_the_index_names_each_reflex_it_points_at: _INSTRUCTIONS keeps a
  one-line pointer for each session-start reflex
- shared_with declares the one deliberate sharing: the note-check question
  lives in create_note and in using-scribe for two different moments,
  already pinned by test_verification_guidance_survives
- test_the_ownership_guards_can_fail shows each guard turning red (rule 167)
- the process topic now keys on get_process's "follow the returned body";
  it had been passing on an unrelated "verbatim" in two other tools

tests/test_instruction_surfaces_agree.py keeps only what ownership cannot
enforce: the fold budget, rules-bind-names-preferences, and using-scribe's
pointer to reporting-back. Retired: the every-session-start-surface ask and
absence tests, the Systems and snippet owner pins (now registry topics), and
the SessionStart-without-ask test (#2497's shape). The push has carried no
rules since milestone 394, and requiring the ask beside every mention of it
would force a copy. The module docstring records where each protection went.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
bvandeusen
2026-09-14 13:29:34 -04:00
co-authored by Claude Opus 5
parent 15621fa873
commit c440c49f5b
2 changed files with 203 additions and 297 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_instruction_surfaces_agree.py
+27 -188
View File
@@ -1,39 +1,35 @@
"""The instruction surfaces must agree on how a rule reaches a session.
"""What the instruction surfaces must say that one owner per topic cannot enforce.
WHY THIS EXISTS
WHY THIS FILE IS SMALLER THAN IT WAS
Rule #119 makes the instruction surfaces the SPECIFICATION for product
behaviour — there is no other place the "load the operator's rules" obligation
is written down, and no code path enforces it. So a surface that states it
differently isn't a documentation slip; it is the product behaving differently.
Until milestone 410 this file pinned REDUNDANCY: every session-start surface —
the MCP `_INSTRUCTIONS`, the plugin's static context, the `using-scribe` skill —
had to restate how a rule reaches a session, what an empty session means, the
Systems reflex and the snippet-recording triggers. That was the design #2494
chose after #2497 (two surfaces disagreeing about who loads the rules) and
#2198 (every hook silently inert): say it everywhere, so no single failure
loses it.
That happened (#2497). `_INSTRUCTIONS` said the SessionStart hook "is the
bridge" for getting rules into a session, while the `using-scribe` skill said to
pull them yourself and treat any push as a bonus. An agent weighting the first
would reasonably skip the pull.
Decision #4027 replaced that with ownership, and tests/test_guidance_ownership.py
now enforces it: every topic stated in full by exactly one owner, with a
one-line pointer in the server's index. The tests that required every surface
to repeat itself were retired there, because they enforced the very duplication
that drifted (#4022). What they protected is still protected:
- "state how to ask for a rule" / "say an empty session is not an empty
rulebook" → the rules topic: owned by using-scribe, indexed in
`_INSTRUCTIONS` with both phrases;
- the Systems reflex and the snippet-recording triggers → their topics'
markers, required on their owners;
- "never name the SessionStart push without stating the ask" (#2497's exact
shape) → retired outright. Since milestone 394 the push carries no rules,
so naming it can no longer imply rules were handled, and requiring every
surface that mentions it to restate the ask would force a copy.
#2198 is the case where that is wrong: every plugin hook was silently inert for
an extended period, and nothing announced it. An agent trusting the push would
have run with no binding rules and no signal — while those rules govern branch,
commit, push and other hard-to-reverse actions.
WHAT STAYS HERE
The asymmetry is the whole argument, and it is what these tests pin: asking
when a rule had already arrived costs one redundant call; not asking when
nothing arrived costs the operator's rules entirely.
MILESTONE 394 SHARPENED IT RATHER THAN RETIRING IT. There is no longer a
resident set to pull, so "no rule in front of me" went from a rare and
suspicious state to the ordinary state of most turns. The instruction that
used to be supplementary — go and ask — is now the only route a rule has, and
the surfaces must additionally say what an EMPTY session means, or a session
reads silence as permission on nearly every turn.
WHAT THIS DOES NOT DO
It cannot tell whether two surfaces contradict each other in prose generally —
that needs a reader. It pins the instructions whose absence is known to be
load-bearing, and the specific shape the #2497 defect took: naming the push
without also stating how to ask.
Properties of a CLAIM rather than of who owns a topic: the index fits the
fold; a surface that says rules bind also names what does not; using-scribe
still sends a session to the reporting-back skill.
"""
from __future__ import annotations
@@ -41,50 +37,6 @@ import pathlib
ROOT = pathlib.Path(__file__).resolve().parents[1]
# THE PULL IS NOW THE ASK (milestone 394). This was `list_always_on_rules`,
# the call that fetched the resident set. There is no resident set and no such
# call: a rule reaches a session by retrieval, and the only thing a session can
# DO about a rule it has not been handed is go looking for one.
#
# So the two halves this file used to pin separately — "pull the resident set"
# and "and retrieve the conditional ones too" — have collapsed into one
# instruction, and it is the load-bearing one rather than the supplementary
# one it used to be.
ASK = 'content_type="rule"'
# A surface must also say what an EMPTY session means, which is the half that
# is newly dangerous. Under residency, "no rule in front of me" was rare and
# suspicious. Under retrieval it is the ordinary state of most turns, so a
# session that reads it as "there is no rule" is wrong on nearly every turn
# rather than occasionally — the #3720 defect at session scale.
#
# Claim phrases, not a single word, for the reason BINDING_CLAIMS gives below:
# a bare "matched" or "silence" appears in prose that is not making this claim
# at all. A surface passes by asserting the distinction however it words it.
ABSENCE_CLAIMS = (
"nothing matched",
"is not the same as \"there is no rule",
"never \"there is no rule",
"silence is not absence",
"not evidence there is none",
)
# The surfaces that STATE the rules reflex. Hand-written because "who owns
# this?" is an editorial fact, not a derivable one — but each entry is asserted
# to EXIST, so a move or rename fails loudly here instead of quietly dropping
# that surface from the check.
#
# Since milestone 410 (decision #4027) that is the owner and the index, not
# every surface a session loads: `using-scribe` states the reflex in full and
# the server's `_INSTRUCTIONS` gives it one line for every MCP client. The
# Claude Code adapter's static context used to be a third copy; it now points
# at the skill instead, and tests/test_guidance_ownership.py keeps the topic
# from falling off.
SESSION_START_SURFACES = (
ROOT / "src" / "scribe" / "mcp" / "server.py",
ROOT / "plugin" / "skills" / "using-scribe" / "SKILL.md",
)
def _all_surfaces() -> list[tuple[str, str]]:
"""(label, text) for every file a SESSION loads as instructions.
@@ -105,54 +57,6 @@ def _all_surfaces() -> list[tuple[str, str]]:
return found
def test_every_session_start_surface_states_the_ask():
"""Retrieval is the only delivery, so asking is the only recourse."""
missing = []
for path in SESSION_START_SURFACES:
assert path.exists(), (
f"{path.relative_to(ROOT)} is gone — it was one of the surfaces "
f"carrying the rules instruction. If it moved, update "
f"SESSION_START_SURFACES; if it was retired, check the instruction "
f"still lives somewhere a fresh session reads."
)
if ASK not in path.read_text():
missing.append(str(path.relative_to(ROOT)))
assert not missing, (
f"these surfaces never tell the agent how to ask for a rule "
f"({ASK}): {missing}. Nothing is pushed and nothing is resident, so a "
f"surface that omits this leaves a session with no way to reach a rule "
f"it was not handed — bound by nothing (#2198, #2497, milestone 394)."
)
def test_every_session_start_surface_says_an_empty_session_is_not_an_empty_rulebook():
"""The half that got dangerous when residency went away.
Under the old model a session opened holding every applicable rule, so
"nothing is in front of me" was a rare state and a suspicious one. Under
retrieval it is the NORMAL state of most turns. A surface that describes
where rules come from, without also saying what their absence means, leaves
a session reading silence as permission — on nearly every turn rather than
occasionally.
That is #3720's defect ("absence reads as non-existence") moved from a
readout to the session itself, and this milestone is what makes every
session start in the absent state.
"""
missing = []
for path in SESSION_START_SURFACES:
text = path.read_text().lower()
if not any(c.lower() in text for c in ABSENCE_CLAIMS):
missing.append(str(path.relative_to(ROOT)))
assert not missing, (
f"these surfaces say how a rule arrives but never what it means when "
f"none does: {missing}. 'No rule arrived' means 'nothing matched', "
f"never 'there is no rule' — and only one of those has been checked. "
f"Say it however you like; one of {ABSENCE_CLAIMS} is what this looks "
f"for."
)
def _instructions_text() -> str:
"""The _INSTRUCTIONS literal from server.py, as the client would see it."""
import re
@@ -180,71 +84,6 @@ def test_instructions_fit_the_fold():
)
def test_the_systems_reflex_is_stated_by_its_owner():
"""Write-time tagging guidance must be stated as a reflex, not only per tool.
#2562's behavioral finding: with the guidance only in tool descriptions,
sessions filed records untagged. The tag-as-you-write reflex was pinned on
the static context then; since milestone 410 its owner is using-scribe
(decision #4027), with the in-band `systems_hint` as the half that fires on
its own.
"""
owner = (ROOT / "plugin" / "skills" / "using-scribe" / "SKILL.md").read_text()
for needle in ("system_ids", "create_system", "systems_hint"):
assert needle in owner, (
f"plugin/skills/using-scribe/SKILL.md no longer mentions "
f"{needle} — the Systems tagging reflex must be stated by its "
f"owner, not only in tool descriptions (#2562)."
)
def test_the_snippet_recording_triggers_are_stated_by_their_owner():
"""The pattern-library recording model must be stated, by name.
#2664's behavioral finding: recording guidance as a trailing clause of the
reuse bullet converted zero times outside snippet-minded sessions. The
2026-08-16 ruling (decision #2686) then replaced the reactive model
entirely: every shape is recorded at FIRST build — no "will it recur?"
judgment — and second-copy consolidation is only the backstop. All three
elements must stay stated: the tool, the first-build trigger, and the
backstop. Since milestone 410 the owner is the reusing-code skill.
"""
owner = " ".join((ROOT / "plugin" / "skills" / "reusing-code" / "SKILL.md").read_text().split())
for needle in ("create_snippet", "first build", "second copy"):
assert needle in owner, (
f"plugin/skills/reusing-code/SKILL.md no longer states the "
f"snippet-recording model ({needle!r}) — record-every-shape-at-"
f"first-build with second-copy consolidation as the backstop must "
f"be stated by its owner (#2664, decision #2686)."
)
# The "displaced from _INSTRUCTIONS" topics (#2562) used to be listed here with
# their own delivered-surface check. They are folded into the one topic registry
# in tests/test_guidance_ownership.py (milestone 410), which covers every topic
# of the ownership map and defines "delivered surface" once.
def test_no_surface_names_the_push_without_stating_the_ask():
"""The exact shape #2497 took.
Mentioning the SessionStart hook is fine and often useful. Mentioning it
*instead of* the pull is the defect: it reads as "this is handled", and the
surface that says so is the one an agent has least reason to doubt.
"""
offenders = [
label for label, text in _all_surfaces()
if "SessionStart" in text and ASK not in text
]
assert not offenders, (
f"these surfaces describe the SessionStart push but never state how to "
f"ask: {offenders}. The push is a delivery optimisation, not the "
f"bridge — it can be absent without saying so, and since milestone 394 "
f"it carries no rules at all. Name it if it helps, but say how to ask "
f"({ASK}) regardless."
)
# ── force: a surface that says rules bind must say what does not ────────
#
# Added with the preference kind (milestone 399). Before it, "rules bind" was