Compare commits

...
4 Commits
Author SHA1 Message Date
bvandeusenandClaude Opus 5 dd1e6e2645 feat(rules): the rule arms stop filtering the corpus to one tier (#3702)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 1m3s
CI & Build / Python lint (push) Successful in 4s
CI & Build / integration (push) Successful in 33s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 1m29s
Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:

  PRESENT IN CONTEXT    the rule was delivered at session start
  SALIENT AT THE MOMENT the rule is in front of the reader when the action
                        it governs is about to be taken

A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.

Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.

MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.

Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.

The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-08 00:24:02 -04:00
bvandeusenandClaude Opus 5 c3ecdf0972 feat(rules): the rule gate becomes a practice with a question, not a prohibition (#3557)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / TypeScript typecheck (push) Successful in 11s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 25s
The first cut opened "NOT YOURS TO CALL UNPROMPTED", and that is the wrong
instrument. A caller reading a prohibition stops NOTICING rule-shaped things
rather than noticing them and asking — which trades a small failure for a
larger one. The wanted behaviour is more proposals, not fewer.

So both docstrings now describe the practice: propose readily, state the
four things, and close with a question the operator answers in one word —
approve it as written / let's talk about it / no. Named options where the
interface has them, three written-out options where it does not.

"Approve it as written" is what makes element 1 load-bearing: they approved
TEXT, so that text is stored verbatim. "Let's talk about it" is framed as
the expected answer rather than a setback. "No" routes the observation to
create_note, which records without binding.

The argument for asking is also better than consent. The operator's yes is
the one moment the rule is certainly in front of them: afterwards a
conditional rule is not read aloud at session start, and a project rule is
absent from an unfiltered list_rules(). The proposal IS the review.

Guard gains the answers-offered-back element and drops the wording that
forbade; its header records why the framing changed, so the prohibition does
not get reintroduced as a tidy-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-04 20:11:42 -04:00
bvandeusenandClaude Opus 5 1a34363059 feat(rules): the rule-creation tools ask for approval, and ask what would enforce it (#3557)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 32s
CI & Build / TypeScript typecheck (push) Successful in 34s
CI & Build / Python tests (push) Successful in 1m6s
CI & Build / Build & push image (push) Successful in 41s
Every gate on create_rule and create_project_rule was about SHAPE — rule vs
process vs snippet, one-thing-you-could-violate, general-enough, not-a-dupe.
All of them improve a rule someone has already decided to write. None asked
the prior question: has the person this will bind agreed to be bound by it?

Both docstrings now open with the gate, and with the four things a proposal
carries: what it would require in the words it would carry, its intent, why
now, and how it would be enforced.

The fourth is the one that decides it. "A test, a CI check, a hook, a schema
constraint... or nothing" is a question that sometimes dissolves the rule:
what a test can assert should BE that test, and a rule is what is left when
nothing mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose.

create_project_rule needs the gate more, not less, and says so: a project
rule is absent from an unfiltered list_rules(), and a conditional one is
absent from session start too, so one written there can bind for months
without ever having been in front of the person it binds.

test_rule_creation_asks_first pins structure, never wording — each element
matches a family of synonyms, and the gate must precede the Args: block,
because a caller who has decided to make the call reads the parameters and
not the prose under them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-04 20:02:05 -04:00
bvandeusenandClaude Opus 5 30d87e461a feat(rules): the instruction surfaces say to RETRIEVE a rule, not only to receive one (#3523)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 32s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / TypeScript typecheck (push) Successful in 4m18s
CI & Build / Build & push image (push) Successful in 25s
Every instruction surface told a session to load the always-on rules and
stopped there. None said the loaded set is partial, so an empty one read as
"no rule applies" when it only ever meant "none was pushed" — different
claims, and only one of them has been checked.

That is #2198's asymmetry one level in. The earlier defect was trusting the
SessionStart push over the explicit pull; this is trusting the resident TIER
as if it were the whole rulebook.

It is also why the always-on tier was the only one that worked, on any install
rather than this one (rule 115): a rule nothing retrieves must be resident to
bind at all, and a resident rule costs tokens in every session forever — so a
rulebook that only delivers cannot grow past what one session holds.
Retrieval lifts that ceiling, and it fires only if something asks. A
tool-choice reflex asks least of all (#3476, #161).

The same obligation now lands on all three session-start surfaces, because
rule 119 makes them the specification jointly and a surface stating it
differently IS the product behaving differently (#2497). Pinned by
test_every_session_start_surface_states_the_conditional_retrieval, mirroring
the pull test beside it.

THE BUDGET TRADE. _INSTRUCTIONS sat at 1978 against a 2000 test budget, and
its own comment says an addition there is a trade, never an append. Bought the
new clause by trading out "Processes are saved procedures (follow verbatim)"
and "Deletes are trash-recoverable" — both already in DISPLACED_TOPICS and
already stated on a delivered surface, and both per-tool guidance, which by
this block's doctrine belongs in the tool docstring. Now 1976. Recorded in the
comment above the block so it is not silently reversed.

Plugin version minted: shipped plugin content moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-03 21:41:53 -04:00
10 changed files with 369 additions and 25 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "scribe",
"description": "Scribe system-of-record for Claude Code: MCP tools over your notes/tasks/projects/rules, a session-start push channel that surfaces your always-on rules + active-project context, process-skills (writing-plans, systematic-debugging, verification, brainstorming, reusing-code), and your saved Scribe Processes auto-surfaced as skills (/scribe:sync). Replaces superpowers + file-memory with one app-backed plugin.",
"version": "2026.09.03.0329",
"version": "2026.09.04.0140",
"author": {
"name": "Bryan Van Deusen"
},
+10
View File
@@ -21,6 +21,16 @@ for the operator's work, and as your own working memory across sessions.
compaction — call `list_always_on_rules()` (and `enter_project()` when a
project is in scope) BEFORE acting. When a loaded rule and a default habit
disagree, the rule wins; if no rule speaks to it, ask rather than assume.
- **What you loaded is not all of the rules.** Only the always-on tier arrives
that way; conditional rules are RETRIEVED, and one you were never handed
binds exactly as hard. So before a consequential act, `search` for a rule
about it (`content_type="rule"`) rather than concluding from an empty
loaded set that nothing applies. "I was not told" is not the same as "there
is no rule," and only one of those is checkable.
This bites hardest on which TOOL to reach for — curling an API that has an
MCP client, standing up a local stack, running a suite CI owns. Those feel
like mechanics rather than decisions, so they raise no doubt and generate no
query; the moment you are most confident is the moment to look.
- **Recall before acting** — before you answer anything about the operator's
work or start a task, `search` Scribe first; assume a related note, task, or
decision already exists. Concretely, reach for recall whenever a request
+25 -4
View File
@@ -56,10 +56,31 @@ Two constraints on *how* that's achieved:
re-deriving it or opening a duplicate. When a project is in scope, pass its
`project_id` so results stay scoped.
2. **Standing rules are binding.** Load them via `list_always_on_rules()` at
session start (see "Do this first"); treat every one as binding. Pull a
rule's full statement with `get_rule(id)` when it's about to bite. When a
project is in scope, `enter_project(id)` also returns its applicable rules.
2. **Standing rules are binding — and the ones you were handed are not all of
them.** Load the resident set via `list_always_on_rules()` at session start
(see "Do this first"); treat every one as binding. Pull a rule's full
statement with `get_rule(id)` when it's about to bite. When a project is in
scope, `enter_project(id)` also returns its applicable rules.
Rules come in two tiers. **Always-on** rules are delivered — they arrive
whether or not you ask. **Conditional** rules are RETRIEVED, and one binds
just as hard for never having been handed to you. So before a consequential
act, `search(content_type="rule")` on what you are about to do. An empty
loaded set is not evidence that no rule applies; it is only evidence that
none was pushed, and those are different claims.
The tier split exists because delivery does not scale: every resident rule
costs tokens in every session forever, so a rulebook that grows past a few
dozen either stops growing or stops fitting. Retrieval is what lets the
rulebook keep growing — but retrieval only fires if something asks.
**Ask hardest where you feel most certain.** Rules about which TOOL to reach
for — use the forge's MCP client rather than curling its API, don't stand up
a local stack, don't run the suite CI owns — govern moves that feel like
mechanics rather than decisions. A reflex raises no doubt, so it generates
no query, so the rule that would have stopped it is never retrieved. That is
the failure this instruction exists to prevent, and confidence is its only
warning sign.
3. **Update over duplicate.** When recording, prefer updating an existing
note/rule/task over creating a new one. Search first; revise what's there.
+28 -3
View File
@@ -45,6 +45,31 @@ from quart import Quart
# The accepted cost: an agent that never opens create_note's docstring never
# learns the field exists. Guidance lives in the create_note / update_note
# docstrings and the using-scribe skill instead.
#
# Milestone 333 step 3 (2026-09-04) bought the HOW bullet's second clause —
# search(content_type="rule") before a consequential act — by TRADING OUT
# "Processes are saved procedures (follow verbatim)" and "Deletes are
# trash-recoverable". Recorded so the trade is not silently reversed:
# - Both were already in test_instruction_surfaces_agree's DISPLACED_TOPICS
# and already stated on a delivered surface, so nothing fell off: the
# process reflex is in every scribe-proc-* skill listing (each says the
# process governs and is followed verbatim), and trash recovery is in the
# delete_*/list_trash/restore docstrings, which is where per-tool guidance
# belongs by this block's own doctrine.
# - What it bought is not per-tool guidance and has nowhere else to live at
# session-start altitude. Rules were retrievable only by RESIDENCY: the
# always-on preload put them in front of the agent, and nothing told a
# session to go looking for one it had not been handed. The tier split is
# therefore load-bearing on ANY install (rule 115): a delivered rule costs
# tokens in every session forever, so a rulebook that only delivers cannot
# grow past what one session can hold, and every rule worth keeping has to
# become resident to bind at all. Retrieval is what lets it keep growing —
# and retrieval fires only if something asks, which nothing told a session
# to do. A tool-choice reflex asks least of all (#3476, #161).
# - This states the PULL for conditional rules, exactly as the surrounding
# line states it for always-on ones. Rule 119 makes these surfaces the
# specification, so the same sentence lands on all three session-start
# surfaces, and test_instruction_surfaces_agree pins it.
_INSTRUCTIONS = """
Scribe is the operator's self-hosted second brain and system of record — and
yours: recall from it before acting, record as you go. Keep no parallel copy
@@ -63,13 +88,13 @@ Hierarchy: Project -> Milestone -> Task/Note. The map, by purpose:
active project_id to stay in scope.
- WHERE work happens: Systems. Tag records with system_ids as you write;
create_system when the area is unmodelled.
- HOW: rules are binding — list_always_on_rules() at session start.
- HOW: rules bind. list_always_on_rules() at start; before a consequential
act, search(content_type="rule") — the resident set is not all of them.
- UI: the project's design system is binding — resolve_design_system /
get_design_system_stylesheet before hand-writing a value.
- REUSE: search snippets before writing a helper; record what you build with
create_snippet; classify shapes against canon (classify_shapes) — a
consumer map is rows, never prose. Processes are saved procedures (follow
verbatim). Deletes are trash-recoverable.
consumer map is rows, never prose.
A task is a note with status (*_note vs *_task tools).
Creates are duplicate-gated: a near-match BLOCKS and returns the existing
+65
View File
@@ -315,6 +315,61 @@ async def create_rule(
) -> dict:
"""Create a new rule in a rulebook (a SHARED rule — keep it general).
PROPOSE RULES READILY, AND WRITE ONE WHEN THE OPERATOR SAYS YES. Noticing
that something has hardened into a standing instruction is valuable work,
and a session that notices it and says nothing has thrown the observation
away. So raise it whenever you see one. The single step that belongs
between noticing and writing is the operator's yes: a rule binds every
future session, and they are the person it binds.
Their yes is also the only moment the rule is reliably IN FRONT of them.
After the write it may not be again for months — a conditional rule is not
read aloud at session start, and a project-scoped one does not appear in
an unfiltered list_rules() at all. So the proposal is the review.
When the operator asks for a rule in so many words, that IS the yes —
write it and move on. The loop below is for the rule you thought of.
A PROPOSAL CARRIES FOUR THINGS, and the fourth is the one that decides it:
1. WHAT it would require — the statement, in the words it would carry,
not a gloss of them. The operator is agreeing to text.
2. INTENT — what it changes about how work gets done, and what goes
wrong today without it. "Be careful about X" is not an intent; the
behaviour that would differ tomorrow is.
3. WHY NOW — the incident, observation or decision behind it. Pass that
record as arose_from_id, and say it in the conversation too: the
field is for the reader six months out, the sentence is for the
person deciding.
4. HOW IT WOULD BE ENFORCED — a test, a CI check, a hook, a schema
constraint, a duplicate gate, a review step... or nothing, in which
case say so plainly: "nothing — this is prose a session has to
remember." Answer this one honestly and it will sometimes dissolve
the rule, which is the point rather than a side effect. What a test
can assert should BE that test; a rule is what remains when nothing
mechanical can hold the thing. A rulebook grows by default and
shrinks only on purpose, so a question that prevents a rule is worth
more than any question that improves one's wording.
THEN CLOSE WITH A QUESTION THEY CAN ANSWER IN ONE WORD. Offer three
answers, and make the middle one the easy one:
* "Approve it AS WRITTEN" — you create it with the statement exactly as
shown. This is what makes element 1 load-bearing: they approved TEXT,
so that text is what gets stored, verbatim.
* "LET'S TALK ABOUT IT" — the wording, the scope, the tier, whether it
wants to be a rule at all. Most good rules arrive this way, so treat
this answer as the expected one rather than a setback.
* "NO" — let it go. If the observation is still worth keeping, it is a
note (create_note): recorded, findable, and binding on nobody.
Where the interface offers structured choices, ask it that way — a
question with named options is answered in a click, while the same
question inside a paragraph is answered by scrolling past. Where it does
not, write the three options out as three options. Either way ask once
and let the answer stand; re-raising a declined proposal argues a rule
into existence, which is the thing this whole loop exists to prevent.
A rulebook rule is shared by every project that gets the rulebook: an
always_on rulebook binds ALL your projects; a subscribed rulebook binds the
projects that opt in. So a rulebook rule must read as a general standard —
@@ -433,6 +488,16 @@ async def create_project_rule(
the rule is returned in get_project's applicable_rules (under
project_rules) and in list_rules(project_id=...).
PROPOSE, THEN WRITE ON A YES — create_rule's opening carries the whole
loop: the four things a proposal states (what it would require, its
intent, why now, and how it would be enforced) and the one-word question
that closes it (approve as written / talk about it / no). All of it
applies here unchanged. Reach for that loop MORE readily on this surface,
not less: a project rule stays out of an unfiltered list_rules(), and a
conditional one stays out of session start too, so the operator's yes is
the one moment this rule is certain to have been seen by the person it
binds.
Check first whether a rule is the right shape at all — create_rule's
opening asks that question and it applies identically here. A visual
standard is a design system; a procedure is a process (create_process);
+11 -4
View File
@@ -774,10 +774,17 @@ async def semantic_search_rules(
is the surfacing question, and it has its own machinery
(get_applicable_rules) rather than a second, subtly different copy here.
`tier` narrows to one tier. The write-path hint passes "conditional",
because an always-on rule is ALREADY in the session — surfacing it again as
a suggestion is pure noise, and noise on a hint that fires on every write
is how a hint gets ignored.
`tier` narrows to one tier, and NONE is the ordinary case. The write-path
and pre-tool hints deliberately pass nothing: an always-on rule is already
in the session, but being in a list from turn zero is not the same as being
in front of the reader when the action it governs is taken, and filtering
on tier made a whole class of rules permanently ineligible for the one
mechanism that surfaces a rule AT the moment. Relevance is the threshold's
job; see the block above RULEHINT_LIMIT in services/plugin_context.py for
the argument and for what the resulting scores are being read against.
Pass a tier when a caller genuinely wants one class — a listing, an audit,
a UI that renders the tiers apart. Not to approximate relevance.
Collapses to best-chunk-per-rule like the note search, so a long rule split
across chunks competes once rather than crowding the results with itself.
+60 -12
View File
@@ -94,13 +94,17 @@ WRITEPATH_DEFAULT_THRESHOLD = 0.68
# THE STRUCTURAL ARGUMENT, which is the only kind admissible here (rule 115).
# Two facts hold on any install, including one with six rules and no telemetry:
#
# 1. The eligible corpus is TINY. The arm searches `tier="conditional"`
# rules only — a handful to a few dozen documents against thousands of
# notes. A top-k over forty candidates always returns something, so
# "the best match cleared the bar" stops meaning "a good match exists"
# and starts meaning "forty things were ranked". A bar calibrated for
# best-of-thousands is cleared by best-of-forty as arithmetic, not
# relevance.
# 1. The eligible corpus is SMALL — every rule an install owns, still only
# a few dozen documents against thousands of notes. A top-k over a small
# pool always returns something, so "the best match cleared the bar"
# drifts from "a good match exists" toward "N things were ranked". A bar
# calibrated for best-of-thousands is cleared by best-of-forty as
# arithmetic rather than relevance.
# This argument WEAKENED when the arms stopped filtering to one tier
# (see the note on that below): a larger pool makes clearing the bar
# mean more, not less. The threshold was deliberately left where it was
# anyway — moving two variables at once would make the resulting
# distribution unreadable, and this one errs toward silence on purpose.
# 2. Rules are short imperative technical English — a far more HOMOGENEOUS
# corpus than note prose. #2223 measured the floor for code against prose
# at 0.55-0.63 and set 0.68 above it. A more homogeneous corpus has a
@@ -128,9 +132,14 @@ RULEHINT_DEFAULT_THRESHOLD = 0.72
# ONE rule per write, not two — and this is deliberately NOT a knob.
#
# With a corpus this small, top-k does as much damage as the threshold: k=2
# over forty candidates means the second line is almost always the second-best
# noise, arriving with the same confident framing as the first. Halving k
# halves that regardless of where the bar sits.
# over a few dozen candidates means the second line is almost always the
# second-best noise, arriving with the same confident framing as the first.
# Halving k halves that regardless of where the bar sits.
#
# It also BOUNDS the blast radius of widening the pool (below): with k=1 a
# wider corpus can change WHICH rule surfaces and how often one does, but it
# can never make a single hint longer. The loudness of one hint and the
# eligibility of a rule are separate controls, and only one of them moved.
#
# It stays a constant because it is a decision about how LOUD one hint may be,
# not a per-install tuning question. The hint already carries prior art, shape
@@ -140,6 +149,45 @@ RULEHINT_DEFAULT_THRESHOLD = 0.72
# adds a way to misconfigure the surface (rule 25 cuts both ways).
RULEHINT_LIMIT = 1
# WHY THE ARMS NO LONGER FILTER TO ONE TIER (#3702).
#
# Both arms used to pass `tier="conditional"`, on the reasoning that an
# always-on rule is already in the session, so surfacing it again is pure
# noise. That reasoning conflates two different things:
#
# PRESENT IN CONTEXT — the rule was delivered at session start.
# SALIENT AT THE MOMENT — the rule is in front of the reader when the
# action it governs is about to be taken.
#
# A rule handed over in a list at turn zero is present while a session writes
# a config value three hundred turns later. It is not surfaced. So the filter
# did not merely skip a redundant hint — it made a whole class of rules
# permanently ineligible for the only mechanism that puts a rule in front of
# an agent AT the moment, and the more important a rule is, the more likely
# it was in that class.
#
# The deeper defect is that the filter was doing the THRESHOLD's job. Whether
# a rule belongs in this hint is a relevance question, and a similarity bar is
# the control for relevance. A categorical exclusion standing in for a
# relevance judgment cannot be tuned, cannot be measured, and cannot be wrong
# in a way anybody notices.
#
# THIS IS A MEASURED CHANGE, NOT A SETTLED ONE. The old comment's fear is
# real — a hint that fires on every write and says obvious things teaches the
# reader to skip the block, and the surface is then lost along with its true
# positives. That fear had simply never been checked. `retrieval_logs` already
# records top_score, result_count and the query for every call, so the
# evidence now arrives on its own:
#
# - rules clear the bar often and at high scores -> the fear was justified,
# the filter was a crude proxy for a bar set too low, and the WORK IS THE
# BAR. Any reinstated filter should then carry a measured reason.
# - rules clear rarely, in a thin band near the bar -> the filter was never
# the right instrument and relevance was always sufficient.
#
# Only the eligibility moved. The bar and k=1 were both left exactly where
# they were, so the resulting distribution has one cause.
# How much of a command reaches the embedding (#3476). A shell call is not a
# file: most are short, and the ones that are not are usually a heredoc or a
# pasted script whose bulk says nothing about which rule applies. The VERB AND
@@ -1205,7 +1253,7 @@ async def build_write_path_hint(
rule_t0 = time.perf_counter()
hits = await semantic_search_rules(
user_id, code or path, limit=RULEHINT_LIMIT,
threshold=cfg["rule_threshold"], tier="conditional",
threshold=cfg["rule_threshold"],
)
rule_ms = (time.perf_counter() - rule_t0) * 1000.0
fresh = [(score, rule) for score, rule in hits if rule.id not in already]
@@ -1337,7 +1385,7 @@ async def build_tool_rule_hint(
t0 = time.perf_counter()
hits = await semantic_search_rules(
user_id, query, limit=RULEHINT_LIMIT,
threshold=cfg["rule_threshold"], tier="conditional",
threshold=cfg["rule_threshold"],
)
duration_ms = (time.perf_counter() - t0) * 1000.0
+40
View File
@@ -195,6 +195,46 @@ def test_displaced_topics_live_on_a_delivered_surface():
)
# The SECOND pull (milestone 333 step 3). `list_always_on_rules()` fetches the
# resident tier; this one says that tier is not all of them, and that a
# conditional rule has to be gone looking for. However a surface words the
# surrounding prose, it names the call.
RETRIEVE = 'content_type="rule"'
def test_every_session_start_surface_states_the_conditional_retrieval():
"""The push/pull asymmetry, one level in.
The tests above pin that a session PULLS the resident rules rather than
trusting the SessionStart push. This pins the same shape between the two
TIERS: an always-on rule is delivered, a conditional one is retrieved, and
a surface that states only the first leaves a session reading its loaded
set as the whole rulebook.
That reading is wrong in the direction that costs something. "Nothing was
pushed" and "no rule applies" are different claims, and only one of them
has been checked — the same asymmetry as #2198, now between tiers instead
of between channels.
It is also what made the always-on tier the only one that worked, on any
install rather than this one (rule 115). A rule nothing retrieves has to be
resident to bind at all, so every rule worth keeping becomes resident; and
a resident rule costs tokens in every session forever, so a rulebook that
only delivers cannot grow past what one session can hold. Retrieval is what
lifts that ceiling — and it only fires if something asks.
"""
missing = []
for path in SESSION_START_SURFACES:
if RETRIEVE not in path.read_text():
missing.append(str(path.relative_to(ROOT)))
assert not missing, (
f"these surfaces state the always-on pull but never tell the agent to "
f"retrieve a conditional rule ({RETRIEVE}): {missing}. A session that "
f"reads its loaded set as the whole rulebook will act on \"I was not "
f"told\" as if it meant \"there is no rule\" (milestone 333 step 3)."
)
def test_no_surface_names_the_push_without_stating_the_pull():
"""The exact shape #2497 took.
+120
View File
@@ -0,0 +1,120 @@
"""Both rule-creation tools run the propose-then-approve loop (#3557).
WHY THIS EXISTS
Every other gate on `create_rule` and `create_project_rule` is about SHAPE:
is this a rule or a process, is it one thing you could violate, is it general
enough for a rulebook, is it a near-duplicate. All of those improve a rule
someone has already decided to write. None of them asks the prior question —
whether the person the rule will bind has agreed to be bound by it.
That question belongs at the tool, because the tool is the last surface a
caller reads before the write, and because the write is less reversible than
it looks. The operator's yes is not merely consent; it is the one moment the
rule is certainly IN FRONT of them. Afterwards it may not be again for
months: a conditional rule is not read aloud at session start, and a
project-scoped rule does not appear in an unfiltered `list_rules()` at all.
The proposal IS the review, so there had better be one.
WHY IT IS PHRASED AS A PRACTICE AND NOT A PROHIBITION
The first cut of this guidance opened "NOT YOURS TO CALL UNPROMPTED." That is
the wrong instrument, and the failure it invites is worse than the one it
prevents: a caller reading a prohibition stops NOTICING rule-shaped things,
rather than noticing them and asking. The wanted behaviour is more proposals,
not fewer — spotting that something has hardened into a standing instruction
is valuable work, and the only step that was ever missing came after it.
So the docstrings describe what to DO: propose readily, state four things,
close with a question the operator answers in one word. This test is written
the same way — it asserts the parts of the loop are present, and has nothing
to say about any wording that forbids.
The fourth element — how the rule would be ENFORCED — is not ceremony. It is
the part that sometimes dissolves the rule: a thing a test can assert should
be that test, and a rule is what is left when nothing mechanical can hold it.
A rulebook grows by default and shrinks only on purpose, so the question that
prevents a rule earns more than any question that improves one's wording.
WHAT THIS PINS, AND WHAT IT DOES NOT
STRUCTURE, never wording — the same bargain the disambiguator guard (#3123)
strikes next door. Each element matches a family of synonyms, so the prose
stays free to be rewritten, reordered or sharpened; only DELETING one fails.
Pinning phrasing would make every improvement a red build, and a test that
punishes editing is a test someone deletes.
It cannot tell whether an agent actually proposes. Nothing in a docstring
can. It catches the regression that really happens: guidance tidied away in
a later pass by someone who read it as throat-clearing in front of the Args.
"""
import pytest
from tests.helpers import tool_doc as _doc
# Both surfaces, because the one that needs it most is the one that looks
# minor. A project rule is the least visible record the system can hold —
# absent from an unfiltered list_rules(), and absent from session start too
# whenever it is conditional — so the surface that writes one carries the
# larger risk while reading as the smaller act.
_SURFACES = [
("scribe.mcp.tools.rulebooks", "create_rule"),
("scribe.mcp.tools.rulebooks", "create_project_rule"),
]
# The loop, element by element, each as a family of ways to say it. A
# docstring satisfies an element by containing ANY member — that is the room
# left for rewriting. The families deliberately exclude bare words a
# docstring would hold by accident ("why", "how", "reason", "rule"), which
# would let the assertion pass on prose that says nothing of the kind.
_ELEMENTS = {
"the invitation to propose": ("propose", "proposal"),
"the operator's approval": ("approve", "approval", "says yes", "a yes"),
"the rule's intent": ("intent", "what it changes about how work"),
"why it is being proposed now": (
"why now", "arose_from_id", "the incident", "prompted it",
),
"how it would be enforced": ("enforc",),
"the answers offered back": ("as written", "talk about it", "discuss"),
}
@pytest.mark.parametrize(("module", "name"), _SURFACES)
@pytest.mark.parametrize("element", sorted(_ELEMENTS))
def test_a_rule_surface_carries_every_part_of_the_proposal_loop(
module, name, element
):
"""Each element of propose → state four things → ask survives."""
doc = _doc(module, name).lower()
assert any(token in doc for token in _ELEMENTS[element]), (
f"{name}'s docstring no longer mentions {element}. A caller reads "
f"this immediately before writing a rule that will bind every future "
f"session, and the proposal is the one moment that rule is certain to "
f"be seen by the operator. Say it in whatever words you like; this "
f"guard only checks it is still said. See create_rule's opening."
)
@pytest.mark.parametrize(("module", "name"), _SURFACES)
def test_the_proposal_loop_comes_before_the_parameter_contract(module, name):
"""It has to be read to work, and the Args: block is where reading stops.
A caller who has decided to make the call skims down to the parameters.
Guidance parked below them — or folded into one argument's description —
arrives after the decision it was meant to inform, which is the same as
not being there.
"""
doc = _doc(module, name).lower()
args_at = doc.find("args:")
assert args_at > 0, f"{name}'s docstring has no Args: block"
loop_at = min(
(doc.find(t) for t in _ELEMENTS["the invitation to propose"]
if doc.find(t) >= 0),
default=-1,
)
assert 0 <= loop_at < args_at, (
f"{name} introduces the proposal loop at or after its Args: block "
f"(loop {loop_at}, args {args_at}). Move it to the opening — a "
f"caller who has already decided to write the rule reads the "
f"parameters, not the prose under them."
)
+9 -1
View File
@@ -160,7 +160,15 @@ async def test_the_arm_searches_on_its_OWN_bar_not_the_code_one():
kw = search.await_args.kwargs
assert kw["threshold"] == 0.81, "the arm is still using the code threshold"
assert kw["limit"] == pc.RULEHINT_LIMIT
assert kw["tier"] == "conditional"
# NO tier filter (#3702). The arms search every rule the caller owns,
# because "already in the session" is not the same as "in front of the
# reader at the moment it applies" — and relevance is the threshold's
# job, not a category's. If this assertion is failing because a tier
# argument came back, read the block above RULEHINT_LIMIT first: the
# filter may legitimately return, but only carrying a measured reason.
assert "tier" not in kw or kw["tier"] is None, (
"the arm is filtering the rule corpus by tier again"
)
@pytest.mark.asyncio