Retire the always-on tier — every rule arrives by retrieval (milestone 394) #152
@@ -420,6 +420,14 @@ async def create_rule(
|
||||
brought it back as the top hit. Where a rule prevents a specific
|
||||
failure, put that failure's vocabulary here — the error text,
|
||||
the wrong behaviour, the dead end.
|
||||
The two spellings, side by side:
|
||||
RETRIEVES: "the migration failed with a check violation on a
|
||||
column we just extended"
|
||||
COLLAPSES: "when working on migrations"
|
||||
The second names a CATEGORY. No session ever produces a
|
||||
category — it produces the command, the error, the half-formed
|
||||
ask — so a trigger written that way leaves the embedded
|
||||
document to be carried by the title alone.
|
||||
tier: "always_on" (default) or "conditional".
|
||||
The test: can you name the trigger WITHOUT naming a system, an
|
||||
artifact type or a moment? If the honest answer is "whenever you
|
||||
@@ -519,10 +527,11 @@ async def create_project_rule(
|
||||
when_to_apply: WHEN this rule fires — the trigger, not the
|
||||
instruction, and the rule's retrieval surface: name the SYMPTOM,
|
||||
the words someone would type while stuck. Show the moment rather
|
||||
than classifying it — "the migration failed with a check
|
||||
violation on a column we just extended" retrieves; "when working
|
||||
on migrations" does not, because it names a category no session
|
||||
ever types. See create_rule for the full argument. It informs the
|
||||
than classifying it:
|
||||
RETRIEVES: "the CI job passed locally and fails on the runner
|
||||
with a permission error"
|
||||
COLLAPSES: "when touching CI config"
|
||||
See create_rule for the full argument. It informs the
|
||||
tier below rather than deciding it,
|
||||
since a project rule's tier turns on area-scope, not on whether
|
||||
the trigger can be named.
|
||||
@@ -619,7 +628,12 @@ async def update_rule(
|
||||
written, the operator's actual message — and rewrite it in that vocabulary
|
||||
if it does not. Prefer the words someone produces while the rule applies,
|
||||
including the rationalisation they would be drafting to talk themselves out
|
||||
of it. See create_rule for the full argument and the measurement behind it.
|
||||
of it — that rationalisation is often the only text in existence at the
|
||||
moment the rule should fire:
|
||||
RETRIEVES: "catching yourself drafting 'this is small enough to not
|
||||
count' about a rule you have already read"
|
||||
COLLAPSES: "when the next action would conflict with a standing rule"
|
||||
See create_rule for the full argument and the measurement behind it.
|
||||
|
||||
TO EMPTY A FIELD, NAME IT: clear_fields=["verify_with"]. Passing "" cannot
|
||||
do it — "" means "leave this alone" here, which is what lets you update
|
||||
@@ -716,13 +730,15 @@ async def create_preference(
|
||||
session would actually be producing then: the command it is about to run,
|
||||
the code it is writing, the thing the operator just asked for.
|
||||
|
||||
Show the moment rather than classifying it. Write "the operator pasted a stack
|
||||
trace and said it is still broken" rather than "during hard debugging";
|
||||
write "about to `git push` with an earlier CI run still unread" rather
|
||||
than "when pacing actions". The second spelling of each is a category, and
|
||||
no session ever produces a category — it produces the command, the error
|
||||
text, the half-formed ask. A trigger naming the abstraction collapses the
|
||||
record toward its title and it never arrives.
|
||||
Show the moment rather than classifying it:
|
||||
RETRIEVES: "the operator pasted a stack trace and said it is still
|
||||
broken"
|
||||
COLLAPSES: "during hard debugging"
|
||||
The second is a category, and no session ever produces a category — it
|
||||
produces the command, the error text, the half-formed ask. A trigger
|
||||
naming the abstraction collapses the record toward its title and it
|
||||
never arrives. Both of those describe the same preference; only one of
|
||||
them can be found at the moment it applies.
|
||||
|
||||
`arose_from_id` IS REQUIRED for the same kind of reason. A preference is
|
||||
expected to change as the work teaches it, and a corpus that drifts with
|
||||
@@ -826,7 +842,11 @@ async def update_preference(
|
||||
operator's message that should have summoned it. If it describes a
|
||||
situation rather than quoting the moment, rewrite it in the words they
|
||||
actually type — and in the words YOU would be producing while about to get
|
||||
this wrong. update_rule carries the full argument.
|
||||
this wrong:
|
||||
RETRIEVES: "the operator said 'clean this up' or 'make it work like',
|
||||
naming an outcome rather than a change"
|
||||
COLLAPSES: "when reading any request from the operator"
|
||||
update_rule carries the full argument.
|
||||
|
||||
Empty strings leave fields unchanged; clear_fields empties them by name,
|
||||
exactly as update_rule does.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Every rule-write surface documents the trigger it can write (#3855).
|
||||
"""Every rule-write surface SHOWS what a trigger looks like (#3855).
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
@@ -19,10 +19,45 @@ trigger that already reads fine as English is the one nobody rewrites.
|
||||
|
||||
WHAT THIS PINS
|
||||
|
||||
One structural property, no wording: a tool that accepts `when_to_apply`
|
||||
mentions it in its docstring. That is exactly what `update_preference`
|
||||
failed before #3855 — the parameter existed and the docstring never said so,
|
||||
which is how six triggers got written against no contract at all.
|
||||
Two properties, neither of them wording:
|
||||
|
||||
1. A tool accepting `when_to_apply` mentions it. Exactly what
|
||||
`update_preference` failed before #3855 — the parameter existed and
|
||||
the docstring never said so.
|
||||
2. It carries a worked CONTRAST: one `RETRIEVES:` example and one
|
||||
`COLLAPSES:` example, quoted, and different from each other.
|
||||
|
||||
WHY A CONVENTION RATHER THAN A PROSE HEURISTIC
|
||||
|
||||
The second property guards the softer regression — guidance kept but
|
||||
abstracted back to "name the moment in session vocabulary", which is advice
|
||||
about being concrete that is not itself concrete, and is the shape that was
|
||||
already on file while the corpus filled with categories.
|
||||
|
||||
Two attempts to detect that in free prose were written and discarded, and
|
||||
the reason is worth keeping because it generalises:
|
||||
|
||||
- Counting quoted multi-word phrases anywhere in the docstring measured
|
||||
ambient quotation, not demonstrated triggers. It PASSED the abstracted
|
||||
version by scoring unrelated prose, and text with an odd number of
|
||||
quote characters produced matches spanning the gap BETWEEN two
|
||||
unrelated phrases.
|
||||
- Scoping that count to a window after each "trigger"/"when_to_apply"
|
||||
mention then FAILED `create_preference` in its correct state, because
|
||||
its examples sit further from the first mention than any defensible
|
||||
window reaches.
|
||||
|
||||
Both were proxies trying to infer demonstration from prose. The fix was to
|
||||
stop inferring: a two-line labelled contrast is unambiguous to parse, free
|
||||
in its wording, and a better teaching form than the sentences it replaced —
|
||||
`RETRIEVES:`/`COLLAPSES:` names the mechanism, so the label does work for
|
||||
the reader instead of only for the test. Where a property cannot be
|
||||
measured, changing the shape of the thing is cheaper than a cleverer
|
||||
measurement.
|
||||
|
||||
What it still cannot do is judge whether a GOOD example is good. It catches
|
||||
what actually happened: a write surface shipping with the contrast absent,
|
||||
half-present, or tidied back into abstract advice.
|
||||
|
||||
THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
|
||||
`rulebooks.register()` actually hands the server, filtered to signatures
|
||||
@@ -30,34 +65,9 @@ taking `when_to_apply`, so a rule-write tool added later is in scope on the
|
||||
day it is added. A hand-kept list has to be remembered by the person least
|
||||
likely to know it exists — the argument `RANKED_SOURCES` makes in
|
||||
services/rule_usage.py, and the reason #3855 happened at all.
|
||||
|
||||
WHAT THIS DELIBERATELY DOES NOT PIN, AND WHY IT IS NOT HERE
|
||||
|
||||
The regression worth catching is the one above's *softer* twin: guidance
|
||||
kept but abstracted back to "name the moment in session vocabulary", which
|
||||
is advice about being concrete that is not itself concrete — the shape that
|
||||
was already on file while the corpus filled with categories.
|
||||
|
||||
Two predicates for it were written and both were discarded, because each was
|
||||
falsified and each failed:
|
||||
|
||||
- Counting quoted multi-word phrases anywhere in the docstring measured
|
||||
ambient quotation, not demonstrated triggers. It passed the broken
|
||||
version by scoring unrelated prose ("what happens if someone doesn't do
|
||||
this"), and prose with an odd number of quote characters produced
|
||||
matches spanning the gap BETWEEN two unrelated phrases.
|
||||
- Scoping that count to a window after each "trigger"/"when_to_apply"
|
||||
mention then failed `create_preference` in its CORRECT state, because
|
||||
its examples sit further from the first mention than any defensible
|
||||
window reaches.
|
||||
|
||||
Rule 167's standard settles it: a check that passes on the broken code reads
|
||||
as coverage and stops anyone looking again, so no check is the better of the
|
||||
two. Whether a docstring teaches the shape WELL stays a reading judgment,
|
||||
and the record of that decision lives here so the next author does not spend
|
||||
the same two rounds discovering it.
|
||||
"""
|
||||
import inspect
|
||||
import re
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -65,6 +75,11 @@ from tests.helpers import tool_doc as _doc
|
||||
|
||||
_MODULE = "scribe.mcp.tools.rulebooks"
|
||||
|
||||
# The worked contrast. Quoted so the example's own punctuation stays inside
|
||||
# it, labelled so no proximity guess is needed to find it.
|
||||
_RETRIEVES = re.compile(r'RETRIEVES:\s*"([^"]+)"')
|
||||
_COLLAPSES = re.compile(r'COLLAPSES:\s*"([^"]+)"')
|
||||
|
||||
|
||||
def _registered_tools() -> list:
|
||||
"""Every function `rulebooks.register()` actually hands to the server.
|
||||
@@ -111,6 +126,31 @@ def test_a_trigger_writing_surface_documents_the_field(name):
|
||||
"docstring. The tool docstring is the agent-facing contract "
|
||||
"(rule 119), and a trigger written against no contract is the #3855 "
|
||||
"defect: six preferences named a category instead of a moment, and "
|
||||
"a category is not a thing any session ever types. Say what the "
|
||||
"field is for, and show a moment — see create_rule."
|
||||
"a category is not a thing any session ever types."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", _trigger_writers())
|
||||
def test_a_trigger_writing_surface_shows_the_contrast(name):
|
||||
"""Advice about being concrete has to be concrete itself."""
|
||||
doc = _doc(_MODULE, name)
|
||||
retrieves, collapses = _RETRIEVES.findall(doc), _COLLAPSES.findall(doc)
|
||||
|
||||
assert retrieves and collapses, (
|
||||
f"{name} is missing its worked contrast — found "
|
||||
f"{len(retrieves)} RETRIEVES and {len(collapses)} COLLAPSES example(s), "
|
||||
"and one of each is required. Telling an author to 'name the moment "
|
||||
"in session vocabulary' without showing one is the guidance that was "
|
||||
"already on file while the corpus filled with categories. Add two "
|
||||
'quoted lines: RETRIEVES: "<a moment a session actually produces — a '
|
||||
'command, an error, a half-formed ask>" and COLLAPSES: "<the same '
|
||||
'thing named as a category>".'
|
||||
)
|
||||
|
||||
same = set(retrieves) & set(collapses)
|
||||
assert not same, (
|
||||
f"{name} shows the same text as both RETRIEVES and COLLAPSES "
|
||||
f"({next(iter(same))!r}). The pair teaches by DIFFERING — one moment "
|
||||
"written the way a session would produce it, and the same moment "
|
||||
"written as a category. Identical halves demonstrate nothing."
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user