Retire the always-on tier — every rule arrives by retrieval (milestone 394) #152
@@ -420,6 +420,14 @@ async def create_rule(
|
|||||||
brought it back as the top hit. Where a rule prevents a specific
|
brought it back as the top hit. Where a rule prevents a specific
|
||||||
failure, put that failure's vocabulary here — the error text,
|
failure, put that failure's vocabulary here — the error text,
|
||||||
the wrong behaviour, the dead end.
|
the wrong behaviour, the dead end.
|
||||||
|
The two spellings, side by side:
|
||||||
|
RETRIEVES: "the migration failed with a check violation on a
|
||||||
|
column we just extended"
|
||||||
|
COLLAPSES: "when working on migrations"
|
||||||
|
The second names a CATEGORY. No session ever produces a
|
||||||
|
category — it produces the command, the error, the half-formed
|
||||||
|
ask — so a trigger written that way leaves the embedded
|
||||||
|
document to be carried by the title alone.
|
||||||
tier: "always_on" (default) or "conditional".
|
tier: "always_on" (default) or "conditional".
|
||||||
The test: can you name the trigger WITHOUT naming a system, an
|
The test: can you name the trigger WITHOUT naming a system, an
|
||||||
artifact type or a moment? If the honest answer is "whenever you
|
artifact type or a moment? If the honest answer is "whenever you
|
||||||
@@ -519,10 +527,11 @@ async def create_project_rule(
|
|||||||
when_to_apply: WHEN this rule fires — the trigger, not the
|
when_to_apply: WHEN this rule fires — the trigger, not the
|
||||||
instruction, and the rule's retrieval surface: name the SYMPTOM,
|
instruction, and the rule's retrieval surface: name the SYMPTOM,
|
||||||
the words someone would type while stuck. Show the moment rather
|
the words someone would type while stuck. Show the moment rather
|
||||||
than classifying it — "the migration failed with a check
|
than classifying it:
|
||||||
violation on a column we just extended" retrieves; "when working
|
RETRIEVES: "the CI job passed locally and fails on the runner
|
||||||
on migrations" does not, because it names a category no session
|
with a permission error"
|
||||||
ever types. See create_rule for the full argument. It informs the
|
COLLAPSES: "when touching CI config"
|
||||||
|
See create_rule for the full argument. It informs the
|
||||||
tier below rather than deciding it,
|
tier below rather than deciding it,
|
||||||
since a project rule's tier turns on area-scope, not on whether
|
since a project rule's tier turns on area-scope, not on whether
|
||||||
the trigger can be named.
|
the trigger can be named.
|
||||||
@@ -619,7 +628,12 @@ async def update_rule(
|
|||||||
written, the operator's actual message — and rewrite it in that vocabulary
|
written, the operator's actual message — and rewrite it in that vocabulary
|
||||||
if it does not. Prefer the words someone produces while the rule applies,
|
if it does not. Prefer the words someone produces while the rule applies,
|
||||||
including the rationalisation they would be drafting to talk themselves out
|
including the rationalisation they would be drafting to talk themselves out
|
||||||
of it. See create_rule for the full argument and the measurement behind it.
|
of it — that rationalisation is often the only text in existence at the
|
||||||
|
moment the rule should fire:
|
||||||
|
RETRIEVES: "catching yourself drafting 'this is small enough to not
|
||||||
|
count' about a rule you have already read"
|
||||||
|
COLLAPSES: "when the next action would conflict with a standing rule"
|
||||||
|
See create_rule for the full argument and the measurement behind it.
|
||||||
|
|
||||||
TO EMPTY A FIELD, NAME IT: clear_fields=["verify_with"]. Passing "" cannot
|
TO EMPTY A FIELD, NAME IT: clear_fields=["verify_with"]. Passing "" cannot
|
||||||
do it — "" means "leave this alone" here, which is what lets you update
|
do it — "" means "leave this alone" here, which is what lets you update
|
||||||
@@ -716,13 +730,15 @@ async def create_preference(
|
|||||||
session would actually be producing then: the command it is about to run,
|
session would actually be producing then: the command it is about to run,
|
||||||
the code it is writing, the thing the operator just asked for.
|
the code it is writing, the thing the operator just asked for.
|
||||||
|
|
||||||
Show the moment rather than classifying it. Write "the operator pasted a stack
|
Show the moment rather than classifying it:
|
||||||
trace and said it is still broken" rather than "during hard debugging";
|
RETRIEVES: "the operator pasted a stack trace and said it is still
|
||||||
write "about to `git push` with an earlier CI run still unread" rather
|
broken"
|
||||||
than "when pacing actions". The second spelling of each is a category, and
|
COLLAPSES: "during hard debugging"
|
||||||
no session ever produces a category — it produces the command, the error
|
The second is a category, and no session ever produces a category — it
|
||||||
text, the half-formed ask. A trigger naming the abstraction collapses the
|
produces the command, the error text, the half-formed ask. A trigger
|
||||||
record toward its title and it never arrives.
|
naming the abstraction collapses the record toward its title and it
|
||||||
|
never arrives. Both of those describe the same preference; only one of
|
||||||
|
them can be found at the moment it applies.
|
||||||
|
|
||||||
`arose_from_id` IS REQUIRED for the same kind of reason. A preference is
|
`arose_from_id` IS REQUIRED for the same kind of reason. A preference is
|
||||||
expected to change as the work teaches it, and a corpus that drifts with
|
expected to change as the work teaches it, and a corpus that drifts with
|
||||||
@@ -826,7 +842,11 @@ async def update_preference(
|
|||||||
operator's message that should have summoned it. If it describes a
|
operator's message that should have summoned it. If it describes a
|
||||||
situation rather than quoting the moment, rewrite it in the words they
|
situation rather than quoting the moment, rewrite it in the words they
|
||||||
actually type — and in the words YOU would be producing while about to get
|
actually type — and in the words YOU would be producing while about to get
|
||||||
this wrong. update_rule carries the full argument.
|
this wrong:
|
||||||
|
RETRIEVES: "the operator said 'clean this up' or 'make it work like',
|
||||||
|
naming an outcome rather than a change"
|
||||||
|
COLLAPSES: "when reading any request from the operator"
|
||||||
|
update_rule carries the full argument.
|
||||||
|
|
||||||
Empty strings leave fields unchanged; clear_fields empties them by name,
|
Empty strings leave fields unchanged; clear_fields empties them by name,
|
||||||
exactly as update_rule does.
|
exactly as update_rule does.
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Every rule-write surface documents the trigger it can write (#3855).
|
"""Every rule-write surface SHOWS what a trigger looks like (#3855).
|
||||||
|
|
||||||
WHY THIS EXISTS
|
WHY THIS EXISTS
|
||||||
|
|
||||||
@@ -19,10 +19,45 @@ trigger that already reads fine as English is the one nobody rewrites.
|
|||||||
|
|
||||||
WHAT THIS PINS
|
WHAT THIS PINS
|
||||||
|
|
||||||
One structural property, no wording: a tool that accepts `when_to_apply`
|
Two properties, neither of them wording:
|
||||||
mentions it in its docstring. That is exactly what `update_preference`
|
|
||||||
failed before #3855 — the parameter existed and the docstring never said so,
|
1. A tool accepting `when_to_apply` mentions it. Exactly what
|
||||||
which is how six triggers got written against no contract at all.
|
`update_preference` failed before #3855 — the parameter existed and
|
||||||
|
the docstring never said so.
|
||||||
|
2. It carries a worked CONTRAST: one `RETRIEVES:` example and one
|
||||||
|
`COLLAPSES:` example, quoted, and different from each other.
|
||||||
|
|
||||||
|
WHY A CONVENTION RATHER THAN A PROSE HEURISTIC
|
||||||
|
|
||||||
|
The second property guards the softer regression — guidance kept but
|
||||||
|
abstracted back to "name the moment in session vocabulary", which is advice
|
||||||
|
about being concrete that is not itself concrete, and is the shape that was
|
||||||
|
already on file while the corpus filled with categories.
|
||||||
|
|
||||||
|
Two attempts to detect that in free prose were written and discarded, and
|
||||||
|
the reason is worth keeping because it generalises:
|
||||||
|
|
||||||
|
- Counting quoted multi-word phrases anywhere in the docstring measured
|
||||||
|
ambient quotation, not demonstrated triggers. It PASSED the abstracted
|
||||||
|
version by scoring unrelated prose, and text with an odd number of
|
||||||
|
quote characters produced matches spanning the gap BETWEEN two
|
||||||
|
unrelated phrases.
|
||||||
|
- Scoping that count to a window after each "trigger"/"when_to_apply"
|
||||||
|
mention then FAILED `create_preference` in its correct state, because
|
||||||
|
its examples sit further from the first mention than any defensible
|
||||||
|
window reaches.
|
||||||
|
|
||||||
|
Both were proxies trying to infer demonstration from prose. The fix was to
|
||||||
|
stop inferring: a two-line labelled contrast is unambiguous to parse, free
|
||||||
|
in its wording, and a better teaching form than the sentences it replaced —
|
||||||
|
`RETRIEVES:`/`COLLAPSES:` names the mechanism, so the label does work for
|
||||||
|
the reader instead of only for the test. Where a property cannot be
|
||||||
|
measured, changing the shape of the thing is cheaper than a cleverer
|
||||||
|
measurement.
|
||||||
|
|
||||||
|
What it still cannot do is judge whether a GOOD example is good. It catches
|
||||||
|
what actually happened: a write surface shipping with the contrast absent,
|
||||||
|
half-present, or tidied back into abstract advice.
|
||||||
|
|
||||||
THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
|
THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
|
||||||
`rulebooks.register()` actually hands the server, filtered to signatures
|
`rulebooks.register()` actually hands the server, filtered to signatures
|
||||||
@@ -30,34 +65,9 @@ taking `when_to_apply`, so a rule-write tool added later is in scope on the
|
|||||||
day it is added. A hand-kept list has to be remembered by the person least
|
day it is added. A hand-kept list has to be remembered by the person least
|
||||||
likely to know it exists — the argument `RANKED_SOURCES` makes in
|
likely to know it exists — the argument `RANKED_SOURCES` makes in
|
||||||
services/rule_usage.py, and the reason #3855 happened at all.
|
services/rule_usage.py, and the reason #3855 happened at all.
|
||||||
|
|
||||||
WHAT THIS DELIBERATELY DOES NOT PIN, AND WHY IT IS NOT HERE
|
|
||||||
|
|
||||||
The regression worth catching is the one above's *softer* twin: guidance
|
|
||||||
kept but abstracted back to "name the moment in session vocabulary", which
|
|
||||||
is advice about being concrete that is not itself concrete — the shape that
|
|
||||||
was already on file while the corpus filled with categories.
|
|
||||||
|
|
||||||
Two predicates for it were written and both were discarded, because each was
|
|
||||||
falsified and each failed:
|
|
||||||
|
|
||||||
- Counting quoted multi-word phrases anywhere in the docstring measured
|
|
||||||
ambient quotation, not demonstrated triggers. It passed the broken
|
|
||||||
version by scoring unrelated prose ("what happens if someone doesn't do
|
|
||||||
this"), and prose with an odd number of quote characters produced
|
|
||||||
matches spanning the gap BETWEEN two unrelated phrases.
|
|
||||||
- Scoping that count to a window after each "trigger"/"when_to_apply"
|
|
||||||
mention then failed `create_preference` in its CORRECT state, because
|
|
||||||
its examples sit further from the first mention than any defensible
|
|
||||||
window reaches.
|
|
||||||
|
|
||||||
Rule 167's standard settles it: a check that passes on the broken code reads
|
|
||||||
as coverage and stops anyone looking again, so no check is the better of the
|
|
||||||
two. Whether a docstring teaches the shape WELL stays a reading judgment,
|
|
||||||
and the record of that decision lives here so the next author does not spend
|
|
||||||
the same two rounds discovering it.
|
|
||||||
"""
|
"""
|
||||||
import inspect
|
import inspect
|
||||||
|
import re
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
@@ -65,6 +75,11 @@ from tests.helpers import tool_doc as _doc
|
|||||||
|
|
||||||
_MODULE = "scribe.mcp.tools.rulebooks"
|
_MODULE = "scribe.mcp.tools.rulebooks"
|
||||||
|
|
||||||
|
# The worked contrast. Quoted so the example's own punctuation stays inside
|
||||||
|
# it, labelled so no proximity guess is needed to find it.
|
||||||
|
_RETRIEVES = re.compile(r'RETRIEVES:\s*"([^"]+)"')
|
||||||
|
_COLLAPSES = re.compile(r'COLLAPSES:\s*"([^"]+)"')
|
||||||
|
|
||||||
|
|
||||||
def _registered_tools() -> list:
|
def _registered_tools() -> list:
|
||||||
"""Every function `rulebooks.register()` actually hands to the server.
|
"""Every function `rulebooks.register()` actually hands to the server.
|
||||||
@@ -111,6 +126,31 @@ def test_a_trigger_writing_surface_documents_the_field(name):
|
|||||||
"docstring. The tool docstring is the agent-facing contract "
|
"docstring. The tool docstring is the agent-facing contract "
|
||||||
"(rule 119), and a trigger written against no contract is the #3855 "
|
"(rule 119), and a trigger written against no contract is the #3855 "
|
||||||
"defect: six preferences named a category instead of a moment, and "
|
"defect: six preferences named a category instead of a moment, and "
|
||||||
"a category is not a thing any session ever types. Say what the "
|
"a category is not a thing any session ever types."
|
||||||
"field is for, and show a moment — see create_rule."
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("name", _trigger_writers())
|
||||||
|
def test_a_trigger_writing_surface_shows_the_contrast(name):
|
||||||
|
"""Advice about being concrete has to be concrete itself."""
|
||||||
|
doc = _doc(_MODULE, name)
|
||||||
|
retrieves, collapses = _RETRIEVES.findall(doc), _COLLAPSES.findall(doc)
|
||||||
|
|
||||||
|
assert retrieves and collapses, (
|
||||||
|
f"{name} is missing its worked contrast — found "
|
||||||
|
f"{len(retrieves)} RETRIEVES and {len(collapses)} COLLAPSES example(s), "
|
||||||
|
"and one of each is required. Telling an author to 'name the moment "
|
||||||
|
"in session vocabulary' without showing one is the guidance that was "
|
||||||
|
"already on file while the corpus filled with categories. Add two "
|
||||||
|
'quoted lines: RETRIEVES: "<a moment a session actually produces — a '
|
||||||
|
'command, an error, a half-formed ask>" and COLLAPSES: "<the same '
|
||||||
|
'thing named as a category>".'
|
||||||
|
)
|
||||||
|
|
||||||
|
same = set(retrieves) & set(collapses)
|
||||||
|
assert not same, (
|
||||||
|
f"{name} shows the same text as both RETRIEVES and COLLAPSES "
|
||||||
|
f"({next(iter(same))!r}). The pair teaches by DIFFERING — one moment "
|
||||||
|
"written the way a session would produce it, and the same moment "
|
||||||
|
"written as a category. Identical halves demonstrate nothing."
|
||||||
)
|
)
|
||||||
|
|||||||
Reference in New Issue
Block a user