docs(6209): rulebooks, pull telemetry, lesson usage, duplicate arm, cited status, trigger shape
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 13s

Six more test files state their pinned properties without the issue and
milestone history: the pull-telemetry guard explains enumerating kinds
instead of listing two getter incidents, cited-record status explains why
a hint's id needs its status instead of recounting one misreported step,
and the trigger-shape guard keeps its reasoning about why prose heuristics
fail while dropping the dated corpus audit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-21 21:28:35 -04:00
co-authored by Claude Opus 5.5
parent ff08500128
commit 17e1931c0b
6 changed files with 91 additions and 112 deletions
+11 -13
View File
@@ -1,14 +1,12 @@
"""A record you merely CITE carries its status (#4154, milestone 409 step 8).
"""A record you merely CITE carries its status.
WHY THIS EXISTS
Step 1 (#4010) made placement cheap for a task whose status CHANGES: the write
returns where it sits, and the report is written from that. It does nothing for
a task a reply only mentions. In this milestone's own step-6 review the session
reported "#4014 is the open step of milestone 409". #4014 had been done for
four days; the open step was #4015. The id did not come from a read — it came
from a retrieval hint, which carries an id, a kind and a title and says nothing
about status, while `list_milestones` said "8 of 9" and would not say which one.
`placement` is cheap for a task whose status CHANGES: the write returns where
it sits, and the report is written from that. It does nothing for a task a
reply only mentions. An id taken from a retrieval hint carries an id, a kind
and a title and says nothing about status, and a count like "8 of 9" does not
say which one is open — so a finished step can be reported as the open one.
Two surfaces, one principle: the status arrives with the id.
@@ -23,7 +21,7 @@ THE ONE THAT MATTERS MOST
answer "what is next" and they must not drift. It is written as a behavioural
cross-check rather than `assert milestones.OPEN_STEP_STATUSES is placement._OPEN`,
which shares one object today and would therefore pass no matter what either
side did with it (rule 167: a guard has to be able to fail). This one fails if
side did with it — a guard has to be able to fail. This one fails if
either side changes its ordering or its notion of "open" alone.
"""
from types import SimpleNamespace
@@ -164,7 +162,7 @@ async def test_the_earliest_open_step_wins_not_the_earliest_step():
@pytest.mark.asyncio
async def test_the_batch_does_not_fan_out_per_milestone():
"""#2384's shape must not come back through the new query.
"""The step query must not fan out per milestone.
Three queries for one milestone and three for forty — the cost is in the
number of QUERIES, not the number of plans. A per-milestone "what's next"
@@ -182,7 +180,7 @@ async def test_the_batch_does_not_fan_out_per_milestone():
@pytest.mark.asyncio
async def test_both_step_queries_take_the_same_visibility_clause():
"""Rule 78, and a correctness point on top of it.
"""Access control, and a correctness point on top of it.
`next_step` names a step; the counts beside it say how many there are. Read
through different clauses, a row could name a step its own progress numbers
@@ -265,7 +263,7 @@ def test_a_record_with_no_status_gains_no_parenthesis():
def test_the_listing_tools_say_what_next_step_is_and_how_to_use_it():
"""The contract every MCP client reads (rule 119, decision #4027).
"""The contract every MCP client reads.
The field is only worth adding if a caller knows it is there. Without this
the docstring can be tidied to a parameter list and the one surface that
@@ -290,7 +288,7 @@ def test_the_reply_skill_says_a_cited_record_still_gets_read():
Presence, not absence — the skill legitimately discusses recall in order to
warn against it, so an absence check here would be satisfied by the warning
itself (snippet #3352).
itself.
"""
import pathlib
+19 -21
View File
@@ -1,9 +1,9 @@
"""The by-name duplicate arm confirms its grep hits against the real extractor (#4227).
"""The by-name duplicate arm confirms its grep hits against the real extractor.
#4222 fixed one end of this defect: the extractors that decide what a payload
DEFINES now blank comment and string spans before any line matcher runs. This
is the other end — `scribe_local_dups`, which decides which OTHER files already
define that name — and it was a plain `git grep`.
The extractors that decide what a payload DEFINES blank comment and string
spans before any line matcher runs. This is the other end — `scribe_local_dups`,
which decides which OTHER files already define that name — and a plain
`git grep` cannot do that.
A grep sees lines, not spans. The line
@@ -13,18 +13,16 @@ is prose from a module docstring in this repo, and it matches the arm's pattern
for `name=with`. So the arm would tell a writer their genuine symbol was
already defined, and point at a docstring.
WHAT THE FIX COSTS, measured rather than guessed, over 141 payload files of
this repo: the arm removed 15 of 210 report lines, and every one was a string
literal, a comment, an import statement, or Vue's `const emit = defineEmits()`
boilerplate. No real definition was lost — where a name had both a real
definition and a phantom, the phantom was dropped and the definition kept.
Timing: mean 193ms → 222ms, worst 569ms → 572ms. The arm was already dominated
by its twelve `git grep` calls, so confirming costs about 15% rather than the
second a per-(name, file) shape would have cost.
WHAT CONFIRMATION COSTS, measured rather than guessed: it drops string
literals, comments, import statements and Vue's `const emit = defineEmits()`
boilerplate, and no real definition — where a name has both a real definition
and a phantom, the phantom goes and the definition stays. The arm is
dominated by its `git grep` calls, so confirming adds about 15% (a mean of
~30ms) rather than the second a per-(name, file) shape would cost.
These cases are written against real git repositories rather than mocks,
because what is being pinned is the interaction between `git grep`, `head` and
the extractor — which is where #4042 lived too.
the extractor — which is also where a `pipefail` + `head` interaction lives.
"""
from __future__ import annotations
@@ -77,7 +75,7 @@ def repo(tmp_path):
def test_a_sentence_about_a_definition_is_not_a_definition(repo):
"""The verify case from #4227: one real `def change`, one docstring that
"""One real `def change`, one docstring that
wraps onto a line beginning `function change`. The writer is told about
the first file and not the second."""
build, run = repo
@@ -162,15 +160,14 @@ def test_an_import_is_not_a_definition(repo):
assert "card.vue" not in out, "an import site is not a definition site"
# ── the caps, which is where #4042 lived ─────────────────────────────────────
# ── the caps ─────────────────────────────────────────────────────────────────
def test_the_candidate_cap_is_raised_above_the_display_cap():
"""Confirmation REMOVES hits, so a cap applied before it runs lets phantom
matches crowd real definitions out of the window — hits dropped before
anyone looked at them, which is #4042's bug in a new place. Asserted on
structure (rule 167) because the ordering is what matters, not the
numbers."""
anyone looked at them. Asserted on structure because the ordering is what
matters, not the numbers."""
src = DEFS.read_text()
for const in ("_SCRIBE_DUP_CANDIDATES=", "_SCRIBE_DUP_SHOWN="):
assert const in src, f"{const.rstrip(chr(61))} is gone — the two caps were folded back together"
@@ -196,7 +193,7 @@ def test_phantoms_ahead_of_the_display_cap_do_not_hide_a_real_definition(repo):
def test_the_early_exiting_head_still_keeps_its_own_output(repo):
"""#4042, re-run through the new shape: under `pipefail` a `head` that
"""Under `pipefail` a `head` that
exits while git grep is still writing must not void the hits it printed.
Every file here holds a REAL definition, so confirmation keeps them all
and only the display cap applies."""
@@ -211,7 +208,8 @@ def test_the_early_exiting_head_still_keeps_its_own_output(repo):
def test_a_selector_named_inside_a_css_comment_is_not_a_rule(repo):
"""#2990's defect, on the lookup side rather than the extraction side."""
"""A selector inside a CSS comment, on the lookup side rather than the
extraction side."""
build, run = repo
build({
"real.css": ".badge { color: red; }\n",
+17 -21
View File
@@ -1,21 +1,18 @@
"""A lesson's surfaced-vs-opened counts reach a reader (#4196).
"""A lesson's surfaced-vs-opened counts reach a reader.
The lesson slot has recorded usage since it shipped, and until this nothing
showed it. `get_lesson`'s REST door attached it; the list did not, neither MCP
door did, and no view rendered it — so the one kind whose central question is
"is this trigger firing on the right situation" was the one kind whose answer
was unreadable.
Every door — the REST list and detail, both MCP doors, and the view — carries
the counts, because the one kind whose central question is "is this trigger
firing on the right situation" needs that answer readable.
WHY THE COUNTS MATTER MORE HERE THAN ON A SNIPPET. #4196 asks whether a lesson
repeatedly followed should become a rule, and names the trap: raw frequency
WHY THE COUNTS MATTER MORE HERE THAN ON A SNIPPET. Whether a lesson repeatedly
followed should become a rule has a trap: raw frequency
cannot tell "this should bind" from "this trigger is too broad", which is the
commoner reading. Surfaced-AND-opened can. Neither question is answerable by
anything that cannot see the numbers, which is why this is the first step and
not the threshold.
NO THRESHOLD IS ASSERTED ANYWHERE HERE, deliberately. At the time of writing
the corpus is 10 lessons with 7 recorded surfacings and 3 opens; a promotion
rule fitted to that would be fitting noise, and `UsageBadge` already declines
NO THRESHOLD IS ASSERTED ANYWHERE HERE, deliberately. A promotion rule fitted
to a small corpus would be fitting noise, and `UsageBadge` already declines
to render a verdict on fewer than three surfacings for the same reason.
"""
import inspect
@@ -115,11 +112,10 @@ async def test_get_lesson_reads_the_count_before_recording_its_own_pull():
assert out["usage"]["pull_count"] == 1
# ── the REST door, on structure (rule 167) ───────────────────────────────────
# ── the REST door, on structure ──────────────────────────────────────────────
#
# MOVED 2026-09-21 (#4230), not weakened. These used to look for
# `usage_for_notes(` and `empty_usage()` in the route body. Both now live in
# ONE seam, `note_usage.attach_usage`, which seven doors share — so the
# `usage_for_notes(` and `empty_usage()` live in ONE seam,
# `note_usage.attach_usage`, which seven doors share — so the
# zero-fill and the single aggregate are pinned once, against the seam, in
# tests/test_usage_attach_seam.py. What stays HERE is what only this route can
# get wrong: that it calls the seam at all, once, and before it records a pull.
@@ -166,7 +162,7 @@ def test_the_rest_detail_door_also_reads_before_it_records():
def test_these_guards_can_fail():
"""Rule 167: falsify the shape they assert against, so a guard that has
"""Falsify the shape they assert against, so a guard that has
quietly stopped describing anything cannot pass by describing nothing."""
src = _route_source("list_lessons_route")
assert "def list_lessons_route" in src
@@ -174,21 +170,21 @@ def test_these_guards_can_fail():
def test_the_detail_view_renders_the_badge_rather_than_respelling_it():
"""Snippet #3460: reach for UsageBadge and pass the kind's own advice; a
view's scoped re-spelling of `.usage-tag` is what that component replaced.
"""Reach for UsageBadge and pass the kind's own advice; a view's scoped
re-spelling of `.usage-tag` is what that component exists to prevent.
The advice is the kind-specific half. For a lesson it points at the
trigger, not at deletion — a lesson nobody opens is usually keyed to a
situation nobody is in, which is `update_lesson`'s own words and the
reading #4196 calls the commoner one.
commoner reading.
"""
view = (Path(__file__).resolve().parents[1]
/ "frontend/src/views/LessonDetailView.vue").read_text()
assert "UsageBadge" in view
assert "usage-tag" not in view, "re-spelled the chip instead of reusing it"
# The sentence moved out of this template into the shared table (#4230,
# recorded on #3460). Assert BOTH halves, so the guard cannot pass by the
# The sentence lives in the shared table, not this template. Assert BOTH
# halves, so the guard cannot pass by the
# view pointing at an entry that no longer says the right thing.
assert "DEAD_WEIGHT_ADVICE.lesson" in view, "the view no longer reads the lesson advice"
table = (Path(__file__).resolve().parents[1]
+14 -21
View File
@@ -11,19 +11,12 @@ surfaced:pulled ratio is what makes dead weight visible and prunable. A getter
that opens a record without recording it leaves that kind permanently at zero
pulls, so it looks like dead weight beside kinds that merely had a counter.
That has now happened twice:
#2245 `get_task` recorded nothing while auto-inject surfaced mostly tasks.
Fixed by adding the call to notes, tasks and snippets.
#2476 `get_process` recorded nothing — and the auto-inject menu header names
`get_process` as the way to open that kind. Processes were embedded
when #2245 was fixed; the fix enumerated the kinds someone thought of
rather than the kinds that exist. The same issue found the REST half:
`rest_snippet` was recorded, note and task detail were not, and the
model's own comment named a `'rest_note'` source nothing wrote.
A fix that enumerates the kinds someone thought of, rather than the kinds that
exist, leaves the next kind's getter silent — and the REST detail views are a
second half that is easy to forget.
A missing call is the shape no behavioural test catches: it changes no return
value (#2278, shape 4). Source inspection is the only thing that sees it.
value. Source inspection is the only thing that sees it.
WHAT MAKES THIS DERIVED RATHER THAN A LIST
@@ -203,14 +196,14 @@ def test_every_named_loader_still_exists():
)
# ── the reading project travels with the event (#4196, #3735) ────────────────
# ── the reading project travels with the event ───────────────────────────────
#
# The same sibling-drift shape as above, one dimension over. `RetrievalLog` has
# always carried `project_id`, so "surfaced on project B" was answerable; the
# usage table had none, so "opened on project B" was not, and the two cannot be
# joined (no session identity server-side — NoteUsageEvent's own docstring).
# That gap sat exactly on milestone 385's acceptance: a lesson's claim is that
# it reaches a session on a project it was NOT written on.
# The same sibling-drift shape as above, one dimension over. `RetrievalLog`
# carries `project_id`, so "surfaced on project B" is answerable; without it on
# the usage row, "opened on project B" is not, and the two cannot be joined (no
# session identity server-side — NoteUsageEvent's own docstring). That is
# exactly a lesson's claim: it reaches a session on a project it was NOT
# written on.
#
# A getter that takes the project and forgets to pass it fails the same way a
# missing record_pulled did: silently, changing no return value, and showing up
@@ -246,7 +239,7 @@ def _takes_reading_project(body: str) -> bool:
def test_every_getter_that_pulls_takes_the_reading_project():
"""Asserted on structure (rule 167): a behavioural test cannot see a
"""Asserted on structure: a behavioural test cannot see a
parameter that was never threaded through."""
missing = [
f"{module}.{name}"
@@ -277,7 +270,7 @@ def test_every_getter_that_pulls_passes_the_project_through():
# The arms are the other half: unlike a getter, a surfacing arm ALWAYS knows
# the project — it is the scope it just searched — so there is no excuse for a
# null there, and `lesson_slot` is the one milestone 385 is measured through.
# null there, and `lesson_slot` is the one a lesson's transfer is measured through.
#
# `process_skill_sync` is exempt and named here rather than pattern-matched: it
# installs every Process the operator can reach, which is not a project-scoped
@@ -310,7 +303,7 @@ def test_every_project_scoped_surfacing_arm_records_its_project():
def test_the_reading_project_guards_can_fail():
"""Rule 167: shown turning red once, so a guard that has quietly stopped
"""Shown turning red once, so a guard that has quietly stopped
matching anything is distinguishable from one that passes."""
assert list(_pulling_getters()), "the pulling-getter detection matches nothing"
fake = "async def get_thing(thing_id: int) -> dict:\n record_pulled(x)"
+9 -9
View File
@@ -12,7 +12,7 @@ from tests.helpers import fake_rule, fake_rulebook, fake_topic, make_mock_sessio
@pytest.fixture(autouse=True)
def _no_exclusions():
"""get_applicable_rules asks for the project's always-on exclusions
(milestone 297) through its own session; these mocked-session tests
through its own session; these mocked-session tests
script the rule queries only, so the exclusions lookup is stubbed empty."""
if True:
yield
@@ -212,12 +212,12 @@ def _areas(*canonical_ids):
@pytest.mark.asyncio
async def test_get_applicable_rules_returns_shape():
"""The listing is a project's own rules plus the global rules tagged to its
areas (milestone 414) — no subscriptions or suppressions in the shape."""
areas — no subscriptions or suppressions in the shape."""
mock_session = make_mock_session()
rules_result = MagicMock()
rules_result.all.return_value = [
# The query selects the ENTITY plus three labels, so rule_brief stays
# the one place deciding what a surfaced rule carries (note 3026).
# the one place deciding what a surfaced rule carries.
(fake_rule(id=i, title=f"Rule {i}", statement=f"Statement {i}", topic_id=2),
"git-workflow", 1, "FabledSword family")
for i in range(50)
@@ -330,7 +330,7 @@ async def test_a_co_surfaces_partner_on_another_project_is_not_dragged_in():
assert [r["id"] for r in result["rules"]] == [200, 201]
# ── rule_brief (milestone 307) ──────────────────────────────────────────
# ── rule_brief ──────────────────────────────────────────────────────────
def test_rule_brief_carries_age_but_not_the_deep_fields():
"""The shape a SURFACED rule takes, and the reason it exists.
@@ -338,7 +338,7 @@ def test_rule_brief_carries_age_but_not_the_deep_fields():
There were three hand-written copies of this dict and they had already
diverged — none carried the timestamps the model has always held, which is
why a rule written before the capability it duplicates was
indistinguishable at read time from one still doing work (note 3026).
indistinguishable at read time from one still doing work.
"""
from scribe.services.rulebooks import rule_brief
@@ -356,7 +356,7 @@ def test_rule_brief_carries_age_but_not_the_deep_fields():
def test_rule_brief_omits_keys_a_rule_has_no_value_for():
"""#2483: a null key reads as a capability the record has and isn't using,
"""A null key reads as a capability the record has and isn't using,
which is a different claim from not having one."""
from scribe.services.rulebooks import rule_brief
@@ -367,7 +367,7 @@ def test_rule_brief_omits_keys_a_rule_has_no_value_for():
# ── verify_with / expires_when (milestone 312) ──────────────────────────
# ── verify_with / expires_when ──────────────────────────────────────────
def test_a_rule_with_no_check_says_nothing_about_verification():
"""The empty case is the COMMON case, and it must stay silent.
@@ -385,7 +385,7 @@ def test_a_rule_with_no_check_says_nothing_about_verification():
def test_an_unverified_constraint_reads_never_rather_than_null():
"""#2483 again: a null key reads as a capability going unused. "never" is
"""A null key reads as a capability going unused. "never" is
a different and much stronger claim — this rule asserts a fact about
someone else's software and nobody has ever confirmed it."""
from scribe.services.rulebooks import last_verified_label, rule_brief
@@ -421,7 +421,7 @@ def test_the_check_text_itself_never_enters_a_listing():
assert "expires_when" not in out
# ── the sweep's row shape (milestone 312 step 3) ────────────────────────
# ── the sweep's row shape ───────────────────────────────────────────────
def test_a_sweep_row_carries_the_check_in_full():
"""The OPPOSITE call from rule_brief, and deliberately so.
+21 -27
View File
@@ -1,19 +1,17 @@
"""Every rule-write surface SHOWS what a trigger looks like (#3855).
"""Every rule-write surface SHOWS what a trigger looks like.
WHY THIS EXISTS
`when_to_apply` is not documentation. `rule_document()` uses it twice — as
the embedded title's second half and again above the body — so it dominates
the vector, and a rule whose trigger names a CATEGORY rather than a moment
collapses toward its title and never arrives. Measured in #3835 across 113
rules, then again in #3855 on eight preferences.
collapses toward its title and never arrives.
The corpus damage traced to an uneven contract rather than to careless
authors. `create_rule` had carried the full argument since 2026-08-27, and
the two preferences written that day got good triggers; the six written
before it — "during hard debugging", "when reading any request from the
operator" — got categories. The guidance worked wherever it existed. It
simply did not exist on either `update_*` surface, which is where every
Bad triggers come from an uneven contract rather than from careless authors.
Triggers written where the guidance exists name a moment; triggers written
without it — "during hard debugging", "when reading any request from the
operator" — name categories. The guidance has to exist on the `update_*`
surfaces too, which is where every
RETROFITTED trigger is written, and retrofitting is most of the work: a
trigger that already reads fine as English is the one nobody rewrites.
@@ -21,9 +19,8 @@ WHAT THIS PINS
Two properties, neither of them wording:
1. A tool accepting `when_to_apply` mentions it. Exactly what
`update_preference` failed before #3855 — the parameter existed and
the docstring never said so.
1. A tool accepting `when_to_apply` mentions it — a parameter the
docstring never names is one nobody fills well.
2. It carries a worked CONTRAST: one `RETRIEVES:` example and one
`COLLAPSES:` example, quoted, and different from each other.
@@ -31,23 +28,21 @@ WHY A CONVENTION RATHER THAN A PROSE HEURISTIC
The second property guards the softer regression — guidance kept but
abstracted back to "name the moment in session vocabulary", which is advice
about being concrete that is not itself concrete, and is the shape that was
already on file while the corpus filled with categories.
about being concrete that is not itself concrete.
Two attempts to detect that in free prose were written and discarded, and
the reason is worth keeping because it generalises:
Two ways to detect that in free prose fail, and the reason generalises:
- Counting quoted multi-word phrases anywhere in the docstring measured
ambient quotation, not demonstrated triggers. It PASSED the abstracted
ambient quotation, not demonstrated triggers. It PASSES the abstracted
version by scoring unrelated prose, and text with an odd number of
quote characters produced matches spanning the gap BETWEEN two
unrelated phrases.
- Scoping that count to a window after each "trigger"/"when_to_apply"
mention then FAILED `create_preference` in its correct state, because
mention FAILS `create_preference` in its correct state, because
its examples sit further from the first mention than any defensible
window reaches.
Both were proxies trying to infer demonstration from prose. The fix was to
Both are proxies trying to infer demonstration from prose. The answer is to
stop inferring: a two-line labelled contrast is unambiguous to parse, free
in its wording, and a better teaching form than the sentences it replaced —
`RETRIEVES:`/`COLLAPSES:` names the mechanism, so the label does work for
@@ -56,7 +51,7 @@ measured, changing the shape of the thing is cheaper than a cleverer
measurement.
What it still cannot do is judge whether a GOOD example is good. It catches
what actually happened: a write surface shipping with the contrast absent,
the likely failure: a write surface shipping with the contrast absent,
half-present, or tidied back into abstract advice.
THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
@@ -64,7 +59,7 @@ THE SURFACE LIST IS DERIVED, NEVER HAND-KEPT. It comes from what
taking `when_to_apply`, so a rule-write tool added later is in scope on the
day it is added. A hand-kept list has to be remembered by the person least
likely to know it exists — the argument `RANKED_SOURCES` makes in
services/rule_usage.py, and the reason #3855 happened at all.
services/rule_usage.py.
"""
import inspect
import re
@@ -111,8 +106,8 @@ def _trigger_writers() -> list[str]:
if "when_to_apply" in inspect.signature(fn).parameters
]
# An empty list would make every case below vacuous and the guard would
# pass by having nothing to check — absence read as non-existence, the
# #3720 shape. Fail loudly instead.
# pass by having nothing to check — absence read as non-existence. Fail
# loudly instead.
assert names, "no rule-write surface takes when_to_apply; the guard is blind"
return sorted(names)
@@ -123,10 +118,9 @@ def test_a_trigger_writing_surface_documents_the_field(name):
doc = _doc(_MODULE, name)
assert "when_to_apply" in doc, (
f"{name} accepts when_to_apply but never mentions it in its "
"docstring. The tool docstring is the agent-facing contract "
"(rule 119), and a trigger written against no contract is the #3855 "
"defect: six preferences named a category instead of a moment, and "
"a category is not a thing any session ever types."
"docstring. The tool docstring is the agent-facing contract, and a "
"trigger written against no contract names a category instead of a "
"moment — and a category is not a thing any session ever types."
)