docs(6209): wide net, startup, stamps, slippage, miss route and parity tests
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 15s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Canceled after 0s
CI & Build / Python tests (push) Canceled after 1m23s
Eight more test files state the property they pin as a present-tense fact. Incident narratives (a dated storage stall, a week of zero tuning events, a two-project poisoned ledger, a twenty-site owner audit) become the failure shape each test defends; one-install figures that name a rule become a general worked example with the calibration numbers kept. The family-grouping regression guard is renamed for what it checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,21 +1,18 @@
|
||||
"""A semantic MISS is not evidence, and the divergence check never reads one (#4208).
|
||||
"""A semantic MISS is not evidence, and the divergence check never reads one.
|
||||
|
||||
WHAT THIS IS ABOUT. #4204 gave the divergence check a structural gate, which
|
||||
silenced one of the five false prompts it was filed for. The other four are
|
||||
`def` helpers beside an `async def` service canon — callables beside a
|
||||
callable — and differ from #2793's acceptance case (a hand-rolled sync
|
||||
`confirmDanger` where an async confirm helper is canon) only by the JOB they
|
||||
do. A signature does not carry a job.
|
||||
WHAT THIS IS ABOUT. The divergence check's structural gate separates a type
|
||||
from a callable canon. `def` helpers beside an `async def` service canon —
|
||||
callables beside a callable — differ from the readout's acceptance case (a
|
||||
hand-rolled sync `confirmDanger` where an async confirm helper is canon) only
|
||||
by the JOB they do. A signature does not carry a job.
|
||||
|
||||
#4208 tried meaning: store the proposer's semantic "compared, nothing cleared
|
||||
the floor" and let it withdraw the prompt. Measured live on 2026-09-22 it
|
||||
cannot discriminate. Judged instances of the service-unit canon score 0.68–
|
||||
0.71 against it at best, most below 0.66; helpers score 0.66–0.75 against
|
||||
snippets unrelated to them. The then-floor of 0.8 was reached by nothing, so
|
||||
the "miss" fired for nearly every body — the real divergence silenced exactly
|
||||
as the helper was. The gate was removed; the prompt asks and the judge
|
||||
answers. The same measurement moved the arm's floor down to the write-path
|
||||
hint's: 0.8 was sized for proposals nobody reads, and it proposed 0 of 150.
|
||||
Meaning cannot withdraw the prompt either. Judged instances of a service-unit
|
||||
canon score 0.68–0.71 against it at best, most below 0.66; helpers score
|
||||
0.66–0.75 against snippets unrelated to them. No floor separates the two, so a
|
||||
"compared, nothing cleared the floor" miss fires for nearly every body,
|
||||
silencing the real divergence exactly as it silences the helper. So the prompt
|
||||
asks and the judge answers, and the arm proposes at the write-path hint's
|
||||
floor: one sized for proposals nobody reads proposes almost nothing.
|
||||
|
||||
These tests pin what stayed: the arm proposes on a hit and says nothing on a
|
||||
miss, the divergence check does not consult it, and the capped pass reads new
|
||||
@@ -32,8 +29,8 @@ from scribe.services import shape_ledger
|
||||
from scribe.services.shape_ledger import _semantic_canon
|
||||
|
||||
# Comfortably over WRITEPATH_MIN_CODE_CHARS (48 non-whitespace characters), so
|
||||
# these tests exercise the comparison rather than the substance guard. One of
|
||||
# #4204's four survivors, quoted rather than invented.
|
||||
# these tests exercise the comparison rather than the substance guard. A real
|
||||
# helper body, quoted rather than invented.
|
||||
BODY = (
|
||||
"def is_registered(source: str) -> bool:\n"
|
||||
" return source in _REGISTRY and _REGISTRY[source].enabled\n"
|
||||
@@ -87,7 +84,7 @@ async def test_a_body_too_thin_to_embed_spends_no_embedding() -> None:
|
||||
|
||||
|
||||
async def test_no_allowed_canon_spends_no_search() -> None:
|
||||
"""The language-family gate (#2871) empties the allowed set routinely."""
|
||||
"""The language-family gate empties the allowed set routinely."""
|
||||
mock = _hits()
|
||||
with _patch(mock):
|
||||
assert await _semantic_canon(1, BODY, set()) is None
|
||||
@@ -97,7 +94,7 @@ async def test_no_allowed_canon_spends_no_search() -> None:
|
||||
async def test_the_arm_proposes_at_the_write_path_floor() -> None:
|
||||
"""It asks the write-path hint's question of the same documents, so it
|
||||
uses that surface's floor — not a stricter private one that yields no
|
||||
proposals for the judge to weigh (#4208)."""
|
||||
proposals for the judge to weigh."""
|
||||
from scribe.services.plugin_context import WRITEPATH_DEFAULT_THRESHOLD
|
||||
|
||||
mock = _hits()
|
||||
@@ -111,14 +108,14 @@ async def test_the_arm_proposes_at_the_write_path_floor() -> None:
|
||||
|
||||
def test_the_divergence_check_does_not_read_the_proposal_basis() -> None:
|
||||
"""The retired gate keyed on `proposal_basis`. Asserted on the function's
|
||||
structure (rule 167): any attribute read of it inside `flag_divergence` is
|
||||
structure: any attribute read of it inside `flag_divergence` is
|
||||
the miss being consulted again, whatever the constant is called."""
|
||||
tree = ast.parse(textwrap.dedent(inspect.getsource(shape_ledger.flag_divergence)))
|
||||
reads = [n for n in ast.walk(tree)
|
||||
if isinstance(n, ast.Attribute) and n.attr == "proposal_basis"]
|
||||
assert not reads, (
|
||||
"flag_divergence reads proposal_basis — a semantic miss is 'cannot "
|
||||
"tell' at the arm's floor (#4208), never 'not the canon'"
|
||||
"tell' at the arm's floor, never 'not the canon'"
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,15 +1,14 @@
|
||||
"""One definition of what SHIPS in the plugin, and the drift guards on it.
|
||||
|
||||
WHAT THIS IS ABOUT (#3127 §3, milestone 334 step 2). Scribe publishes two
|
||||
artifacts. `plugin/` is not in the Docker image — installs fetch it from this
|
||||
WHAT THIS IS ABOUT. Scribe publishes two artifacts. `plugin/` is not in the Docker image — installs fetch it from this
|
||||
repo through `.claude-plugin/marketplace.json`, so **a push IS the release**,
|
||||
with no build step in between. That makes "which files reach an install?" a
|
||||
question with real consequences, and it has been answered wrong twice:
|
||||
question with real consequences, and it can be answered wrong two ways:
|
||||
|
||||
- #2198 — `plugin/**` was in no `paths:` filter, so four broken hooks
|
||||
reached live installs having triggered no CI at all.
|
||||
- #2209 — the fix for that shipped and still could not reach an install,
|
||||
because the manifest version had not moved.
|
||||
- a shipped path in no `paths:` filter reaches live installs having
|
||||
triggered no CI at all;
|
||||
- a fix that ships without the manifest version moving cannot reach an
|
||||
install.
|
||||
|
||||
The set lives in `scripts/check_plugin.py`. Its second consumer is the
|
||||
workflow's `paths:` trigger, which is YAML and cannot import Python — so the
|
||||
@@ -66,7 +65,7 @@ def test_every_shipped_path_exists():
|
||||
|
||||
|
||||
def test_every_shipped_path_triggers_ci():
|
||||
"""#2198's exact hole, stated as an assertion.
|
||||
"""A shipped path that runs no CI, stated as an assertion.
|
||||
|
||||
Directional on purpose: the trigger is a superset (it also fires on
|
||||
`src/**`, `tests/**` and friends). What must never happen is a path that
|
||||
@@ -77,7 +76,7 @@ def test_every_shipped_path_triggers_ci():
|
||||
covered = any(t == path or t.startswith(f"{path}/") for t in triggers)
|
||||
assert covered, (
|
||||
f"{path} ships to installs but no `paths:` entry covers it — "
|
||||
f"changes there would reach a live install having run no CI (#2198)"
|
||||
f"changes there would reach a live install having run no CI"
|
||||
)
|
||||
|
||||
|
||||
@@ -94,10 +93,8 @@ def test_the_checker_itself_triggers_ci():
|
||||
def test_no_trigger_path_names_something_that_does_not_exist():
|
||||
"""The guard that catches scaffolding outliving its subsystem.
|
||||
|
||||
`fable-mcp/**` sat in this list for three months after the directory was
|
||||
deleted (commit 91bafb6, 2026-05-27), and `assets/**` named a path that
|
||||
never existed at all. Neither ever failed anything — a `paths:` entry
|
||||
matching nothing simply never fires — which is precisely why a list kept
|
||||
A `paths:` entry matching nothing simply never fires, so an entry for a
|
||||
deleted directory, or for a path that never existed, fails nothing — which is precisely why a list kept
|
||||
by hand drifts and nobody finds out.
|
||||
"""
|
||||
missing = [
|
||||
@@ -114,7 +111,7 @@ def test_every_deriver_exists():
|
||||
"""§3's table, kept honest.
|
||||
|
||||
The point of the table is that the next artifact is a one-line addition
|
||||
(milestone 334 step 3 adds the plugin's mint script). A row pointing at a
|
||||
(the plugin's mint script is one). A row pointing at a
|
||||
file that has moved would make the table read as complete when it is not.
|
||||
"""
|
||||
for path, artifacts in DERIVERS.items():
|
||||
@@ -162,7 +159,7 @@ def test_every_OTHER_manifest_field_still_demands_a_new_version(field, value):
|
||||
`plugin.json` carries description, mcpServers and userConfig alongside the
|
||||
version, and all of them reach an install. Excluding the file wholesale
|
||||
would mean a userConfig-only edit computes an unchanged version and never
|
||||
refreshes — #2209 again, with a narrower trigger and the same silence.
|
||||
refreshes — a fix that cannot reach an install, in silence.
|
||||
"""
|
||||
assert manifest_differs_beyond_version(manifest(**{field: value}), manifest()) is True
|
||||
|
||||
@@ -181,7 +178,7 @@ def test_reformatting_is_not_a_content_change():
|
||||
def test_unreadable_input_demands_a_new_version(bad):
|
||||
"""The conservative direction, chosen deliberately.
|
||||
|
||||
A spurious bump costs one cache refresh. A missed one is #2209 — the fix
|
||||
A spurious bump costs one cache refresh. A missed one means the fix
|
||||
reaches the repo and stops there, and the only detector is a human saying
|
||||
"I don't think it updated."
|
||||
"""
|
||||
@@ -238,9 +235,8 @@ def test_shipped_content_changed_reports_a_hook_edit_as_changed(monkeypatch):
|
||||
def test_a_failed_diff_is_None_and_never_False(monkeypatch):
|
||||
"""Could-not-tell and nothing-changed must not collapse into one value.
|
||||
|
||||
#2663 is the precedent: a read that failed inside a broad except reported
|
||||
the same zero as a genuinely empty window, and every counter read zero for
|
||||
weeks with nothing to distinguish the two.
|
||||
A read that fails inside a broad except and reports the same zero as a
|
||||
genuinely empty window leaves nothing to distinguish the two.
|
||||
"""
|
||||
from scripts import check_plugin
|
||||
|
||||
|
||||
@@ -1,18 +1,13 @@
|
||||
"""A recognized retrieval miss has a route, and the route is ordered (#4133).
|
||||
"""A recognized retrieval miss has a route, and the route is ordered.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
Milestone 416 built the tuning loop across steps 4 and 6 — `retrieval_surfaces`
|
||||
to read a dial, `tune_retrieval` to move one with its argument attached,
|
||||
`migrate_retrieval_floor` to carry it across a model change — and then measured,
|
||||
on 2026-09-17, that `retrieval_tuning_history` returned **zero events**. Not one
|
||||
call, ever.
|
||||
|
||||
The cause was not a tool being wrong. The three tools appeared on NO instruction
|
||||
surface: not the skills, not the hooks, not the MCP `_INSTRUCTIONS`. They were
|
||||
classified in `server.py`'s read/write lists and named nowhere a session would
|
||||
read them. The operator's decision for #4102 — *"the model should be the thing
|
||||
handling it 9 times out of 10"* — could not begin to happen.
|
||||
The tuning loop is three tools — `retrieval_surfaces` to read a dial,
|
||||
`tune_retrieval` to move one with its argument attached,
|
||||
`migrate_retrieval_floor` to carry it across a model change — and the model is
|
||||
meant to be the one using them most of the time. A tool classified in
|
||||
`server.py`'s read/write lists and named on no instruction surface is never
|
||||
called: nothing a session reads tells it the tool exists.
|
||||
|
||||
So this file guards a ROUTE, not a behaviour: that a session which notices a
|
||||
rule missed the moment it governed has somewhere written down to take it.
|
||||
@@ -27,13 +22,12 @@ computed against, so a miss is evidence about that text first; a floor is one
|
||||
number governing every record on the surface and cannot tell a badly-worded
|
||||
trigger from a genuinely distant one.
|
||||
|
||||
Measured on this install, which is why the order is asserted rather than
|
||||
suggested: rule 1 "`dev` is home" scored 0.6515 and ranked 5th for the moment it
|
||||
governed, behind three rules that RESTRAINED the same act. Every percentile said
|
||||
"lower the floor" — and at 0.60 the arm delivered those three restraints and
|
||||
still not rule 1. Rewriting the trigger to lead with the symptom put the same
|
||||
rule 1st at 0.7130. The dial-first path was available, statistically supported,
|
||||
and wrong.
|
||||
Which is why the order is asserted rather than suggested: a rule can score
|
||||
0.65 and rank fifth for the moment it governs, behind rules that RESTRAIN the
|
||||
same act. Every percentile says "lower the floor" — and at 0.60 the arm
|
||||
delivers those restraints and still not that rule, while rewriting its trigger
|
||||
to lead with the symptom puts it first at 0.71. The dial-first path can be
|
||||
available, statistically supported, and wrong.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -50,8 +44,8 @@ TUNING_TOOLS = ("retrieval_telemetry", "update_rule", "retrieval_surfaces",
|
||||
|
||||
|
||||
def _skill() -> str:
|
||||
# The route lives in missed-retrieval.md, a reference file of using-scribe
|
||||
# (#4398); the skill is SKILL.md and its references read together.
|
||||
# The route lives in missed-retrieval.md, a reference file of using-scribe;
|
||||
# the skill is SKILL.md and its references read together.
|
||||
from tests.helpers import skill_text
|
||||
return skill_text("using-scribe")
|
||||
|
||||
@@ -142,17 +136,16 @@ def test_writing_a_new_record_asks_what_already_covers_the_moment():
|
||||
|
||||
|
||||
def test_the_route_stays_off_the_index():
|
||||
"""`_INSTRUCTIONS` is the only surface every MCP client gets (decision #4027).
|
||||
"""`_INSTRUCTIONS` is the only surface every MCP client gets.
|
||||
|
||||
It carried a one-line MISSED pointer to this route until #4389, which found
|
||||
the index was being used as a rulebook: a missed rule is noticed mid-work,
|
||||
not at session start, and the spec asks server instructions for the
|
||||
It carries no pointer to this route, because the index is not a rulebook:
|
||||
a missed rule is noticed mid-work, not at session start, and the spec asks server instructions for the
|
||||
workflow across tools, not stance. using-scribe states the route (the tests
|
||||
above pin it); the index keeps what_might_apply, the tool that opens it.
|
||||
|
||||
Whitespace-flattened before matching: the block is hard-wrapped, so a
|
||||
phrase straddles a line break the moment anything before it changes length
|
||||
(#4103).
|
||||
phrase straddles a line break the moment anything before it changes
|
||||
length.
|
||||
"""
|
||||
text = " ".join(_instructions().split()).lower()
|
||||
assert "what_might_apply" in text
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Which rules fired this session, which changed an action, which did not (#4216).
|
||||
"""Which rules fired this session, which changed an action, which did not.
|
||||
|
||||
WHY THIS CANNOT BE ASKED OF THE SERVER
|
||||
|
||||
@@ -7,13 +7,13 @@ a SESSION-scoped answer has to be assembled where a session is a thing that
|
||||
exists. That is the plugin, from four ledgers four hooks already write:
|
||||
|
||||
.rules.ids an arm NAMED the rule (a teaser was shown)
|
||||
.opened.ids the session called get_rule (#4100)
|
||||
.acted.ids the session called rule_outcome (#4216, new here)
|
||||
.checkpoint.ids the rule HELD an act (#4214)
|
||||
.opened.ids the session called get_rule
|
||||
.acted.ids the session called rule_outcome
|
||||
.checkpoint.ids the rule HELD an act
|
||||
|
||||
EVERY LINE IS AN OBSERVED TOOL CALL. Nothing asks the model what it followed;
|
||||
milestone 386 ruled that out because a model asked "did you apply rule 156?"
|
||||
will say yes. These record what happened.
|
||||
EVERY LINE IS AN OBSERVED TOOL CALL. Nothing asks the model what it followed,
|
||||
because a model asked "did you apply rule 156?" will say yes. These record
|
||||
what happened.
|
||||
|
||||
WHY THE COMPACTION SEAM
|
||||
|
||||
@@ -21,8 +21,8 @@ A rule read and left unresolved is invisible by construction — it looks
|
||||
exactly like a rule that worked. The compaction is where that invisibility
|
||||
becomes permanent: the turns holding the evidence are summarised away, and an
|
||||
unjudged thing that survives as nothing is how a decision quietly becomes
|
||||
nobody's. A PreCompact hook's stdout becomes the summariser's instructions
|
||||
(#3680), so this does not say "you slipped" — it says which ids must be
|
||||
nobody's. A PreCompact hook's stdout becomes the summariser's instructions,
|
||||
so this does not say "you slipped" — it says which ids must be
|
||||
carried through, which is the one thing a summary can do about it.
|
||||
|
||||
WHAT IS PINNED: the arithmetic, and which conditions produce which lines. NOT
|
||||
@@ -63,8 +63,7 @@ def ledgers(tmp_path, *, named=(), opened=(), acted=(), held=()) -> Path:
|
||||
|
||||
Not hand-rolled files: `scribe_rules_append` is what creates the evidence
|
||||
twin these read from, so a fixture that wrote the bytes itself would keep
|
||||
passing if the twin stopped being written — which is the whole defect
|
||||
#4217 found.
|
||||
passing if the twin stopped being written.
|
||||
"""
|
||||
d = tmp_path / "scribe-priorart"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
@@ -140,7 +139,7 @@ def test_a_rule_read_with_no_outcome_is_the_headline(tmp_path):
|
||||
|
||||
def test_a_rule_that_held_an_act_is_reported_separately(tmp_path):
|
||||
"""The strongest evidence a rule changed something: it stopped a call
|
||||
before it ran (#4214). Kept apart from `read` because reading a rule and
|
||||
before it ran. Kept apart from `read` because reading a rule and
|
||||
having it alter what you did are different claims."""
|
||||
out = readout(ledgers(tmp_path, named=[156], opened=[156], held=[156]))
|
||||
assert "held an act" in out and "156" in out
|
||||
@@ -158,7 +157,7 @@ def test_a_session_that_resolved_everything_makes_no_accusation(tmp_path):
|
||||
|
||||
|
||||
def test_a_session_no_rule_touched_says_nothing_at_all(tmp_path):
|
||||
"""Rule 115's reasoning: a fresh install must not be told something is
|
||||
"""A fresh install must not be told something is
|
||||
wrong when the truth is that nothing has happened yet."""
|
||||
d = tmp_path / "scribe-priorart"
|
||||
d.mkdir(parents=True)
|
||||
@@ -256,14 +255,12 @@ def test_the_recorder_is_shell_valid():
|
||||
subprocess.run(["bash", "-n", str(RECORDER)], check=True)
|
||||
|
||||
|
||||
# ── Evidence outlives the compaction; exclusions do not (#4217) ────────────
|
||||
# ── Evidence outlives the compaction; exclusions do not ───────────────────
|
||||
#
|
||||
# THE DEFECT THIS PINS, measured before it was fixed: across six sessions on
|
||||
# the instance this was built on, 208 `get_rule` calls produced 3 surviving
|
||||
# ledger entries. `.opened.ids` was doing two jobs with opposite lifetimes —
|
||||
# "this context holds the rule" (must be forgotten at a compaction, and three
|
||||
# hooks depend on that) and "this was opened" (which no compaction makes
|
||||
# untrue). The sweep, correct for the first, was deleting the second.
|
||||
# Two jobs with opposite lifetimes: "this context holds the rule" (must be
|
||||
# forgotten at a compaction, and three hooks depend on that) and "this was
|
||||
# opened" (which no compaction makes untrue). One ledger doing both loses
|
||||
# nearly every opened rule to the sweep, which is correct only for the first.
|
||||
|
||||
def test_the_readout_is_unchanged_by_the_sweep(tmp_path):
|
||||
"""The readout runs AT the compaction. If the sweep took its inputs it
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
"""The review surface: rows whose evidence no longer meets the bar, and
|
||||
canons whose own rows no longer agree what they are (#4204, #4208).
|
||||
canons whose own rows no longer agree what they are.
|
||||
|
||||
WHAT THESE GUARD, and it is a property rather than an output shape. The
|
||||
ledger was poisoned by a machine writing permanent classifications that
|
||||
nobody read — 32 rows on one project, 334 on another. The correction is NOT
|
||||
a machine that un-writes them; that is the same mistake pointed the other
|
||||
WHAT THESE GUARD, and it is a property rather than an output shape. A machine
|
||||
writing permanent classifications that nobody reads poisons the ledger. The
|
||||
correction is NOT a machine that un-writes them; that is the same mistake pointed the other
|
||||
way. It is a reader that puts the evidence in front of a judge.
|
||||
|
||||
So the tests below assert two things hardest:
|
||||
@@ -15,8 +14,8 @@ So the tests below assert two things hardest:
|
||||
* the listing changes nothing, which is checked by giving it rows and
|
||||
asserting the rows come back untouched.
|
||||
|
||||
The round-trip test exists because the score used to live only inside a
|
||||
prose sentence. Writer and parser are now one constant apart, and a change
|
||||
The round-trip test exists because the score is carried inside a prose
|
||||
sentence. Writer and parser are one constant apart, and a change
|
||||
to one that the other does not follow is exactly the kind of silent rot the
|
||||
hook's own history is made of.
|
||||
"""
|
||||
@@ -135,10 +134,9 @@ def test_a_canon_whose_members_agree_is_not_listed() -> None:
|
||||
|
||||
|
||||
def test_a_canon_whose_members_are_all_different_things_is_listed() -> None:
|
||||
"""Portal's #3283, in miniature: a class, a getter, a test and a binding
|
||||
recorded as instances of one pattern — and in four different files, as
|
||||
they really were. The paths matter now: a callable sharing a file with a
|
||||
class is read as that class's method (#4220), so putting them all in one
|
||||
"""A class, a getter, a test and a binding recorded as instances of one
|
||||
pattern — in four different files. The paths matter: a callable sharing a
|
||||
file with a class is read as that class's method, so putting them all in one
|
||||
file would test the excuse rather than the disagreement."""
|
||||
rows = [_member("class SessionAbsent(RuntimeError):", path="src/errors.py",
|
||||
symbol="SessionAbsent"),
|
||||
@@ -198,7 +196,7 @@ def test_the_review_changes_nothing_about_the_rows_it_reads() -> None:
|
||||
|
||||
|
||||
def test_the_service_carries_no_machinery_for_bulk_withdrawal() -> None:
|
||||
"""A structural guard (rule 167) on the design decision, not on output.
|
||||
"""A structural guard on the design decision, not on output.
|
||||
If someone adds an auto-retire path to the ledger, this fails and the
|
||||
conversation about whether a machine may un-judge in bulk happens again
|
||||
— deliberately, rather than in a diff nobody read."""
|
||||
@@ -209,19 +207,18 @@ def test_the_service_carries_no_machinery_for_bulk_withdrawal() -> None:
|
||||
assert banned not in body, banned
|
||||
|
||||
|
||||
# ── coherence is about what a canon CANNOT account for (#4220) ────────────
|
||||
# ── coherence is about what a canon CANNOT account for ────────────────────
|
||||
#
|
||||
# The first version of this check asked whether every member shared a form.
|
||||
# On its first live day the only two canons it reported were both sound,
|
||||
# while the genuinely polluted one had already been cleaned by hand — a
|
||||
# surface whose whole output is noise is one that stops being read.
|
||||
# Asking whether every member shares a form reports sound canons — a class and
|
||||
# its methods are two forms of one shape — and a surface whose whole output is
|
||||
# noise stops being read.
|
||||
#
|
||||
# The obvious repair — group by family, keep the majority test — is a trap,
|
||||
# and `test_family_grouping_alone_would_have_gone_blind` is the guard that
|
||||
# stops anyone walking back into it.
|
||||
# and `test_family_grouping_alone_passes_a_polluted_canon` is the guard that
|
||||
# stops anyone walking into it.
|
||||
|
||||
def _model(sym, path):
|
||||
"""A SQLAlchemy model class, as #2844 holds one."""
|
||||
"""A SQLAlchemy model class, as the model-convention canon holds one."""
|
||||
return _member(f"class {sym}(Base, TimestampMixin):", path=path)
|
||||
|
||||
|
||||
@@ -231,10 +228,9 @@ def _to_dict(path):
|
||||
|
||||
|
||||
def test_a_class_and_the_to_dict_beside_it_are_one_shape() -> None:
|
||||
"""#2844 exactly: the SQLAlchemy model convention, whose own text is
|
||||
about the class AND the to_dict it must carry. Live it scored
|
||||
37/62 = 0.597 and was called incoherent for containing precisely what it
|
||||
says it contains."""
|
||||
"""The SQLAlchemy model convention, whose own text is about the class AND
|
||||
the to_dict it must carry. A form-level check scores it 37/62 = 0.597 and
|
||||
calls it incoherent for containing precisely what it says it contains."""
|
||||
rows = []
|
||||
for i in range(6):
|
||||
rows += [_model(f"M{i}", f"src/scribe/models/m{i}.py"),
|
||||
@@ -244,9 +240,9 @@ def test_a_class_and_the_to_dict_beside_it_are_one_shape() -> None:
|
||||
|
||||
|
||||
def test_a_sync_starter_and_the_async_tick_it_schedules_are_one_shape() -> None:
|
||||
"""#2849: four loop-starters and three ticks. `shape_family` already
|
||||
collapses fn and async-fn for the divergence gate; this surface was the
|
||||
one place still asking at form level."""
|
||||
"""Four loop-starters and three ticks. `shape_family` collapses fn and
|
||||
async-fn for the divergence gate, and this surface asks at the same
|
||||
level."""
|
||||
rows = [_member("def start_notification_loop() -> None:", path="src/n.py"),
|
||||
_member("def start_log_retention_loop() -> None:", path="src/l.py"),
|
||||
_member("def start_auth_token_retention_loop() -> None:", path="src/a.py"),
|
||||
@@ -256,10 +252,10 @@ def test_a_sync_starter_and_the_async_tick_it_schedules_are_one_shape() -> None:
|
||||
assert stamps_to_review(rows)["incoherent_count"] == 0
|
||||
|
||||
|
||||
def test_family_grouping_alone_would_have_gone_blind() -> None:
|
||||
"""THE REGRESSION GUARD. Polluted #2844 as it stood this morning: 37
|
||||
model classes, 25 to_dict methods beside them, and 31 rows that had no
|
||||
business being there. Counted as families that is 56 callables to 37
|
||||
def test_family_grouping_alone_passes_a_polluted_canon() -> None:
|
||||
"""THE REGRESSION GUARD. A polluted model canon: 37 model classes, 25
|
||||
to_dict methods beside them, and 31 rows that have no business being
|
||||
there. Counted as families that is 56 callables to 37
|
||||
types — 0.602, a clean pass under any majority test. Those 31 rows are
|
||||
the ones the surface exists to find, so a change that lets this canon
|
||||
read as coherent has broken the feature while keeping every other test
|
||||
@@ -352,12 +348,11 @@ def test_the_coherence_verdict_reads_rows_and_writes_nothing() -> None:
|
||||
for r in rows] == before
|
||||
|
||||
|
||||
# ── a weak stamp is listed for judgment and counted by nobody (#4608) ─────
|
||||
# ── a weak stamp is listed for judgment and counted by nobody ─────────────
|
||||
#
|
||||
# The floor guarded new writes; the rows written before it went on electing
|
||||
# canons. On two projects a YAML CI snippet "dominated" a Svelte directory on
|
||||
# the strength of 0.68-0.72 stamps, and every write there was told it
|
||||
# diverged from it. The readers below must agree with the review surface
|
||||
# The floor guards new writes; rows written below it would otherwise go on
|
||||
# electing canons, so an unrelated snippet "dominates" a directory on the
|
||||
# strength of 0.68-0.72 stamps and every write there is told it diverged. The readers below must agree with the review surface
|
||||
# about what "weak" means — one predicate, so they cannot drift apart.
|
||||
|
||||
def test_one_predicate_says_what_weak_is() -> None:
|
||||
|
||||
@@ -1,16 +1,13 @@
|
||||
"""A sick database must not stop the app from starting — #4181, rule 156.
|
||||
"""A sick database must not stop the app from starting.
|
||||
|
||||
THE INCIDENT THESE GUARD, so a later reader knows what is being defended:
|
||||
THE FAILURE THESE GUARD. A storage stall can make Postgres answer in minutes
|
||||
instead of milliseconds. If the app starts into one, a startup read with no
|
||||
deadline hangs; Hypercorn kills the worker at its 60-second lifespan timeout;
|
||||
and nothing retries a failed lifespan. A short disk hiccup becomes an outage
|
||||
that only a human restart can clear.
|
||||
|
||||
On 2026-09-19 a host storage stall made one Postgres checkpoint of 14 buffers
|
||||
take 281 seconds against a 1.3-second baseline. The app restarted into the tail
|
||||
of it; `get_maintenance_hour()` — the first database read in `before_serving` —
|
||||
hung with no deadline; Hypercorn killed the worker at its 60-second lifespan
|
||||
timeout; and nothing retries a failed lifespan. A five-minute disk hiccup became
|
||||
a three-hour outage that only a human restart could clear.
|
||||
|
||||
Every assertion here is about the SHAPE that made that possible, not about the
|
||||
stall, which no test can reproduce and no code can prevent:
|
||||
Every assertion here is about the SHAPE that makes that possible, not about
|
||||
the stall, which no test can reproduce and no code can prevent:
|
||||
|
||||
- the startup read returns on a schedule of its own rather than the
|
||||
database's;
|
||||
@@ -48,9 +45,9 @@ async def test_a_hanging_settings_read_does_not_hang_startup():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_timeout_is_logged_loudly_enough_to_find():
|
||||
"""#4181 cost hours because the app logged three scheduler lines and
|
||||
stopped; the cause was only recoverable from Postgres's own log. A WARNING
|
||||
here is the breadcrumb that would have named it in seconds."""
|
||||
"""Without a WARNING the app logs a few scheduler lines and stops, and the
|
||||
cause is only recoverable from Postgres's own log. This is the breadcrumb
|
||||
that names it."""
|
||||
from scribe.services import db_maintenance_scheduler as sched
|
||||
|
||||
async def _never_returns(*_a, **_kw):
|
||||
@@ -60,7 +57,7 @@ async def test_the_timeout_is_logged_loudly_enough_to_find():
|
||||
patch.object(sched, "_STARTUP_READ_TIMEOUT", 0.05), \
|
||||
patch.object(sched.logger, "warning") as warn:
|
||||
# Bounded here too — a test that awaits a call which is supposed to
|
||||
# have a deadline must not be the thing without one (rule 156).
|
||||
# have a deadline must not be the thing without one.
|
||||
await asyncio.wait_for(sched.get_maintenance_hour(), timeout=2)
|
||||
|
||||
assert warn.called
|
||||
@@ -105,11 +102,9 @@ def test_the_startup_deadline_leaves_room_inside_the_lifespan_budget():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_deferred_backfill_waits_for_the_startup_flag():
|
||||
"""`_delayed_backfill`'s comment said it "never blocks the server from
|
||||
accepting requests" — true of requests, false of startup, because startup
|
||||
had not finished and the two competed for one connection pool. Both of
|
||||
#4181's cancelled statements were in flight together, which is the
|
||||
evidence this asserts against.
|
||||
""""Never blocks the server from accepting requests" is true of requests
|
||||
and false of startup: a backfill that starts before startup finishes
|
||||
competes with it for one connection pool.
|
||||
|
||||
Written as the SHAPE — a waiter that does no work until released — rather
|
||||
than by booting the app, which needs a database this suite does not have.
|
||||
@@ -133,7 +128,7 @@ async def test_the_deferred_backfill_waits_for_the_startup_flag():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_flag_is_released_even_when_startup_fails():
|
||||
"""Rules 156 and 157 together. The waiter has no deadline of its own, so
|
||||
"""The waiter has no deadline of its own, so
|
||||
the release has to be unmissable: a `finally`, never a last line. Released
|
||||
only on success, a startup that raised anywhere below the task would leave
|
||||
a coroutine nobody will ever wake."""
|
||||
@@ -164,7 +159,7 @@ def test_startup_releases_the_flag_in_a_finally():
|
||||
finally_at = body.index("finally:")
|
||||
assert finally_at < body.index("_startup_finished.set()"), (
|
||||
"the flag is released after the work rather than in a finally — a "
|
||||
"startup that raises would strand the backfill forever (rule 157)"
|
||||
"startup that raises would strand the backfill forever"
|
||||
)
|
||||
assert "await _startup_finished.wait()" in src
|
||||
|
||||
@@ -184,7 +179,7 @@ def test_the_engine_bounds_how_long_a_connect_may_take():
|
||||
from scribe.models import _CONNECT_ARGS
|
||||
|
||||
timeout = _CONNECT_ARGS.get("timeout")
|
||||
assert timeout is not None, "a connect with no deadline (rule 156)"
|
||||
assert timeout is not None, "a connect with no deadline"
|
||||
assert 0 < timeout < 60
|
||||
|
||||
|
||||
@@ -192,7 +187,7 @@ def test_no_blanket_command_timeout_was_added_with_it():
|
||||
"""The deliberate omission, guarded so nobody adds it as an obvious
|
||||
follow-up. `command_timeout` applies to EVERY statement, and this app runs
|
||||
long ones on purpose — the embedding backfills, VACUUM ANALYZE. It would
|
||||
trade #4181's failure mode for a worse one."""
|
||||
trade a hung startup for killed maintenance."""
|
||||
from scribe.models import _CONNECT_ARGS
|
||||
|
||||
assert "command_timeout" not in _CONNECT_ARGS
|
||||
@@ -211,7 +206,7 @@ def test_the_engine_actually_uses_those_args():
|
||||
|
||||
|
||||
def test_the_startup_guards_can_fail():
|
||||
"""Rule 167: each assertion above has to be able to bite."""
|
||||
"""Each assertion above has to be able to bite."""
|
||||
from scribe.services.db_maintenance_scheduler import _STARTUP_READ_TIMEOUT
|
||||
|
||||
# An unbounded read is the thing being prevented.
|
||||
|
||||
@@ -1,34 +1,23 @@
|
||||
"""Both doors can file a record under a System, at create AND at update (#4249).
|
||||
"""Both doors can file a record under a System, at create AND at update.
|
||||
|
||||
WHAT THIS IS ABOUT. A System tag is how `list_system_records` gathers an
|
||||
area's whole pile. A record that cannot be tagged is reachable by search and
|
||||
by nothing else — so a missing `system_ids` parameter is not a cosmetic API
|
||||
gap, it is a record that quietly never joins the area it belongs to.
|
||||
|
||||
THE DEFECT IS THE ASYMMETRY, NOT ANY ONE PARAMETER. When #4249 was filed it
|
||||
read as "the REST door can do something the MCP door cannot", on three
|
||||
instances in a row. Filling in the whole table showed that was the wrong
|
||||
generalisation:
|
||||
|
||||
kind MCP create MCP update REST create REST update
|
||||
lesson yes NO yes yes
|
||||
preference NO yes - -
|
||||
note yes yes NO NO
|
||||
process NO NO NO NO
|
||||
|
||||
`preference` was the exact mirror of `lesson` — the same capability missing
|
||||
from the opposite end of the same lifecycle — and `note` ran the opposite way
|
||||
round from the issue's own premise. The real pattern is that **whichever door
|
||||
nobody exercised for a kind is the one that never grew the parameter**, and
|
||||
that is not a thing a human reviewer reliably notices, because each door is
|
||||
only ever read on its own.
|
||||
THE DEFECT IS THE ASYMMETRY, NOT ANY ONE PARAMETER. A missing door shows up
|
||||
one kind at a time and reads like a one-off — REST lacking what MCP has for
|
||||
one kind, the opposite for the next, a create without the update. The real
|
||||
pattern is that **whichever door nobody exercises for a kind is the one that
|
||||
never grows the parameter**, and that is not a thing a human reviewer reliably
|
||||
notices, because each door is only ever read on its own.
|
||||
|
||||
So this file does not assert that any particular function takes the argument.
|
||||
It asserts the SHAPE OF THE TABLE: every kind that can be System-tagged
|
||||
anywhere can be System-tagged everywhere it is written. A fifth kind added
|
||||
next month fails here rather than shipping half-wired — the same reasoning
|
||||
test_derived_mirror_generic_door.py records for derived mirrors (#3734), where
|
||||
testing one constant let the second kind arrive with none of the protection.
|
||||
test_derived_mirror_generic_door.py records for derived mirrors, where testing
|
||||
one constant lets the second kind arrive with none of the protection.
|
||||
|
||||
NOT A GAP — MILESTONES. `RecordSystem.note_id` is a ForeignKey to `notes.id`
|
||||
and milestones live in their own table, so milestones cannot be System-tagged
|
||||
@@ -94,8 +83,8 @@ def _rest_source(module: str) -> str:
|
||||
def test_the_mcp_door_takes_system_ids_at_both_ends(kind, mcp_mod, create, update, _rest):
|
||||
"""Create AND update. Tagging available only at create is tagging at the
|
||||
moment of LEAST information — a lesson written at the end of a piece of
|
||||
work is exactly when the argument gets dropped, and before #4249 there was
|
||||
then no way to file it ever again."""
|
||||
work is exactly when the argument gets dropped, and without the update
|
||||
door there is no way to file it ever again."""
|
||||
tree = _mcp(mcp_mod)
|
||||
for fn_name in (create, update):
|
||||
fn = _find(tree, fn_name)
|
||||
@@ -138,7 +127,7 @@ def test_every_tool_module_that_tags_is_in_the_registry():
|
||||
"""The guard that makes the two tests above keep working.
|
||||
|
||||
Without this, the registry is a list somebody has to remember to extend,
|
||||
and #4249 IS the failure of somebody remembering. Any MCP tool module that
|
||||
and somebody remembering is what fails. Any MCP tool module that
|
||||
ACCEPTS `system_ids` must be registered — so a new kind that wires up
|
||||
tagging is dragged into the parity checks by its own first use of the
|
||||
parameter, rather than being half-wired quietly.
|
||||
@@ -147,7 +136,7 @@ def test_every_tool_module_that_tags_is_in_the_registry():
|
||||
`projects.py`, `search.py` and `systems.py`, which only name the argument
|
||||
in prose while explaining it to the reader — and the fix for that is a
|
||||
structural check, not a hand-kept skip list that would itself go stale
|
||||
(rule 167: a guard asserts on structure).
|
||||
(a guard asserts on structure).
|
||||
"""
|
||||
registered = {r[1] for r in REGISTRY}
|
||||
tagging = set()
|
||||
@@ -195,11 +184,8 @@ def test_every_tagging_write_acts_as_the_caller():
|
||||
the caller's write access first (a route's `can_write_note`, or a service
|
||||
that raises PermissionError). What this decides is WHOSE reach the tagging
|
||||
runs with, which is exactly the kind of difference that survives review
|
||||
because each call site reads fine on its own.
|
||||
|
||||
Four of twenty sites passed the owner before #4249 — two routes, two MCP
|
||||
tools, one of them written earlier in the very session that unified them.
|
||||
That is the tell that this needs a guard rather than care.
|
||||
because each call site reads fine on its own — the tell that this needs a
|
||||
guard rather than care.
|
||||
"""
|
||||
offenders = []
|
||||
for path in ROOT.rglob("*.py"):
|
||||
@@ -227,7 +213,7 @@ def test_every_tagging_write_acts_as_the_caller():
|
||||
assert not offenders, (
|
||||
"set_record_systems must be called with the acting user, not the "
|
||||
"record's owner — passing the owner bypasses its own access check and "
|
||||
"borrows the owner's System visibility (#47, #4249):\n "
|
||||
"borrows the owner's System visibility:\n "
|
||||
+ "\n ".join(offenders)
|
||||
)
|
||||
|
||||
@@ -250,6 +236,6 @@ def test_milestones_are_absent_on_purpose():
|
||||
assert 'ForeignKey("notes.id"' in seg, (
|
||||
"RecordSystem no longer keys on notes.id — if it grew a generic "
|
||||
"record reference, milestones may now be taggable and this test, the "
|
||||
"registry above and #4249's table all need revisiting."
|
||||
"registry above and the door table all need revisiting."
|
||||
)
|
||||
assert "milestone" not in {r[0] for r in REGISTRY}
|
||||
|
||||
+15
-16
@@ -1,10 +1,10 @@
|
||||
"""The wide net: many candidates, no bar, and a cost that stays bounded (#4103).
|
||||
"""The wide net: many candidates, no bar, and a cost that stays bounded.
|
||||
|
||||
WHY THIS EXISTS
|
||||
|
||||
Milestone 416 step 5 moves "do not exclude potentially important data" off the
|
||||
push, where fifty candidates would be a wall nobody reads, and onto a pull,
|
||||
where they arrive only when a session asks for them.
|
||||
"Do not exclude potentially important data" belongs on a pull, not the push:
|
||||
fifty candidates pushed would be a wall nobody reads, and pulled they arrive
|
||||
only when a session asks for them.
|
||||
|
||||
Three things have to hold, and each has a way of quietly failing:
|
||||
|
||||
@@ -12,13 +12,13 @@ Three things have to hold, and each has a way of quietly failing:
|
||||
not trust a threshold to decide for them. A default floor creeping in
|
||||
here would turn the wide net into the narrow one and nothing would look
|
||||
wrong — the results would simply be fewer.
|
||||
2. **A bounded cost.** The step was filed claiming fifty "costs nothing".
|
||||
`_rule_hint_line` had already measured ~143 tokens for a line with its
|
||||
trigger rendered, so fifty is ~7,000. The graduated shape (#3851) is what
|
||||
keeps this affordable, and a change that renders every trigger in full
|
||||
2. **A bounded cost.** Fifty candidates do not cost nothing: a line with
|
||||
its trigger rendered is ~143 tokens, so fifty is ~7,000. The graduated
|
||||
shape is what keeps this affordable, and a change that renders every trigger in full
|
||||
would pass every other test here.
|
||||
3. **A pull, logged as one.** The push arms' near-miss distributions are the
|
||||
evidence #4121 argues from. A pull mixed into them moves those numbers.
|
||||
evidence their floors are tuned from. A pull mixed into them moves those
|
||||
numbers.
|
||||
"""
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
@@ -71,7 +71,7 @@ async def test_the_limit_is_clamped_to_the_cap(asked, expected):
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_head_carries_its_trigger_whole_and_the_tail_is_cut():
|
||||
"""The graduated shape (#3851), which is what makes fifty affordable.
|
||||
"""The graduated shape, which is what makes fifty affordable.
|
||||
|
||||
A regression that rendered every trigger in full would satisfy every other
|
||||
assertion in this file, so the cut is pinned on both sides: the head is
|
||||
@@ -104,8 +104,7 @@ async def test_a_short_trigger_is_never_marked_truncated():
|
||||
|
||||
|
||||
def test_the_cut_breaks_on_a_word_and_says_it_was_cut():
|
||||
"""#4036's lesson, borrowed: a raw slice ends mid-word and reads as the
|
||||
whole thing."""
|
||||
"""A raw slice ends mid-word and reads as the whole thing."""
|
||||
text, cut = wide_net._teaser("alpha beta gamma delta " * 40)
|
||||
assert cut is True
|
||||
assert text.endswith("…")
|
||||
@@ -152,7 +151,7 @@ async def test_the_call_is_logged_under_its_own_pull_source():
|
||||
|
||||
|
||||
def test_the_wide_net_is_not_one_of_the_tunable_push_surfaces():
|
||||
"""THE GUARD that keeps #4121's evidence clean (rule 167).
|
||||
"""THE GUARD that keeps the push arms' tuning evidence clean.
|
||||
|
||||
The registry holds the PUSH arms — the ones with a floor and a budget the
|
||||
model tunes. This source must not appear there: a pull folded into those
|
||||
@@ -175,7 +174,7 @@ def test_the_wide_net_is_not_ambient_either():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_search_that_never_ran_is_not_reported_as_a_decline():
|
||||
"""#3765: an empty query, a dead embedder and a failed query all return
|
||||
"""An empty query, a dead embedder and a failed query all return
|
||||
nothing, and none of them is a ranker declining."""
|
||||
with _patched([], report={"searched": False, "best_available_score": None}), \
|
||||
patch.object(wide_net, "record_retrieval") as rec, \
|
||||
@@ -189,7 +188,7 @@ async def test_a_search_that_never_ran_is_not_reported_as_a_decline():
|
||||
@pytest.mark.asyncio
|
||||
async def test_what_the_bar_turned_away_is_carried_through():
|
||||
"""There is no bar here, but `best_available` still answers "did the corpus
|
||||
have anything at all" for a call that came back empty (#3670)."""
|
||||
have anything at all" for a call that came back empty."""
|
||||
with _patched([], report={"searched": True, "best_available_score": 0.31,
|
||||
"best_available_id": 7}), \
|
||||
patch.object(wide_net, "record_retrieval") as rec, \
|
||||
@@ -230,7 +229,7 @@ def test_it_is_registered_and_readable_with_a_read_key():
|
||||
|
||||
|
||||
def test_the_instruction_surfaces_point_at_it():
|
||||
"""Rule 119: the instruction surfaces ARE the specification for product
|
||||
"""The instruction surfaces ARE the specification for product
|
||||
behaviour, so a tool the reflex never learns about is not shipped."""
|
||||
import pathlib
|
||||
|
||||
|
||||
Reference in New Issue
Block a user