docs(6209): calibration stamp, after-write hook, usage seam, work log, duplicates, skill structure, tuning and rulebook routes, kind surface, prompt filter, moments, strict args, auth, lesson write path
Fourteen more test files: docstrings state the failure each guard defends as a present-tense shape instead of the incident that found it, and issue, milestone and rule numbers leave comments and assertion messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,4 +1,4 @@
|
|||||||
"""The PostToolUse after-write hook (#2901): code written through Bash — sed,
|
"""The PostToolUse after-write hook: code written through Bash — sed,
|
||||||
heredocs, scripts — gets the same prior-art / ledger checks as a Write/Edit.
|
heredocs, scripts — gets the same prior-art / ledger checks as a Write/Edit.
|
||||||
|
|
||||||
Runs the real shell against a temp git repo and a throwaway HTTP sink, like
|
Runs the real shell against a temp git repo and a throwaway HTTP sink, like
|
||||||
@@ -103,7 +103,7 @@ def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
|
|||||||
assert _run(loose, env, session="s-loose") == ""
|
assert _run(loose, env, session="s-loose") == ""
|
||||||
# A change that defines nothing (prose, a call-site edit) → nothing, even
|
# A change that defines nothing (prose, a call-site edit) → nothing, even
|
||||||
# with the server unreachable (port 9 refuses): no definitions, no call
|
# with the server unreachable (port 9 refuses): no definitions, no call
|
||||||
# owed, so not even the #2932 outage line. (a.css above is removed first:
|
# owed, so not even the outage line. (a.css above is removed first:
|
||||||
# it DOES define a shape, and an unanswered call for it would rightly speak.)
|
# it DOES define a shape, and an unanswered call for it would rightly speak.)
|
||||||
(repo / "a.css").unlink()
|
(repo / "a.css").unlink()
|
||||||
(repo / "README.md").write_text("# notes\n")
|
(repo / "README.md").write_text("# notes\n")
|
||||||
@@ -112,9 +112,9 @@ def test_after_write_is_silent_where_it_has_nothing_to_say(tmp_path):
|
|||||||
|
|
||||||
|
|
||||||
def test_after_write_local_arm_works_without_a_server_and_says_the_server_did_not_answer(tmp_path):
|
def test_after_write_local_arm_works_without_a_server_and_says_the_server_did_not_answer(tmp_path):
|
||||||
"""The local by-name arm needs no instance (#2280). A configured instance
|
"""The local by-name arm needs no instance. A configured instance that
|
||||||
that does not ANSWER (a refused connection stands in for it) is said, once
|
does not ANSWER (a refused connection stands in for it) is said, once per
|
||||||
per outage (#2932) — and the record nudge, which claims "nothing recorded",
|
outage — and the record nudge, which claims "nothing recorded",
|
||||||
is withheld: no answer backs that claim."""
|
is withheld: no answer backs that claim."""
|
||||||
env = _env(tmp_path)
|
env = _env(tmp_path)
|
||||||
repo = _repo(tmp_path, env)
|
repo = _repo(tmp_path, env)
|
||||||
@@ -138,7 +138,7 @@ def test_after_write_local_arm_works_without_a_server_and_says_the_server_did_no
|
|||||||
|
|
||||||
def test_after_write_unconfigured_install_owes_no_call_and_keeps_the_record_nudge(tmp_path):
|
def test_after_write_unconfigured_install_owes_no_call_and_keeps_the_record_nudge(tmp_path):
|
||||||
"""No URL/token → no call was owed, so nothing is "unreached"; the local
|
"""No URL/token → no call was owed, so nothing is "unreached"; the local
|
||||||
arm and the record nudge (#2664) stand on their own, as before."""
|
arm and the record nudge stand on their own."""
|
||||||
env = {k: v for k, v in _env(tmp_path).items() if k not in ("SCRIBE_URL", "SCRIBE_TOKEN")}
|
env = {k: v for k, v in _env(tmp_path).items() if k not in ("SCRIBE_URL", "SCRIBE_TOKEN")}
|
||||||
repo = _repo(tmp_path, env)
|
repo = _repo(tmp_path, env)
|
||||||
(repo / "d.py").write_text("def slug(t):\n return t.lower()\n")
|
(repo / "d.py").write_text("def slug(t):\n return t.lower()\n")
|
||||||
|
|||||||
@@ -1,10 +1,9 @@
|
|||||||
"""A tuned number carries the space it was measured in — and only that (#4104).
|
"""A tuned number carries the space it was measured in — and only that.
|
||||||
|
|
||||||
WHY THIS EXISTS
|
WHY THIS EXISTS
|
||||||
|
|
||||||
Milestone 416 step 6 answers *"a path for thresholds to be inherited by the next
|
A threshold should be inheritable by the next version or a different model
|
||||||
version or different model so that they don't have to recalibrate a lot."* The
|
without recalibrating from scratch. The mechanism is a stamp: every tuning event records the embedding model and
|
||||||
mechanism is a stamp: every tuning event records the embedding model and
|
|
||||||
document shape the number was chosen under, and a mismatch is reported rather
|
document shape the number was chosen under, and a mismatch is reported rather
|
||||||
than left to be noticed.
|
than left to be noticed.
|
||||||
|
|
||||||
@@ -49,7 +48,7 @@ def test_calibration_stamp_names_the_embedder_and_the_shape():
|
|||||||
"embedding_model": emb.EMBEDDING_MODEL,
|
"embedding_model": emb.EMBEDDING_MODEL,
|
||||||
"shape_version": emb.CHUNKER_VERSION,
|
"shape_version": emb.CHUNKER_VERSION,
|
||||||
}
|
}
|
||||||
# Two keys, never one fused string (rule 149): a mismatch has to be able to
|
# Two keys, never one fused string: a mismatch has to be able to
|
||||||
# say WHICH half moved, because a new embedder and a re-cut document
|
# say WHICH half moved, because a new embedder and a re-cut document
|
||||||
# invalidate the same numbers for different reasons.
|
# invalidate the same numbers for different reasons.
|
||||||
assert len(stamp) == 2
|
assert len(stamp) == 2
|
||||||
@@ -136,7 +135,7 @@ def test_a_dial_moved_before_stamps_existed_is_unknown_not_fine():
|
|||||||
|
|
||||||
|
|
||||||
def test_a_model_change_and_a_shape_change_are_reported_apart():
|
def test_a_model_change_and_a_shape_change_are_reported_apart():
|
||||||
"""Rule 149. Two conditions, two answers — they call for different work."""
|
"""Two conditions, two answers — they call for different work."""
|
||||||
s = get_surface("prompt_rule")
|
s = get_surface("prompt_rule")
|
||||||
live = {"embedding_model": "new/model", "shape_version": 1}
|
live = {"embedding_model": "new/model", "shape_version": 1}
|
||||||
cal = rt._calibration(_row(model="old/model", shape=1), s, live)
|
cal = rt._calibration(_row(model="old/model", shape=1), s, live)
|
||||||
@@ -182,8 +181,7 @@ def test_the_tool_says_nothing_auto_retunes():
|
|||||||
"""The contract an agent reads. A stale stamp is a prompt to MEASURE.
|
"""The contract an agent reads. A stale stamp is a prompt to MEASURE.
|
||||||
|
|
||||||
Without this, the obvious next move on seeing `stale: true` is to move the
|
Without this, the obvious next move on seeing `stale: true` is to move the
|
||||||
dial — which is tuning from a statistic, the exact failure #4102 measured
|
dial — which is tuning from a statistic, which can point the wrong way.
|
||||||
pointing the wrong way.
|
|
||||||
"""
|
"""
|
||||||
doc = tool_doc("scribe.mcp.tools.retrieval_tuning", "retrieval_surfaces")
|
doc = tool_doc("scribe.mcp.tools.retrieval_tuning", "retrieval_surfaces")
|
||||||
assert "calibration" in doc
|
assert "calibration" in doc
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Creating a lesson, and tying it to what taught it (milestone 385 step 4).
|
"""Creating a lesson, and tying it to what taught it.
|
||||||
|
|
||||||
THE TRIGGER IS THE RECORD, so the door refuses a lesson without one.
|
THE TRIGGER IS THE RECORD, so the door refuses a lesson without one.
|
||||||
|
|
||||||
@@ -136,8 +136,8 @@ async def test_the_gate_compares_the_composed_document_not_the_raw_fields():
|
|||||||
|
|
||||||
|
|
||||||
def test_a_lesson_is_judged_at_the_trigger_dominated_bar():
|
def test_a_lesson_is_judged_at_the_trigger_dominated_bar():
|
||||||
"""#2518 measured deliberately-parallel siblings at 0.92 on a document that
|
"""Deliberately-parallel siblings measure 0.92 on a document that is
|
||||||
is mostly prose ABOUT the thing. A lesson's document is that shape, so the
|
mostly prose ABOUT the thing. A lesson's document is that shape, so the
|
||||||
bar sits above that band — otherwise two different lessons about one area
|
bar sits above that band — otherwise two different lessons about one area
|
||||||
block each other, which is the failure step 4 asked to check for."""
|
block each other, which is the failure step 4 asked to check for."""
|
||||||
from scribe.services.dedup import (
|
from scribe.services.dedup import (
|
||||||
@@ -153,7 +153,7 @@ def test_a_lesson_is_judged_at_the_trigger_dominated_bar():
|
|||||||
)
|
)
|
||||||
assert _LESSON_SEMANTIC_THRESHOLD > _SEMANTIC_THRESHOLD
|
assert _LESSON_SEMANTIC_THRESHOLD > _SEMANTIC_THRESHOLD
|
||||||
# The carve-out is per kind, not a loosening of the gate: a lesson's bar is
|
# The carve-out is per kind, not a loosening of the gate: a lesson's bar is
|
||||||
# its own, apart from the note's copy band (#4306).
|
# its own, apart from the note's copy band.
|
||||||
from scribe.services.dedup import _NOTE_COPY_THRESHOLD
|
from scribe.services.dedup import _NOTE_COPY_THRESHOLD
|
||||||
|
|
||||||
assert _semantic_threshold("note") == _NOTE_COPY_THRESHOLD
|
assert _semantic_threshold("note") == _NOTE_COPY_THRESHOLD
|
||||||
@@ -163,7 +163,7 @@ def test_a_lesson_is_judged_at_the_trigger_dominated_bar():
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_an_update_recomposes_both_halves_of_the_document():
|
async def test_an_update_recomposes_both_halves_of_the_document():
|
||||||
"""A new trigger has to reach the mirror AND the head of the body — the two
|
"""A new trigger has to reach the mirror AND the head of the body — the two
|
||||||
places the embedded document reads it from (milestone 427). Patching one
|
places the embedded document reads it from. Patching one
|
||||||
would leave a lesson that reads correctly and ranks on the old situation —
|
would leave a lesson that reads correctly and ranks on the old situation —
|
||||||
the failure mode with no symptom. The title stays the subject, even when
|
the failure mode with no symptom. The title stays the subject, even when
|
||||||
the stored one was an un-migrated composed title."""
|
the stored one was an un-migrated composed title."""
|
||||||
|
|||||||
@@ -108,16 +108,15 @@ def test_every_registered_tool_is_classified_exactly_once():
|
|||||||
"""Every tool must be declared a read, a write, or a read-shaped write.
|
"""Every tool must be declared a read, a write, or a read-shaped write.
|
||||||
|
|
||||||
`_READ_ONLY_TOOLS` is hand-maintained, and default-deny means a read
|
`_READ_ONLY_TOOLS` is hand-maintained, and default-deny means a read
|
||||||
omitted from it fails CLOSED — safe, but silent. That is how a read key
|
omitted from it fails CLOSED — safe, but silent: a read key able to
|
||||||
ended up able to `get_note` and not `get_snippet` (#2496), and how
|
`get_note` and not `get_snippet`.
|
||||||
`rules_due_for_verification` and `rule_history` sat denied (#3191).
|
|
||||||
|
|
||||||
The fix keeps the DECISION explicit and derives the CANDIDATES. Deriving
|
So the DECISION stays explicit and the CANDIDATES are derived. Deriving
|
||||||
the decision would make a security boundary follow a naming convention, so
|
the decision would make a security boundary follow a naming convention, so
|
||||||
any future `get_*` grants itself access. This test used to derive
|
any future `get_*` grants itself access. Deriving the candidates from names
|
||||||
candidates from names too — tools starting `get_`, `list_`, `search`… — and
|
— tools starting `get_`, `list_`, `search`… — misses reads whose names do
|
||||||
that narrowing is exactly what let the two #3191 reads through: neither
|
not look like reads (`rules_due_for_verification`, `rule_history`). So the
|
||||||
name looked like a read. So the candidates are now EVERY registered tool.
|
candidates are EVERY registered tool.
|
||||||
"""
|
"""
|
||||||
from scribe.mcp.server import (
|
from scribe.mcp.server import (
|
||||||
_DELIBERATELY_WRITE_SCOPED, _READ_ONLY_TOOLS, _WRITE_TOOLS,
|
_DELIBERATELY_WRITE_SCOPED, _READ_ONLY_TOOLS, _WRITE_TOOLS,
|
||||||
@@ -156,7 +155,7 @@ def test_every_registered_tool_is_classified_exactly_once():
|
|||||||
|
|
||||||
def test_the_completeness_check_can_fail():
|
def test_the_completeness_check_can_fail():
|
||||||
"""A guard that cannot fail is indistinguishable from one that is broken
|
"""A guard that cannot fail is indistinguishable from one that is broken
|
||||||
(rule 167). Drop a real tool from its set and the check must notice."""
|
Drop a real tool from its set and the check must notice."""
|
||||||
from scribe.mcp.server import _READ_ONLY_TOOLS, _WRITE_TOOLS
|
from scribe.mcp.server import _READ_ONLY_TOOLS, _WRITE_TOOLS
|
||||||
|
|
||||||
registered = _registered_tool_names()
|
registered = _registered_tool_names()
|
||||||
|
|||||||
@@ -1,13 +1,12 @@
|
|||||||
"""Unknown tool arguments are rejected, never silently dropped (#2709).
|
"""Unknown tool arguments are rejected, never silently dropped.
|
||||||
|
|
||||||
The failure this pins: MCPServer validates tool arguments with a pydantic model
|
The failure this pins: MCPServer validates tool arguments with a pydantic model
|
||||||
whose extra-field policy is "ignore", so `create_note(content=...)` — a
|
whose extra-field policy is "ignore", so `create_note(content=...)` — a
|
||||||
plausible near-miss for `body=`, primed by add_task_log's `content` — ran
|
plausible near-miss for `body=`, primed by add_task_log's `content` — would
|
||||||
successfully, stored `body: ""`, and left a record embedding/search cannot
|
run, store `body: ""`, and leave a record embedding/search cannot see, while
|
||||||
see. The call REPORTED SUCCESS. Two real notes were persisted body-less
|
REPORTING SUCCESS.
|
||||||
before the pattern was noticed.
|
|
||||||
|
|
||||||
The fix is at the dispatch seam, not per-tool: StrictArgsMCPServer rejects any
|
The rejection is at the dispatch seam, not per-tool: StrictArgsMCPServer rejects any
|
||||||
call carrying arguments the tool does not declare, with a did-you-mean when
|
call carrying arguments the tool does not declare, with a did-you-mean when
|
||||||
one is close. An error the caller sees once beats data half-written forever.
|
one is close. An error the caller sees once beats data half-written forever.
|
||||||
"""
|
"""
|
||||||
@@ -59,11 +58,11 @@ async def test_unknown_tool_keeps_the_upstream_error():
|
|||||||
assert "no_such_tool" in str(exc.value)
|
assert "no_such_tool" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
# --- a refusal's text reaches the agent (#4794) -------------------------------
|
# --- a refusal's text reaches the agent ---------------------------------------
|
||||||
#
|
#
|
||||||
# SDK 2.x passes on only a ToolError's text; anything else is wrapped as
|
# SDK 2.x passes on only a ToolError's text; anything else is wrapped as
|
||||||
# "Error executing tool X" and its message is kept on the server. ValueError is
|
# "Error executing tool X" and its message is kept on the server. ValueError is
|
||||||
# how every Scribe tool refuses, so for weeks every refusal arrived bare.
|
# how every Scribe tool refuses, so without this every refusal arrives bare.
|
||||||
|
|
||||||
|
|
||||||
def _raising_server() -> StrictArgsMCPServer:
|
def _raising_server() -> StrictArgsMCPServer:
|
||||||
@@ -112,7 +111,7 @@ async def test_a_real_tools_refusal_reaches_the_caller():
|
|||||||
|
|
||||||
|
|
||||||
async def test_the_original_regression_create_note_with_content():
|
async def test_the_original_regression_create_note_with_content():
|
||||||
"""Pin #2709 itself against the REAL server: `content=` on create_note
|
"""Against the REAL server: `content=` on create_note
|
||||||
must raise before dispatch — naming the bad argument and listing `body`
|
must raise before dispatch — naming the bad argument and listing `body`
|
||||||
among the accepted ones — instead of creating a body-less note."""
|
among the accepted ones — instead of creating a body-less note."""
|
||||||
from scribe.mcp.server import build_mcp_server
|
from scribe.mcp.server import build_mcp_server
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Delivering mounted rules when their moment happens (milestone 458 step 4).
|
"""Delivering mounted rules when their moment happens.
|
||||||
|
|
||||||
The database is stubbed: the lookup's scoping is pinned against Postgres in
|
The database is stubbed: the lookup's scoping is pinned against Postgres in
|
||||||
tests/test_integration_rule_moments.py. These pin what is decided above it:
|
tests/test_integration_rule_moments.py. These pin what is decided above it:
|
||||||
@@ -186,7 +186,7 @@ async def _reachable(mounted, mappings=()):
|
|||||||
|
|
||||||
|
|
||||||
# The tools the shipped defaults map onto a moment that carries a default reply
|
# The tools the shipped defaults map onto a moment that carries a default reply
|
||||||
# shape (milestone 500): closing work, a structured question, opening a plan.
|
# shape: closing work, a structured question, opening a plan.
|
||||||
SHAPED = ["askuserquestion", "create_milestone", "enterplanmode", "exitplanmode",
|
SHAPED = ["askuserquestion", "create_milestone", "enterplanmode", "exitplanmode",
|
||||||
"skill", "start_planning", "update_milestone", "update_task"]
|
"skill", "start_planning", "update_milestone", "update_task"]
|
||||||
|
|
||||||
@@ -273,7 +273,7 @@ def test_the_hook_is_registered_on_every_tool():
|
|||||||
|
|
||||||
|
|
||||||
def test_the_hook_and_the_routes_agree_on_every_name():
|
def test_the_hook_and_the_routes_agree_on_every_name():
|
||||||
"""Rule 33 across the shell/Python seam: a renamed arg fails silently."""
|
"""Parity across the shell/Python seam: a renamed arg fails silently."""
|
||||||
src = HOOK.read_text()
|
src = HOOK.read_text()
|
||||||
assert "/api/plugin/moment-tools" in src and "/api/plugin/moment" in src
|
assert "/api/plugin/moment-tools" in src and "/api/plugin/moment" in src
|
||||||
assert "exclude_rule_ids" in src and "scribe_held_query" in src
|
assert "exclude_rule_ids" in src and "scribe_held_query" in src
|
||||||
@@ -391,7 +391,7 @@ def test_every_scribe_tool_the_defaults_name_attaches_its_moment_rules():
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# ── reply shapes ride the moments (milestone 500 step 3) ────────────────
|
# ── reply shapes ride the moments ───────────────────────────────────────
|
||||||
|
|
||||||
from scribe.services import reply_shapes # noqa: E402
|
from scribe.services import reply_shapes # noqa: E402
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""The prompt boundary retrieves against PROMPTS, not against plumbing (#4200).
|
"""The prompt boundary retrieves against PROMPTS, not against plumbing.
|
||||||
|
|
||||||
Claude Code submits more than typed words through `UserPromptSubmit`. A task
|
Claude Code submits more than typed words through `UserPromptSubmit`. A task
|
||||||
notification, the echo of a slash command and the caveat banner a local command
|
notification, the echo of a slash command and the caveat banner a local command
|
||||||
@@ -10,11 +10,11 @@ never returns anything. The cost that matters is not the wasted embedding, it
|
|||||||
is the LOG ROW. Every one of those calls counts in the denominator of every
|
is the LOG ROW. Every one of those calls counts in the denominator of every
|
||||||
prompt-boundary surface, so delivery rate reads low for a reason unrelated to
|
prompt-boundary surface, so delivery rate reads low for a reason unrelated to
|
||||||
retrieval quality; and each refusal lands in `near_misses`, where a later
|
retrieval quality; and each refusal lands in `near_misses`, where a later
|
||||||
tuning decision reads it as demand. Measured on #3898: 15 of the top 20
|
tuning decision reads it as demand. Notification blocks can fill most of a
|
||||||
preference near-misses were `<task-notification>` blocks, all matching one
|
preference arm's top near-misses, all matching one record, all within
|
||||||
record, all within thousandths of the floor. A floor lowered to serve that
|
thousandths of the floor. A floor lowered to serve that apparent demand would
|
||||||
apparent demand would have injected that record into every notification — an
|
inject that record into every notification — an instrument arguing for the
|
||||||
instrument arguing for the wrong fix, which is exactly #379.
|
wrong fix.
|
||||||
|
|
||||||
So the assertions split in two, and the SECOND half is the one guarding
|
So the assertions split in two, and the SECOND half is the one guarding
|
||||||
against the obvious wrong implementation:
|
against the obvious wrong implementation:
|
||||||
@@ -52,7 +52,7 @@ SYNTHETIC = [
|
|||||||
"<local-command-stderr>Failed to reconnect</local-command-stderr>",
|
"<local-command-stderr>Failed to reconnect</local-command-stderr>",
|
||||||
"<local-command-caveat>Caveat: The messages below were generated…</local-command-caveat>",
|
"<local-command-caveat>Caveat: The messages below were generated…</local-command-caveat>",
|
||||||
" \n<task-notification>\n<task-id>abc</task-id>", # whitespace must not walk the filter
|
" \n<task-notification>\n<task-id>abc</task-id>", # whitespace must not walk the filter
|
||||||
# A subagent's hand-back (#4785) — it carries attributes, so the tag is
|
# A subagent's hand-back — it carries attributes, so the tag is
|
||||||
# matched up to its name, not to a closing `>`.
|
# matched up to its name, not to a closing `>`.
|
||||||
'<agent-message from="a93b1b587d7a718b0">\n[Subagent hand-back] The text below is the final report',
|
'<agent-message from="a93b1b587d7a718b0">\n[Subagent hand-back] The text below is the final report',
|
||||||
"<agent-message>\nreport</agent-message>",
|
"<agent-message>\nreport</agent-message>",
|
||||||
|
|||||||
@@ -1,20 +1,19 @@
|
|||||||
"""Every typed record kind gets the same doors (#4164).
|
"""Every typed record kind gets the same doors.
|
||||||
|
|
||||||
WHY THIS IS ONE GUARD AND NOT FIFTEEN
|
WHY THIS IS ONE GUARD AND NOT FIFTEEN
|
||||||
|
|
||||||
Milestone 385 shipped `lesson` across four steps and it still arrived with
|
A kind can ship across several steps and still arrive with three of five
|
||||||
three of five tools, no way to list it, and no duplicate report. Nothing was
|
tools, no way to list it, and no duplicate report — with nothing red, because
|
||||||
red, because every test asked "does what I built work" and none asked "is the
|
every test asks "does what I built work" and none asks "is the kind
|
||||||
kind finished". A per-tool test cannot catch a MISSING tool.
|
finished". A per-tool test cannot catch a MISSING tool.
|
||||||
|
|
||||||
So this asserts the property directly — a kind with its own `note_type` has a
|
So this asserts the property directly — a kind with its own `note_type` has a
|
||||||
full create/read/update/delete plus a listing, each one registered and each one
|
full create/read/update/delete plus a listing, each one registered and each one
|
||||||
classified for auth — and derives the expectation from the kinds themselves. A
|
classified for auth — and derives the expectation from the kinds themselves. A
|
||||||
fourth kind added later inherits the bar without anyone remembering to.
|
fourth kind added later inherits the bar without anyone remembering to.
|
||||||
|
|
||||||
#2250 is the same failure one kind earlier: processes could always be deleted
|
A kind deletable only through `delete_note` is the same failure: "a kind whose
|
||||||
through `delete_note`, but nothing said so, and "a kind whose own tools offer
|
own tools offer create/read/update reads as one you cannot retire".
|
||||||
create/read/update reads as one you cannot retire".
|
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -103,7 +102,7 @@ def test_every_kind_can_be_asked_whether_it_holds_duplicates(singular):
|
|||||||
|
|
||||||
|
|
||||||
def test_the_surface_guards_can_fail():
|
def test_the_surface_guards_can_fail():
|
||||||
"""Rule 167: falsify the shape these assert against, so a guard that has
|
"""Falsify the shape these assert against, so a guard that has
|
||||||
quietly stopped describing anything cannot pass by describing nothing."""
|
quietly stopped describing anything cannot pass by describing nothing."""
|
||||||
assert _expected_tools("lesson", "lessons") == [
|
assert _expected_tools("lesson", "lessons") == [
|
||||||
"list_lessons", "create_lesson", "get_lesson",
|
"list_lessons", "create_lesson", "get_lesson",
|
||||||
|
|||||||
@@ -1,9 +1,8 @@
|
|||||||
"""The operator's half of the tuning surface (#4102).
|
"""The operator's half of the tuning surface.
|
||||||
|
|
||||||
WHY THIS EXISTS
|
WHY THIS EXISTS
|
||||||
|
|
||||||
The model moves these dials — that is the operator's decision for milestone
|
The model moves these dials, so the browser's job is not "set the number". It is to show what
|
||||||
416 — so the browser's job is no longer "set the number". It is to show what
|
|
||||||
was set, by whom, and on what argument, and to let the operator disagree.
|
was set, by whom, and on what argument, and to let the operator disagree.
|
||||||
|
|
||||||
The load-bearing guard here is the last one. `/api/settings` is a generic
|
The load-bearing guard here is the last one. `/api/settings` is a generic
|
||||||
@@ -51,7 +50,7 @@ def test_every_endpoint_is_reachable_on_the_app():
|
|||||||
|
|
||||||
|
|
||||||
def test_the_browser_and_the_agent_call_the_same_service():
|
def test_the_browser_and_the_agent_call_the_same_service():
|
||||||
"""Rule 33 parity. Two callers, one service — or the two surfaces drift and
|
"""Parity. Two callers, one service — or the two surfaces drift and
|
||||||
the guard that a reason is required exists on only one of them."""
|
the guard that a reason is required exists on only one of them."""
|
||||||
from scribe.mcp.tools import retrieval_tuning as tool
|
from scribe.mcp.tools import retrieval_tuning as tool
|
||||||
from scribe.routes import retrieval as routes
|
from scribe.routes import retrieval as routes
|
||||||
@@ -138,7 +137,7 @@ async def test_setting_a_dial_to_the_value_it_already_has_records_nothing():
|
|||||||
setter.assert_not_called()
|
setter.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
# ── the moments view (milestone 458 step 6) ─────────────────────────────
|
# ── the moments view ────────────────────────────────────────────────────
|
||||||
|
|
||||||
async def _moments_call(handler_name, path, method="GET", **kw):
|
async def _moments_call(handler_name, path, method="GET", **kw):
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ def test_create_project_rule_route_exists():
|
|||||||
|
|
||||||
|
|
||||||
def test_subscriptions_and_suppressions_are_gone():
|
def test_subscriptions_and_suppressions_are_gone():
|
||||||
"""Milestone 414: a rule's home is its scope. Nothing subscribes a project
|
"""A rule's home is its scope. Nothing subscribes a project
|
||||||
to a rulebook or mutes a rule for one — not a route, a service function or
|
to a rulebook or mutes a rule for one — not a route, a service function or
|
||||||
a table. A partial removal would leave a door that 500s on a missing table.
|
a table. A partial removal would leave a door that 500s on a missing table.
|
||||||
"""
|
"""
|
||||||
@@ -94,14 +94,14 @@ def test_rule_handlers_callable():
|
|||||||
for name in (
|
for name in (
|
||||||
"list_rules", "create_rule", "get_rule", "update_rule", "delete_rule",
|
"list_rules", "create_rule", "get_rule", "update_rule", "delete_rule",
|
||||||
"get_project_rules", "move_rule",
|
"get_project_rules", "move_rule",
|
||||||
# The typed edges — both doors carry them (rule 33).
|
# The typed edges — both doors carry them.
|
||||||
"relate_rules", "unrelate_rules",
|
"relate_rules", "unrelate_rules",
|
||||||
):
|
):
|
||||||
assert callable(getattr(rb_routes, name))
|
assert callable(getattr(rb_routes, name))
|
||||||
|
|
||||||
|
|
||||||
def test_the_rule_list_zero_fills_usage_on_every_row():
|
def test_the_rule_list_zero_fills_usage_on_every_row():
|
||||||
"""Milestone 333 step 5, asserted the only way this harness allows.
|
"""Asserted the only way this harness allows.
|
||||||
|
|
||||||
There is no live-HTTP fixture here (see this module's docstring), so this
|
There is no live-HTTP fixture here (see this module's docstring), so this
|
||||||
reads the handler's source. What it can still prove is the property that
|
reads the handler's source. What it can still prove is the property that
|
||||||
@@ -110,7 +110,7 @@ def test_the_rule_list_zero_fills_usage_on_every_row():
|
|||||||
existing install predates `rule_usage_events`, so a route that only
|
existing install predates `rule_usage_events`, so a route that only
|
||||||
attached the key when it found something would leave the badge component
|
attached the key when it found something would leave the badge component
|
||||||
reading `undefined` on almost every row — and the difference between "no
|
reading `undefined` on almost every row — and the difference between "no
|
||||||
events" and "no field" is exactly the distinction #2663 is about.
|
events" and "no field" is exactly the distinction a readout must keep.
|
||||||
"""
|
"""
|
||||||
import inspect
|
import inspect
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Every bundled skill keeps the shape Agent Skills can load well (#4398).
|
"""Every bundled skill keeps the shape Agent Skills can load well.
|
||||||
|
|
||||||
WHY THIS EXISTS
|
WHY THIS EXISTS
|
||||||
|
|
||||||
@@ -8,11 +8,10 @@ keep the SKILL.md body under 500 lines, split into reference files as it
|
|||||||
nears that, and link each reference file ONE level deep from SKILL.md —
|
nears that, and link each reference file ONE level deep from SKILL.md —
|
||||||
Claude may only preview a file reached through another reference file.
|
Claude may only preview a file reached through another reference file.
|
||||||
|
|
||||||
using-scribe reached 478 lines because every new practice lands there, the
|
using-scribe grows because every new practice lands there. What matters on
|
||||||
same squeeze `_INSTRUCTIONS` was in before #4389. It was split: what matters
|
most turns stays in SKILL.md; what matters at one moment (placing a rule, a
|
||||||
on most turns stays in SKILL.md, what matters at one moment (placing a rule,
|
missed retrieval, starting a project) lives in files SKILL.md names with the
|
||||||
a missed retrieval, starting a project) moved to files SKILL.md names with
|
moment to open them. These tests keep that shape from quietly undoing
|
||||||
the moment to open them. These tests keep that shape from quietly undoing
|
|
||||||
itself.
|
itself.
|
||||||
|
|
||||||
WHAT IT PINS
|
WHAT IT PINS
|
||||||
@@ -71,8 +70,8 @@ def test_every_skill_md_fits_its_budget():
|
|||||||
SKILL_LINE_BUDGET)
|
SKILL_LINE_BUDGET)
|
||||||
assert not found, (
|
assert not found, (
|
||||||
f"SKILL.md over {SKILL_LINE_BUDGET} lines: {found}. Move what matters at "
|
f"SKILL.md over {SKILL_LINE_BUDGET} lines: {found}. Move what matters at "
|
||||||
f"one moment into a reference file SKILL.md links with that moment "
|
f"one moment into a reference file SKILL.md links with that moment, "
|
||||||
f"(#4398), rather than raising the budget toward 500."
|
f"rather than raising the budget toward 500."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -102,7 +101,7 @@ def test_long_reference_files_open_with_contents():
|
|||||||
|
|
||||||
|
|
||||||
def test_the_guards_can_fail():
|
def test_the_guards_can_fail():
|
||||||
"""Rule 167: each guard bites on the failure it exists for."""
|
"""Each guard bites on the failure it exists for."""
|
||||||
assert over_budget({"s": "x\n" * 400}, SKILL_LINE_BUDGET) == ["s: 400 lines"]
|
assert over_budget({"s": "x\n" * 400}, SKILL_LINE_BUDGET) == ["s: 400 lines"]
|
||||||
assert unlinked("see [a](a.md)", ["a.md", "b.md"]) == ["b.md"]
|
assert unlinked("see [a](a.md)", ["a.md", "b.md"]) == ["b.md"]
|
||||||
assert nested({"a.md": "then [b](b.md)"}) == ["a.md → b.md"]
|
assert nested({"a.md": "then [b](b.md)"}) == ["a.md → b.md"]
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Tests for the near-duplicate finder (#2088).
|
"""Tests for the near-duplicate finder.
|
||||||
|
|
||||||
The SQL half needs a database and lives in the integration lane; what's covered
|
The SQL half needs a database and lives in the integration lane; what's covered
|
||||||
here is the grouping rule — which is where the interesting decisions are — plus
|
here is the grouping rule — which is where the interesting decisions are — plus
|
||||||
@@ -98,13 +98,13 @@ def test_the_snippet_report_threshold_is_looser_than_the_write_gate():
|
|||||||
report only suggests a merge the operator reviews, so it has to reach
|
report only suggests a merge the operator reviews, so it has to reach
|
||||||
further or it would never surface the pairs the gate already let through.
|
further or it would never surface the pairs the gate already let through.
|
||||||
Snippet-only on purpose: note/task floors sit ABOVE the gate because chunk
|
Snippet-only on purpose: note/task floors sit ABOVE the gate because chunk
|
||||||
grain (#280) lifts related families over it — see the module comment."""
|
grain lifts related families over it — see the module comment."""
|
||||||
assert DUPLICATE_DEFAULT_THRESHOLDS["snippet"] < dedup_svc._SEMANTIC_THRESHOLD
|
assert DUPLICATE_DEFAULT_THRESHOLDS["snippet"] < dedup_svc._SEMANTIC_THRESHOLD
|
||||||
|
|
||||||
|
|
||||||
def test_the_note_and_task_floors_are_stricter_than_the_snippet_floor():
|
def test_the_note_and_task_floors_are_stricter_than_the_snippet_floor():
|
||||||
"""At chunk grain, 0.82 is a related-families view; the report's default
|
"""At chunk grain, 0.82 is a related-families view; the report's default
|
||||||
must point at genuinely-alike records (measured 2026-08-09, note #2551)."""
|
must point at genuinely-alike records."""
|
||||||
assert DUPLICATE_DEFAULT_THRESHOLDS["note"] == 0.93
|
assert DUPLICATE_DEFAULT_THRESHOLDS["note"] == 0.93
|
||||||
assert DUPLICATE_DEFAULT_THRESHOLDS["task"] == 0.93
|
assert DUPLICATE_DEFAULT_THRESHOLDS["task"] == 0.93
|
||||||
|
|
||||||
@@ -119,7 +119,7 @@ async def test_a_failed_scan_returns_an_empty_report_not_an_error():
|
|||||||
patch.object(dedup_svc, "async_session", side_effect=RuntimeError("boom")),
|
patch.object(dedup_svc, "async_session", side_effect=RuntimeError("boom")),
|
||||||
):
|
):
|
||||||
out = await dedup_svc.find_duplicate_snippets(1)
|
out = await dedup_svc.find_duplicate_snippets(1)
|
||||||
# The empty report still carries its per-kind `suggestion` (#2547) — a
|
# The empty report still carries its per-kind `suggestion` — a
|
||||||
# failed scan must be indistinguishable in SHAPE from a clean one, or every
|
# failed scan must be indistinguishable in SHAPE from a clean one, or every
|
||||||
# consumer needs a second code path for the degraded case.
|
# consumer needs a second code path for the degraded case.
|
||||||
assert out["groups"] == [] and out["pairs"] == []
|
assert out["groups"] == [] and out["pairs"] == []
|
||||||
|
|||||||
@@ -1,10 +1,8 @@
|
|||||||
"""The work log is READABLE from the tools an agent opens a task with (#4241).
|
"""The work log is READABLE from the tools an agent opens a task with.
|
||||||
|
|
||||||
`add_task_log` wrote to a surface no agent could read back: the entries
|
A session that opens a task and sees only the body sees a claim written once,
|
||||||
reached the web UI through routes/task_logs.py and nothing else. So a session
|
before the work, with the record written during it invisible beside it. A
|
||||||
opening a task saw only the body — a claim written once, before the work —
|
stale body has nothing to contradict it, and shipped work gets rebuilt.
|
||||||
with the record written during it invisible beside it. A stale body had
|
|
||||||
nothing to contradict it, and shipped work got rebuilt.
|
|
||||||
|
|
||||||
These pin the read, not the write. The payload shape is tested against the
|
These pin the read, not the write. The payload shape is tested against the
|
||||||
real `work_log_payload`; the three tools are tested by re-patching the arm
|
real `work_log_payload`; the three tools are tested by re-patching the arm
|
||||||
@@ -235,7 +233,7 @@ async def test_log_limit_zero_asks_for_every_entry_and_skips_the_count_query():
|
|||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_get_task_docstring_tells_a_reader_the_log_outranks_the_body():
|
async def test_get_task_docstring_tells_a_reader_the_log_outranks_the_body():
|
||||||
"""#2846: the docstring IS the agent-facing contract. A `work_log` key
|
"""The docstring IS the agent-facing contract. A `work_log` key
|
||||||
that nothing tells the reader to prefer over a stale body is the same
|
that nothing tells the reader to prefer over a stale body is the same
|
||||||
failure one layer up."""
|
failure one layer up."""
|
||||||
doc = (get_task.__doc__ or "").lower()
|
doc = (get_task.__doc__ or "").lower()
|
||||||
@@ -305,7 +303,7 @@ def test_brief_row_without_a_mapping_is_unchanged():
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# The service read — scoped by who may read the TASK (rule #78)
|
# The service read — scoped by who may read the TASK
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
@@ -332,7 +330,7 @@ async def test_logs_for_task_is_not_filtered_by_who_wrote_the_entry():
|
|||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_logs_for_task_refuses_a_task_the_caller_cannot_read():
|
async def test_logs_for_task_refuses_a_task_the_caller_cannot_read():
|
||||||
"""Unscoped would have been the mirror-image hole: rule #78 is about
|
"""Unscoped would be the mirror-image hole: access control is about
|
||||||
routing the question through the access layer, in both directions."""
|
routing the question through the access layer, in both directions."""
|
||||||
opened = MagicMock()
|
opened = MagicMock()
|
||||||
with patch("scribe.services.task_logs.can_read_note",
|
with patch("scribe.services.task_logs.can_read_note",
|
||||||
|
|||||||
@@ -1,19 +1,14 @@
|
|||||||
"""One seam attaches `usage`, and every door that shows it uses that seam (#4230).
|
"""One seam attaches `usage`, and every door that shows it uses that seam.
|
||||||
|
|
||||||
WHAT WENT WRONG. `usage_for_notes` is named for notes and works on every note
|
WHY. `usage_for_notes` is named for notes and works on every note row, and
|
||||||
row. Yet the surfaced-vs-opened chip reached snippets and rules only: notes had
|
seven doors show its result — two REST lists, two REST details, two MCP
|
||||||
it nowhere, and lessons had it collected but shown nowhere a person could
|
lists, one MCP detail. Seven copies of the same few lines each read perfectly
|
||||||
reach, because the only lesson LIST in the UI is the Knowledge browse and that
|
well on their own, and nobody compares them, so "which doors attach usage?"
|
||||||
route never attached it.
|
has no answer anywhere in the code. That is the same failure
|
||||||
|
`test_system_tagging_door_parity.py` records for System tagging: whichever
|
||||||
The cause was not any one missing line. Seven call sites carried their own copy
|
door nobody exercises for a kind is the one that
|
||||||
of the same few lines — two REST lists, two REST details, two MCP lists, one
|
never grows the feature, and a human reviewer does not reliably catch it
|
||||||
MCP detail — and each read perfectly well on its own. Nobody was comparing
|
because each door is only ever read alone.
|
||||||
them, so "which doors attach usage?" had no answer anywhere in the code. That
|
|
||||||
is the same failure `test_system_tagging_door_parity.py` records for System
|
|
||||||
tagging (#4249): whichever door nobody exercised for a kind is the one that
|
|
||||||
never grew the feature, and a human reviewer does not reliably catch it because
|
|
||||||
each door is only ever read alone.
|
|
||||||
|
|
||||||
So this file asserts the PROPERTY, not the behaviour of one route: the attach
|
So this file asserts the PROPERTY, not the behaviour of one route: the attach
|
||||||
logic exists once, and no door re-implements it. A kind added next month either
|
logic exists once, and no door re-implements it. A kind added next month either
|
||||||
@@ -156,7 +151,7 @@ def test_no_door_calls_the_aggregate_directly() -> None:
|
|||||||
|
|
||||||
Keyed on the CALL, not on the text, so a module that merely names the
|
Keyed on the CALL, not on the text, so a module that merely names the
|
||||||
function in a comment explaining the seam is not a false positive — and a
|
function in a comment explaining the seam is not a false positive — and a
|
||||||
hand-kept skip list, which would itself go stale, is not needed (rule 167).
|
hand-kept skip list, which would itself go stale, is not needed.
|
||||||
"""
|
"""
|
||||||
offenders = []
|
offenders = []
|
||||||
for path in _door_modules():
|
for path in _door_modules():
|
||||||
|
|||||||
Reference in New Issue
Block a user