docs(6209): rule usage, write path and telemetry tests state the behaviour they pin
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m18s
CI & Build / Python tests (push) Successful in 2m2s
CI & Build / Build & push image (push) Successful in 17s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / integration (push) Successful in 1m18s
CI & Build / Python tests (push) Successful in 2m2s
CI & Build / Build & push image (push) Successful in 17s
Docstrings, comments and assertion messages in the three most-cited test files now say which property each test holds and why, without the issue, milestone or rule numbers, dates and one-install measurements that recorded how each property was discovered. Fixture ids and calibration scores stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+106
-127
@@ -1,7 +1,7 @@
|
||||
"""Tests for the write-path trigger (#2082) — prior art at the moment code is written.
|
||||
"""Tests for the write-path trigger — prior art at the moment code is written.
|
||||
|
||||
Covers the service (`build_write_path_hint`): the two arms and their precedence,
|
||||
the anti-bloat gates carried over from milestone 93, telemetry under its own
|
||||
the anti-bloat gates, telemetry under its own
|
||||
source, and the two ways this must stay silent. Plus the plugin hook contract —
|
||||
a PreToolUse hook that returns a permission decision would be able to block the
|
||||
operator's edit, which this feature must never do.
|
||||
@@ -19,7 +19,7 @@ from tests.helpers import fake_note, http_sink, writepath_cfg
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _stored_settings(stored):
|
||||
"""Patch BOTH readers, because one call now uses two (#4102).
|
||||
"""Patch BOTH readers, because one call uses two.
|
||||
|
||||
`get_writepath_config` reads its `enabled` switch through
|
||||
`plugin_context.get_setting` and all six tunable numbers through
|
||||
@@ -49,8 +49,7 @@ def _snippet_item(nid, title, user_id=1):
|
||||
|
||||
|
||||
def _cfg(**over):
|
||||
# `rule_threshold` is the standing-rule arm's own bar (milestone 333 step
|
||||
# 4). It belongs in the stand-in even though most tests here never reach
|
||||
# `rule_threshold` is the standing-rule arm's own bar. It belongs in the stand-in even though most tests here never reach
|
||||
# that arm: the arm reads it while BUILDING its search arguments, so a
|
||||
# missing key raises inside its fail-open except and turns the arm into a
|
||||
# silent no-op — which is indistinguishable from it working and finding
|
||||
@@ -59,10 +58,9 @@ def _cfg(**over):
|
||||
|
||||
|
||||
# The semantic arm ignores payloads carrying less than WRITEPATH_MIN_CODE_CHARS
|
||||
# of non-whitespace substance (#2223), so any test that expects it to RUN has to
|
||||
# pass something a real Edit would plausibly contain. `code="x"` used to be the
|
||||
# fixture here, which meant the gate tests were passing payloads that in
|
||||
# production are exactly the noise the floor now drops.
|
||||
# of non-whitespace substance, so any test that expects it to RUN has to pass
|
||||
# something a real Edit would plausibly contain. A payload like `code="x"` is
|
||||
# exactly the noise the floor drops.
|
||||
REAL_CODE = '''def debounce(fn, wait=0.25):
|
||||
"""Rate-limit a callback so it fires once after the last call."""
|
||||
timer = None
|
||||
@@ -121,7 +119,7 @@ async def test_nothing_recorded_is_silent():
|
||||
@pytest.mark.asyncio
|
||||
async def test_snippet_at_this_path_becomes_the_sync_nudge():
|
||||
"""A snippet recorded AT the exact file is not a reuse suggestion — it IS
|
||||
the record of the file being edited (#2708). It surfaces without a score,
|
||||
the record of the file being edited. It surfaces without a score,
|
||||
framed as "updating the record is part of this edit", and is reported in
|
||||
sync_note_ids so the hook feeds the sync dedup channel."""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -143,8 +141,8 @@ async def test_snippet_at_this_path_becomes_the_sync_nudge():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_reuse_dedup_never_suppresses_the_sync_nudge():
|
||||
"""THE bug #2708 names: a title surfaced as a reuse hint twenty turns ago
|
||||
landed in exclude_ids — and then silenced "you are editing the recorded
|
||||
"""A title surfaced as a reuse hint twenty turns ago lands in exclude_ids,
|
||||
and must not silence "you are editing the recorded
|
||||
file right now", the one claim that must fire at the moment of change. The
|
||||
sync class dedups only against its own channel."""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -252,9 +250,9 @@ async def test_semantic_arm_is_snippet_only_and_browse_scoped():
|
||||
patch.object(pc, "record_retrieval", MagicMock()):
|
||||
await pc.build_write_path_hint(1, "src/x.py", code=REAL_CODE)
|
||||
kwargs = search.await_args.kwargs
|
||||
# Prior art is snippets AND recorded experience (#2246) — an issue saying
|
||||
# "we tried this and it broke" belongs here — AND lessons (milestone 385
|
||||
# step 5), whose founding example is a lesson about a code shape. What
|
||||
# Prior art is snippets AND recorded experience — an issue saying "we tried
|
||||
# this and it broke" belongs here — AND lessons, which are often about a
|
||||
# code shape. What
|
||||
# stays out is the open to-do list, which resembles the code and answers
|
||||
# nothing.
|
||||
assert kwargs["note_type"] == ("snippet", "note", "lesson")
|
||||
@@ -338,11 +336,10 @@ async def test_session_dedup_marks_the_reuse_arms_rather_than_silencing_them():
|
||||
"""exclude_ids governs the REUSE classes — nearby and semantic. (The sync
|
||||
class has its own channel; see the tests above.)
|
||||
|
||||
WHAT CHANGED AND WHY (#4101). This used to assert the opposite: the nearby
|
||||
hit was dropped and its id pushed into the search's `exclude_ids`. That was
|
||||
#3750's defect on the note arms — the second time a record was the best
|
||||
answer, the session got silence, which reads exactly like "nothing is
|
||||
recorded here". Now the repeat is rendered with a `seen` marker, and the
|
||||
WHY A MARKER AND NOT SILENCE. Dropping the hit and pushing its id into the
|
||||
search's `exclude_ids` means that the second time a record is the best
|
||||
answer, the session gets silence, which reads exactly like "nothing is
|
||||
recorded here". So the repeat is rendered with a `seen` marker, and the
|
||||
ledger never reaches the query, so the score the search reports describes
|
||||
the bar rather than a candidate set the caller had already edited.
|
||||
|
||||
@@ -381,7 +378,7 @@ async def test_session_dedup_marks_the_reuse_arms_rather_than_silencing_them():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_another_users_snippet_is_attributed():
|
||||
"""Rule #47 / the #2084 lesson: an unattributed line reads as your own record."""
|
||||
"""An unattributed line reads as your own record."""
|
||||
from scribe.services import plugin_context as pc
|
||||
with patch.object(pc, "get_writepath_config", AsyncMock(return_value=_cfg())), \
|
||||
patch.object(pc.snippets_svc, "list_snippets",
|
||||
@@ -413,15 +410,15 @@ async def test_telemetry_uses_its_own_source():
|
||||
assert note_arm.kwargs["project_id"] == 4
|
||||
|
||||
# The rule arm rides along on the same hint and logs its own call even when
|
||||
# it finds nothing (#3497). This assertion used to be `assert_called_once`,
|
||||
# which passed only because that row was never written — the test encoded
|
||||
# the defect. The second row is the point of having two sources.
|
||||
# it finds nothing. An `assert_called_once` here would pass only if that
|
||||
# row were never written. The second row is the point of having two
|
||||
# sources.
|
||||
assert "write_path_rule" in sources, sources
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sync_surfacing_is_measured_under_its_own_usage_source():
|
||||
"""The sync class's pull-through rate is #2708's scoreboard — whether
|
||||
"""The sync class's pull-through rate is its scoreboard — whether
|
||||
sessions actually update the record when told they're editing it. Folding
|
||||
it into write_path_place would make that unmeasurable."""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -455,12 +452,12 @@ async def test_config_has_its_own_switch_threshold_and_inherited_budget():
|
||||
# Its own switch is off while auto-inject stays on...
|
||||
assert cfg["enabled"] is False
|
||||
# ...its threshold is its OWN and stricter than auto-inject's, because code
|
||||
# embeddings have a higher similarity floor than prose (#2223). Sharing 0.55
|
||||
# made unrelated code — including `x = 1` at 0.58 — clear the bar.
|
||||
# embeddings have a higher similarity floor than prose. At 0.55, unrelated
|
||||
# code — including `x = 1` at 0.58 — clears the bar.
|
||||
assert cfg["threshold"] == pc.WRITEPATH_DEFAULT_THRESHOLD
|
||||
assert cfg["threshold"] > pc.AUTOINJECT_DEFAULT_THRESHOLD
|
||||
# ...and the budget is INHERITED rather than shared (#4102). This arm has
|
||||
# its own key now, because it fires before every Write and Edit while
|
||||
# ...and the budget is INHERITED rather than shared. This arm has its own
|
||||
# key, because it fires before every Write and Edit while
|
||||
# auto-inject fires once a turn, so the same number buys very different
|
||||
# amounts of attention. Unset, it still reads auto-inject's — which is what
|
||||
# stops the split from silently resetting an install that had tuned the
|
||||
@@ -470,7 +467,7 @@ async def test_config_has_its_own_switch_threshold_and_inherited_budget():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_writepath_threshold_is_operator_tunable_and_clamped():
|
||||
"""Rule #25 — it's a product knob, so it has to be settable, and a typo in
|
||||
"""It's a product knob, so it has to be settable, and a typo in
|
||||
the settings box must not be able to open the gate to everything."""
|
||||
from scribe.services import plugin_context as pc
|
||||
|
||||
@@ -489,11 +486,11 @@ async def test_writepath_threshold_is_operator_tunable_and_clamped():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_rule_arm_has_its_own_tunable_bar():
|
||||
"""Rule #25 again, for the THIRD corpus (milestone 333 step 4).
|
||||
"""Settable and clamped again, for the THIRD corpus.
|
||||
|
||||
Separate from the code threshold above and separately settable, because the
|
||||
two are measured against different things: 0.68 was derived from code
|
||||
against note PROSE (#2223), and rules are short imperative technical
|
||||
against note PROSE, and rules are short imperative technical
|
||||
English — a more homogeneous corpus whose noise floor sits higher.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -528,12 +525,10 @@ async def test_the_two_write_path_bars_are_independent():
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_two_act_arms_read_independent_rule_bars():
|
||||
"""#3853's split: the command arm's bar moves without the write path's.
|
||||
"""The command arm's bar moves without the write path's.
|
||||
|
||||
The two act arms shared one key until the telemetry showed them behaving
|
||||
like different subsystems at the same number — the write-path arm speaking
|
||||
on 37% of calls against the command arm's 2%, because a code payload is
|
||||
long and rich where a shell command is short. A config assembler that
|
||||
The two act arms behave like different subsystems at the same number,
|
||||
because a code payload is long and rich where a shell command is short. A config assembler that
|
||||
reads one key into both fields would silently undo that, and the symptom
|
||||
would be invisible: both arms would simply agree again.
|
||||
"""
|
||||
@@ -553,7 +548,7 @@ async def test_a_garbage_command_bar_falls_back_to_its_own_default():
|
||||
|
||||
Falling back to 0.0 would attach a rule to every Bash call in the session;
|
||||
falling back to the sibling's default would quietly re-merge the two bars
|
||||
that #3853 separated, which is the harder failure to see because the arm
|
||||
that are kept separate, which is the harder failure to see because the arm
|
||||
keeps working.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -566,15 +561,13 @@ async def test_a_garbage_command_bar_falls_back_to_its_own_default():
|
||||
|
||||
|
||||
def test_the_command_bar_defaults_below_the_write_path_bar():
|
||||
"""A DIRECTION check, like its sibling above, and for the same rule-115
|
||||
reason: the value is measured against one corpus, the relationship is not.
|
||||
"""A DIRECTION check, like its sibling above, and for the same reason: the
|
||||
value is measured against one corpus, the relationship is not.
|
||||
|
||||
A shell command carries less text than a code payload and therefore scores
|
||||
lower for the same relevance — measured at #3853, where three of four
|
||||
consequential commands retrieved nothing at the shared bar while the
|
||||
write-path arm was healthy at it. Tuning either value stays free; inverting
|
||||
the relationship would reinstate the mute arm that spoke on 2% of 11,768
|
||||
calls.
|
||||
lower for the same relevance; at the write path's bar, consequential
|
||||
commands retrieve nothing. Tuning either value stays free; inverting the
|
||||
relationship would mute the command arm.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
|
||||
@@ -583,7 +576,7 @@ def test_the_command_bar_defaults_below_the_write_path_bar():
|
||||
|
||||
def test_the_rule_bar_defaults_above_the_code_bar():
|
||||
"""Not a number check — a DIRECTION check, and the only part of the default
|
||||
that is defensible without one instance's histogram (rule 115).
|
||||
that is defensible without one instance's histogram.
|
||||
|
||||
The eligible rule corpus is orders of magnitude smaller than the note
|
||||
corpus, so a top-k over it always returns something and a bar calibrated
|
||||
@@ -592,7 +585,7 @@ def test_the_rule_bar_defaults_above_the_code_bar():
|
||||
facts point the same way: this bar must sit ABOVE the one it inherited.
|
||||
|
||||
Pinned as an inequality so tuning the value stays free while inverting the
|
||||
relationship — which would silently reinstate #3311 — does not.
|
||||
relationship — which would silently drown the arm in noise — does not.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
|
||||
@@ -600,17 +593,12 @@ def test_the_rule_bar_defaults_above_the_code_bar():
|
||||
|
||||
|
||||
def test_the_rule_arm_asks_for_a_set_and_lets_the_band_narrow_it():
|
||||
"""WAS `..._asks_for_one_rule_not_two`, pinning `RULEHINT_LIMIT == 1`.
|
||||
"""A set, because this arm is the whole delivery: a `git push` governed by
|
||||
four rules cannot be served by one slot.
|
||||
|
||||
That guarded a real decision: with retrieval SUPPLEMENTING a 33-rule
|
||||
resident set, k=2 over a few dozen candidates made the second line the
|
||||
second-best noise wearing the first line's confident framing. Milestone
|
||||
394 removes residency, so this arm becomes the whole delivery and a
|
||||
`git push` governed by four rules cannot be served by one slot (#3851).
|
||||
|
||||
What replaces it is not simply a bigger k — that is the thing the old
|
||||
test was right to fear, because a fixed k fills its slots whether or not
|
||||
anything deserves them. The cap is a ceiling and the BAND is the control,
|
||||
Not simply a bigger k — a fixed k fills its slots whether or not anything
|
||||
deserves them, and the second line becomes the second-best noise wearing
|
||||
the first line's confident framing. The cap is a ceiling and the BAND is the control,
|
||||
so both must exist for the arm to be shaped as intended.
|
||||
|
||||
Pinned as relationships rather than values, like the threshold test above
|
||||
@@ -627,12 +615,12 @@ def test_the_rule_arm_asks_for_a_set_and_lets_the_band_narrow_it():
|
||||
assert 0 < pc._RULEHINT_BAND < rp._NOTE_BAND
|
||||
|
||||
|
||||
# --- the minimum-substance floor on the semantic arm (#2223) ------------------
|
||||
# --- the minimum-substance floor on the semantic arm --------------------------
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trivial_payload_skips_the_semantic_arm_entirely():
|
||||
"""`x = 1` scored 0.58 against an unrelated helper on the live instance — the
|
||||
embedder's floor for "some code", not a resemblance. A payload too small to
|
||||
"""`x = 1` scores around 0.58 against an unrelated helper — the embedder's
|
||||
floor for "some code", not a resemblance. A payload too small to
|
||||
carry meaning must not reach the embedder at all, and must not cost a
|
||||
retrieval_logs row either."""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -707,12 +695,13 @@ def test_the_floor_stays_below_the_smallest_plausible_helper():
|
||||
assert pc.WRITEPATH_MIN_CODE_CHARS > len("".join("x = 1".split()))
|
||||
|
||||
|
||||
# --- concept extraction: query by what the code IS FOR, not by the code (#2242) ---
|
||||
# --- concept extraction: query by what the code IS FOR, not by the code ---
|
||||
|
||||
def test_concept_query_prefers_the_name_and_docstring():
|
||||
"""Snippet documents are prose-forward — `when_to_use` appears in both the
|
||||
title and the body — so a concept query out-scores the code itself (0.823 vs
|
||||
0.743 measured against #2222). The shape mirrors a snippet's own title."""
|
||||
0.743 against a recorded snippet). The shape mirrors a snippet's own
|
||||
title."""
|
||||
from scribe.services.plugin_context import concept_query
|
||||
q = concept_query(
|
||||
'def collapse_into_clusters(edges):\n'
|
||||
@@ -753,7 +742,7 @@ def test_concept_query_does_not_let_a_shebang_become_the_description():
|
||||
|
||||
def test_concept_query_falls_back_when_there_is_no_doc():
|
||||
"""MEASURED RULE, not a taste call. An identifier alone scored 0.671 against
|
||||
#2222 where the full code body scored 0.743 — same separation from the noise
|
||||
a recorded snippet where the full code body scored 0.743 — same separation from the noise
|
||||
floor (0.113), but below the 0.68 bar, so preferring the bare name would turn
|
||||
a comfortable hit into a miss. Undocumented code keeps the raw payload."""
|
||||
from scribe.services.plugin_context import concept_query
|
||||
@@ -771,7 +760,7 @@ def test_concept_query_falls_back_on_a_doc_that_says_nothing():
|
||||
|
||||
def test_concept_query_falls_back_when_nothing_parses():
|
||||
"""A Vue SFC has neither a recognised declaration nor a doc block. Every
|
||||
unhandled language must degrade to exactly the pre-#2242 behaviour."""
|
||||
unhandled language must degrade to querying by the raw payload."""
|
||||
from scribe.services.plugin_context import concept_query
|
||||
assert concept_query(
|
||||
'<script setup lang="ts">\n'
|
||||
@@ -869,7 +858,7 @@ async def test_the_floor_judges_the_raw_payload_not_the_concept():
|
||||
search2.assert_not_called()
|
||||
|
||||
|
||||
# --- cross-language prior art is disclosed, not filtered (#2244) --------------
|
||||
# --- cross-language prior art is disclosed, not filtered ----------------------
|
||||
|
||||
def test_language_for_path_and_canonicalisation():
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -971,7 +960,7 @@ def test_prior_art_line_takes_a_search_hit_whose_snippet_is_its_excerpt():
|
||||
# --- the route between them --------------------------------------------------
|
||||
|
||||
def test_route_reads_every_arg_the_hook_sends():
|
||||
"""Rule #33: the hook's query string and the route's reader are one contract,
|
||||
"""The hook's query string and the route's reader are one contract,
|
||||
and the hook is the only caller — a rename on either side is silent."""
|
||||
import inspect
|
||||
|
||||
@@ -989,11 +978,10 @@ def test_route_reads_every_arg_the_hook_sends():
|
||||
for arg in ("path=", "code=", "exclude_ids=", "exclude_sync_ids=", "shapes="):
|
||||
assert arg in hook, f"hook never sends {arg}"
|
||||
|
||||
# The project-scope key is no longer spelled in the hook: since #4085 it is
|
||||
# whichever of `repo=` / `project_id=` scribe_scope_query picks, so the hook
|
||||
# The project-scope key is not spelled in the hook: it is whichever of `repo=` / `project_id=` scribe_scope_query picks, so the hook
|
||||
# splices in its output and the helper is the other half of the contract.
|
||||
assert "scribe_scope_query" in hook, "hook no longer asks for a project scope"
|
||||
# Same arrangement for the OPENED ledger (#4100): spelled once in the
|
||||
# Same arrangement for the OPENED ledger: spelled once in the
|
||||
# helper so its ageing and format cannot drift from the naming ledger's.
|
||||
assert "scribe_held_query" in hook, "hook no longer sends the opened ledger"
|
||||
defs = (HOOK.parent / "scribe_defs.sh").read_text()
|
||||
@@ -1003,7 +991,7 @@ def test_route_reads_every_arg_the_hook_sends():
|
||||
assert "printf '&held_rule_ids=" in defs, (
|
||||
"the opened ledger's query key is no longer spelled in the helper"
|
||||
)
|
||||
# The rulings ledger (milestone 444), shared with the tool-rules hook.
|
||||
# The rulings ledger, shared with the tool-rules hook.
|
||||
assert "scribe_rulings_query" in hook, "hook no longer sends the rulings ledger"
|
||||
assert "printf '&seen_ruling_systems=" in defs
|
||||
|
||||
@@ -1083,7 +1071,7 @@ def test_hook_exits_silently_when_unconfigured():
|
||||
|
||||
def test_hook_skips_prose_and_data_files():
|
||||
"""No round-trip for a markdown edit — the server would return nothing
|
||||
anyway. The list lives in the shared library (#2901) and the hook asks it."""
|
||||
anyway. The list lives in the shared library and the hook asks it."""
|
||||
lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
|
||||
skip = re.search(r"scribe_skip_path\(\) \{\n case \"\$1\" in\n(.*?)esac", lib, re.S)
|
||||
assert skip, "expected an extension skip list in scribe_defs.sh"
|
||||
@@ -1097,16 +1085,15 @@ def test_hook_skips_prose_and_data_files():
|
||||
|
||||
|
||||
def test_hook_keeps_sync_and_reuse_dedup_apart():
|
||||
"""#2708's dedup audit, pinned: the hook holds TWO per-session id files and
|
||||
feeds each its own class — sync ids (snippets recording the edited file) to
|
||||
the sync file, the rest to the reuse file. One shared file is exactly the
|
||||
bug this replaced: a reuse hint early in the session silencing the record-
|
||||
sync nudge when the recorded file is edited later."""
|
||||
"""The hook holds TWO per-session id files and feeds each its own class —
|
||||
sync ids (snippets recording the edited file) to the sync file, the rest
|
||||
to the reuse file. One shared file lets a reuse hint early in the session
|
||||
silence the record-sync nudge when the recorded file is edited later."""
|
||||
src = HOOK.read_text()
|
||||
assert ".sync.ids" in src # its own state file
|
||||
assert "exclude_sync_ids=" in src # its own query channel
|
||||
# The reuse file must NOT swallow sync ids — the write-back subtracts them.
|
||||
# Spelled with the readers that replaced jq (#4107); the claim is unchanged.
|
||||
# Spelled with the shell JSON readers in scribe_defs.sh.
|
||||
assert "scribe_json_list_minus \"$body_flat\" '.note_ids' '.sync_note_ids'" in src
|
||||
assert "scribe_json_list \"$body_flat\" '.sync_note_ids'" in src
|
||||
|
||||
@@ -1150,7 +1137,7 @@ def _write_event(repo, session="s-nudge"):
|
||||
|
||||
|
||||
def test_hook_nudges_recording_when_copies_exist_but_nothing_is_recorded(tmp_path):
|
||||
"""#2664: the local arm proves duplication; when Scribe ANSWERS that it has
|
||||
"""The local arm proves duplication; when Scribe ANSWERS that it has
|
||||
no record of it, the same context block must ask for create_snippet — the
|
||||
one moment the recording nudge is earned rather than noise."""
|
||||
with http_sink(b'{"context":"","note_ids":[],"sync_note_ids":[]}') as (port, seen):
|
||||
@@ -1171,7 +1158,7 @@ def test_hook_nudges_recording_when_copies_exist_but_nothing_is_recorded(tmp_pat
|
||||
|
||||
|
||||
def test_hook_says_when_scribe_did_not_answer_once_per_outage(tmp_path):
|
||||
"""#2932: a configured instance that does not answer (refused connection)
|
||||
"""A configured instance that does not answer (refused connection)
|
||||
is SAID — the write went unchecked — instead of the hook failing open in
|
||||
silence; the record nudge's "nothing recorded" claim is withheld. Once per
|
||||
outage: a second miss is quiet, an answer clears the marker, and the next
|
||||
@@ -1249,14 +1236,13 @@ def test_hook_stays_quiet_about_recording_when_nothing_is_duplicated(tmp_path):
|
||||
def test_local_arm_finds_duplicates_in_every_language_family(
|
||||
tmp_path, fname, definition
|
||||
):
|
||||
"""#2682: the definition detector must cover ALL code, not the languages of
|
||||
the repo it was born in. Its original CSS/JS/Python-only patterns silently
|
||||
amputated the local arm — and the #2664 recording nudge gated on it — for
|
||||
every Go/Kotlin/Rust project, which is exactly where the operator observed
|
||||
recording never happening. Each case stages an existing copy and writes the
|
||||
"""The definition detector must cover ALL code, not the languages of the
|
||||
repo it was born in. CSS/JS/Python-only patterns would silently disable
|
||||
the local arm — and the recording nudge gated on it — for every
|
||||
Go/Kotlin/Rust project. Each case stages an existing copy and writes the
|
||||
same definition to a second file; the hook must prove the duplication and
|
||||
ask for the record (the instance ANSWERS "nothing recorded" — since #2932
|
||||
an unanswered call withholds the nudge, so a sink stands in for it)."""
|
||||
ask for the record (the instance ANSWERS "nothing recorded" — an
|
||||
unanswered call withholds the nudge, so a sink stands in for it)."""
|
||||
with http_sink(b'{"context":"","note_ids":[],"sync_note_ids":[]}') as (port, _seen):
|
||||
env = dict(_hook_runtime_env(), SCRIBE_URL=f"http://127.0.0.1:{port}")
|
||||
repo = tmp_path / "repo"
|
||||
@@ -1285,7 +1271,7 @@ def test_local_arm_finds_duplicates_in_every_language_family(
|
||||
assert "create_snippet" in ctx
|
||||
|
||||
|
||||
# --- #2791: the write-path feed — hook evidence lands as ledger rows ----------
|
||||
# --- the write-path feed — hook evidence lands as ledger rows -----------------
|
||||
|
||||
|
||||
def _ts():
|
||||
@@ -1326,12 +1312,10 @@ async def test_a_pulled_snippet_already_seen_is_evidence_not_menu():
|
||||
writes code resembling it. #7 must be scored for this payload and handed to
|
||||
the stamp as resemblance.
|
||||
|
||||
THE MENU HALF INVERTED AT #4101. This asserted that #7 was evidence and NOT
|
||||
menu — scored for the stamp, kept out of the lines. That followed from the
|
||||
ledger being a hard exclusion; it is now a marker, so #7 is both, and the
|
||||
`limit` no longer needs widening for it because nothing is dropped after
|
||||
the search. What survives unchanged is the part the stamp depends on: the
|
||||
pulled id stays in the query and its score reaches `resembles`.
|
||||
The ledger is a marker, not a hard exclusion, so #7 is both evidence and
|
||||
menu, and the `limit` needs no widening for it because nothing is dropped
|
||||
after the search. The part the stamp depends on: the pulled id stays in
|
||||
the query and its score reaches `resembles`.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
search = AsyncMock(return_value=[(0.91, fake_note(id=7, title="pulled", user_id=1, note_type="snippet")), (0.80, fake_note(id=8, title="fresh", user_id=1, note_type="snippet"))])
|
||||
@@ -1438,7 +1422,7 @@ def test_route_stamps_only_for_a_caller_allowed_to_write():
|
||||
|
||||
|
||||
def test_hook_does_not_name_a_type_import_specifier_as_a_shape(tmp_path):
|
||||
"""#2904, the awk mirror of the server rule: `type Foo,` inside an import
|
||||
"""The awk mirror of the server rule: `type Foo,` inside an import
|
||||
list is not a definition; `type Baz = …` on its own line is."""
|
||||
env = _hook_runtime_env()
|
||||
repo = tmp_path / "repo"
|
||||
@@ -1460,7 +1444,7 @@ def test_hook_names_the_shapes_being_written():
|
||||
found by walking the target file upward from the edited lines."""
|
||||
src = HOOK.read_text()
|
||||
lib = (PLUGIN / "hooks" / "scribe_defs.sh").read_text()
|
||||
assert "scribe_defs()" in lib # one extractor, shared (#2901)
|
||||
assert "scribe_defs()" in lib # one extractor, shared
|
||||
assert "scribe_defs()" not in src # ...not a second copy here
|
||||
assert ".tool_input.old_string" in src # the Edit's anchor
|
||||
assert '"$file_path" | scribe_defs | tail -1' in src # nearest definition above
|
||||
@@ -1532,7 +1516,7 @@ def test_hook_sends_the_enclosing_definition_for_a_body_edit(tmp_path):
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_write_time_derive_check_names_a_family_or_a_canon_in_band():
|
||||
"""#2900: the ledger's own word on the names being written — a duplicate
|
||||
"""The ledger's own word on the names being written — a duplicate
|
||||
family with no canon, or a canon recorded elsewhere — rendered at the
|
||||
write even when nothing else does; keyed so the session's exclude
|
||||
channel silences a family already named."""
|
||||
@@ -1578,9 +1562,9 @@ async def test_the_write_time_derive_check_names_a_family_or_a_canon_in_band():
|
||||
assert out["derive_keys"] == ["name:css:log-empty", "canon:2855", "dup:483a"]
|
||||
ctx = out["context"]
|
||||
assert "Shape ledger at `frontend/src/components/New.vue`" in ctx
|
||||
# A CSS family is a repeated NAME (note 2917) and its dismissal is scoped-css;
|
||||
# a code dup family is an identical body and dismisses as convention-plumbing.
|
||||
# A css family says what renders it (milestone 302) before the ask.
|
||||
# A CSS family is a repeated NAME and its dismissal is scoped-css; a code
|
||||
# dup family is an identical body and dismisses as convention-plumbing.
|
||||
# A css family says what renders it before the ask.
|
||||
assert "`.log-empty` is a repeated name with no canon — defined in 5 other file(s): " \
|
||||
"`a/TaskLogSection.vue`, `a/WorkspaceTaskPanel.vue` +3 more; used by 6 templates: " \
|
||||
"`a/TaskLogSection.vue`, `a/V.vue` +4 more; derive it now" in ctx
|
||||
@@ -1612,7 +1596,7 @@ async def test_the_write_time_derive_check_names_a_family_or_a_canon_in_band():
|
||||
|
||||
|
||||
def test_the_hook_keeps_a_derive_channel_and_sends_it_back(tmp_path):
|
||||
"""#2900: derive keys the server returns land in the session's own
|
||||
"""Derive keys the server returns land in the session's own
|
||||
`.derive.ids` file and go back as `exclude_derive` on the next write —
|
||||
a family is named once per session, not at every edit."""
|
||||
reply = (b'{"context":"> family","note_ids":[],"sync_note_ids":[],'
|
||||
@@ -1635,7 +1619,7 @@ def test_the_hook_keeps_a_derive_channel_and_sends_it_back(tmp_path):
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_write_time_divergence_check_is_named_in_band():
|
||||
"""#2793: the hook named a shape at a path whose directory a canon
|
||||
"""The hook named a shape at a path whose directory a canon
|
||||
dominates, and the stamp didn't make it that canon's instance — the hint
|
||||
must say so at the write, even when nothing else renders."""
|
||||
from scribe.services import plugin_context as pc
|
||||
@@ -1653,7 +1637,7 @@ async def test_the_write_time_divergence_check_is_named_in_band():
|
||||
1, "frontend/src/components/Danger.vue", code=REAL_CODE, project_id=24,
|
||||
stamp_shapes=[("sym", "confirmDanger")],
|
||||
)
|
||||
# The payload travels with the call now (#4204): the check compares the
|
||||
# The payload travels with the call: the check compares the
|
||||
# shape being written against the canon's form, and a shape written right
|
||||
# now may exist nowhere but in this code.
|
||||
check.assert_awaited_once_with(24, "frontend/src/components/Danger.vue",
|
||||
@@ -1676,11 +1660,11 @@ async def test_the_write_time_divergence_check_is_named_in_band():
|
||||
|
||||
|
||||
def test_hook_keeps_the_rule_channel_apart_from_the_other_three():
|
||||
"""Milestone 307's arm, pinned the way #2708's was.
|
||||
"""The rule arm's channel, pinned the way the sync channel's is.
|
||||
|
||||
Standing rules dedup on their OWN file and their OWN query parameter. One
|
||||
shared channel is the bug #2708 already fixed once: a hint of one class
|
||||
silencing a different class that had never been shown. A rule named early
|
||||
shared channel lets a hint of one class silence a different class that had
|
||||
never been shown. A rule named early
|
||||
must not be re-offered on every later write, and must not silence — or be
|
||||
silenced by — a snippet suggestion.
|
||||
"""
|
||||
@@ -1694,21 +1678,20 @@ def test_hook_keeps_the_rule_channel_apart_from_the_other_three():
|
||||
|
||||
# ── `best_available` describes the BAR, not this arm's own second filter ──────
|
||||
#
|
||||
# #3739 fixed the rule arms: a record the reader had already been shown was
|
||||
# being logged as something the ranker turned away, which made the near-miss
|
||||
# distribution report scores ABOVE the very threshold it is read against.
|
||||
# A record the reader had already been shown, logged as something the ranker
|
||||
# turned away, makes the near-miss distribution report scores ABOVE the very
|
||||
# threshold it is read against.
|
||||
#
|
||||
# The fix keyed on `suppressed_count`, and its NULL branch was justified by
|
||||
# "null means the caller passed its exclusions INTO the search, so the reported
|
||||
# The rule arms key on `suppressed_count`, and its NULL branch rests on "null
|
||||
# means the caller passed its exclusions INTO the search, so the reported
|
||||
# score is already post-exclusion". That is true of auto_inject and reuse_slot.
|
||||
# It is NOT true here: this is the one note arm that filters twice. `exclude_ids`
|
||||
# takes `seen - pulled_seen` into the search, but the pulled-and-seen ids stay in
|
||||
# the query on purpose (the arm's query doubles as the resemblance test) and are
|
||||
# dropped afterwards in Python.
|
||||
#
|
||||
# Live proof, on the first read after that fix shipped: write_path's near-miss
|
||||
# max was 0.822 while the lowest score it ever RETURNED was 0.6857 — a
|
||||
# "rejection" that beat every acceptance.
|
||||
# The symptom: a near-miss maximum above the lowest score the arm ever
|
||||
# RETURNED — a "rejection" that beat every acceptance.
|
||||
|
||||
|
||||
def _search_reporting(score, note):
|
||||
@@ -1747,18 +1730,15 @@ async def test_a_record_this_arm_withheld_itself_is_not_a_near_miss():
|
||||
captured PRE it, so a withheld record is indistinguishable from one the bar
|
||||
rejected — while scoring higher than anything the bar ever let through.
|
||||
|
||||
REACHED DIFFERENTLY SINCE #4101, and that is the news. The ledger used to
|
||||
produce this case and no longer can: a repeat comes back from the search
|
||||
The ledger cannot produce this case: a repeat comes back from the search
|
||||
and is rendered, so nothing is dropped and `best_available` describes the
|
||||
bar alone. What still produces it is the one post-search filter left — a
|
||||
bar alone. What produces it is the one post-search filter — a
|
||||
PULLED record that this same call already listed by place. It stays in the
|
||||
query because `resembles` needs its score, and is dropped from the menu
|
||||
because it is already on it, which is exactly the pre/post split.
|
||||
|
||||
The property is unchanged and worth as much as it ever was; only the setup
|
||||
that exhibits it moved. Written this way rather than deleted, because the
|
||||
filter is still there and an arm that reported a score for it would be
|
||||
making the same false claim about the bar.
|
||||
The filter is still there, and an arm that reported a score for it would
|
||||
be making a false claim about the bar.
|
||||
"""
|
||||
from scribe.services import plugin_context as pc
|
||||
rec = MagicMock()
|
||||
@@ -1791,20 +1771,19 @@ async def test_a_record_this_arm_withheld_itself_is_not_a_near_miss():
|
||||
assert row.kwargs["best_available"] is None, (
|
||||
"a 0.9 record this arm withheld itself was reported as the best thing "
|
||||
"the THRESHOLD turned away. It would read as a bar set far too high "
|
||||
"when the bar never rejected it at all (#3739)"
|
||||
"when the bar never rejected it at all"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_ledger_repeat_no_longer_produces_that_case_at_all():
|
||||
"""The other direction, and the reason the setup above had to move.
|
||||
"""The other direction: a ledger repeat is not withheld.
|
||||
|
||||
A record on the session ledger now clears the search, is rendered with a
|
||||
marker, and is counted in `suppressed` — so nothing is withheld and the
|
||||
reported score describes the bar. Pinned because the tempting way to keep
|
||||
the old test passing would have been to null the score whenever the ledger
|
||||
matched, which would delete the measurement #3670 was built for on exactly
|
||||
the calls where the bar is most worth reading.
|
||||
reported score describes the bar. Pinned because the tempting shortcut is
|
||||
to null the score whenever the ledger matches, which would delete the near-miss measurement on exactly the calls
|
||||
where the bar is most worth reading.
|
||||
"""
|
||||
rec = MagicMock()
|
||||
row = await _write_path_row(rec, exclude_ids=[7])
|
||||
@@ -1821,7 +1800,7 @@ async def test_a_call_that_withheld_nothing_still_reports_what_the_bar_refused()
|
||||
"""The other half, and what stops the fix being 'never report it'.
|
||||
|
||||
Without this, setting `best_available=None` unconditionally passes the test
|
||||
above while deleting the measurement #3670 was built for.
|
||||
above while deleting the near-miss measurement.
|
||||
"""
|
||||
rec = MagicMock()
|
||||
row = await _write_path_row(rec)
|
||||
|
||||
Reference in New Issue
Block a user