fix(tests): the config stand-in fell behind the real one, and ten arms silently no-opped (#4214)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 44s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m7s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 44s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m7s
CI & Build / Build & push image (push) Skipped
CI 7124 Python tests: 14 failed. One root cause behind ten of them, and the failure was the exact one `tests/helpers.writepath_cfg`'s docstring already warns about in prose — while being unable to prevent this instance of it. Three arms read their numbers out of that config dict inside a fail-open `except`. A missing key raises where nobody sees it, so the arm becomes a silent no-op, indistinguishable from the arm working and finding nothing. The helper derives its keys from `retrieval_surfaces.SURFACES` precisely to stop that — and `checkpoint_threshold` is deliberately NOT a surface, because everything in that table is a floor/budget pair belonging to one query and the checkpoint runs none. The derivation therefore could not see it, the write-path rule arm died before `record_retrieval`, and ten tests went red at once. Fixed at the helper, from the module constant, so there is still exactly one literal and it lives in the product. And the guard the docstring claimed now exists: `test_the_config_stand_in_carries_every_key_the_real_one_does` compares the stand-in's key set against the real `get_writepath_config`, so the next key that is not a surface fails loudly here instead of quietly disabling an arm under test. `test_retrieval_surfaces`'s hand-written key list gains it for the same reason, spelled out in place. The other four were the contract widening itself: `checkpoint` is present on every return of the tool arm, including its early ones, so four assertions comparing the whole dict needed it. That key is deliberately always present — the two arms feed one shell reader where an absent key and an empty one are read the same, so the difference is invisible exactly where it would bite. Verified statically: every cfg in the suite now routes through `writepath_cfg` (test_write_path_trigger's local `_cfg` delegates to it), no hand-written config dict survives, and the real config's eight keys match the stand-in's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
@@ -508,7 +508,7 @@ async def test_the_prompt_arm_says_nothing_when_asked_nothing():
|
||||
stack.enter_context(patch.object(pc, "record_rule_surfaced", MagicMock()))
|
||||
out = await pc.build_prompt_rule_hint(1, " ")
|
||||
|
||||
assert out == {"context": "", "rule_ids": []}
|
||||
assert out == {"context": "", "rule_ids": [], "checkpoint": {}}
|
||||
search.assert_not_called()
|
||||
log.assert_not_called()
|
||||
|
||||
@@ -653,7 +653,7 @@ async def test_an_empty_command_asks_the_ranker_nothing():
|
||||
stack.enter_context(patch.object(pc, "record_rule_surfaced", rec))
|
||||
out = await pc.build_tool_rule_hint(1, "Bash", " ")
|
||||
|
||||
assert out == {"context": "", "rule_ids": []}
|
||||
assert out == {"context": "", "rule_ids": [], "checkpoint": {}}
|
||||
search.assert_not_called()
|
||||
rec.assert_not_called()
|
||||
|
||||
@@ -669,7 +669,7 @@ async def test_the_tool_arm_fails_open():
|
||||
AsyncMock(side_effect=RuntimeError("boom"))))
|
||||
out = await pc.build_tool_rule_hint(1, "Bash", "docker compose up -d")
|
||||
|
||||
assert out == {"context": "", "rule_ids": []}
|
||||
assert out == {"context": "", "rule_ids": [], "checkpoint": {}}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -833,7 +833,7 @@ async def test_the_tool_arm_logs_the_call_that_found_nothing():
|
||||
log, rec = MagicMock(), MagicMock()
|
||||
out = await _run_tool_arm([], rec, retrieval_log=log)
|
||||
|
||||
assert out == {"context": "", "rule_ids": []}
|
||||
assert out == {"context": "", "rule_ids": [], "checkpoint": {}}
|
||||
assert log.call_count == 1
|
||||
assert log.call_args.kwargs["source"] == "pre_tool_rule"
|
||||
assert log.call_args.kwargs["results"] == []
|
||||
|
||||
Reference in New Issue
Block a user