Files
FabledScribe/tests
bvandeusenandClaude Opus 5 690ca0306e
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 4s
CI & Build / TypeScript typecheck (push) Successful in 1m2s
CI & Build / Python tests (push) Successful in 1m34s
CI & Build / integration (push) Successful in 1m10s
CI & Build / Build & push image (push) Successful in 35s
feat(rules): the command arm gets its own bar, measured (#3853)
One threshold served both act arms. The telemetry says they are not the
same problem:

  write_path_rule   2,325 calls, speaks on 37%, near-miss p50 0.6989
  pre_tool_rule    11,768 calls, speaks on  2%, near-miss p50 0.6794

The second is not quiet, it is mute — 11,530 of 11,768 calls said nothing,
with near-miss p90 at 0.7097 against a 0.72 bar. Refused mass piled one
hundredth under the line is what a bar set too high leaves behind, and the
note arms are the control: auto_inject refuses at p90 0.5463, write_path at
0.6738, both far below theirs.

The cause is query shape, not corpus. A write-path query is a code payload,
long and rich — the case 0.72 was calibrated on. A pre-tool query is a shell
command, often under a dozen words: less text, less signal, lower scores for
the same relevance.

MEASURED. Eight replayed queries against the post-#3855 corpus, consequential
acts against innocuous ones:

  0.7571  git push origin dev              consequential
  0.7245  cd ...; git fetch; git add -A    consequential
  0.7193  git pull --rebase origin dev     consequential
  0.6850  docker compose up -d             consequential
  ------------------------------------- 0.68
  0.6735  wc -l src/*.py && date           innocuous
  0.6544  grep -rn useState src/           innocuous
  0.6099  sed -n '120,160p' package.json   innocuous
  0.6056  ls -la && cat README.md          innocuous

At 0.72 three of four consequential acts retrieved nothing, including
`git pull --rebase origin dev`, where rules 153, 1 and 2 all ranked correctly
between 0.7126 and 0.7193 and were all refused.

The separation is 0.0115 wide. That is a direction, not a settled number, and
the comment says so — near_miss_samples on a few days of post-#3855 traffic
is what settles it.

This also corrects an assumption the old comment stated: it argued 0.68 sat
"below where this corpus's noise sits", inferring a higher floor from the
corpus being homogeneous. Measured, the command arm's noise ceiling is 0.6735,
so 0.68 clears it barely rather than sitting under it.

Lowering is safer now than it would have been. Until #3851 this arm had one
slot, so the bar was the only noise control; the band now filters downstream,
so the bar's job shrank and the bar can.

write_path_rule is unchanged — healthy at 0.72 on its own evidence.

Guards: the two bars parse independently, garbage falls back to its OWN
default rather than to the sibling's (which would silently re-merge them),
the command default stays below the write-path default as a direction check,
and each arm both SEARCHES and REPORTS at its own bar. That last one is a
failure the single-bar code could not have had: retrieval_logs.threshold is
what near-miss analysis is read against, so an arm searching at one number
and logging another misreports the refusal and invites moving the bar that
was already right.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
2026-09-11 14:36:18 -04:00
..