feat(rules): the rule arms stop filtering the corpus to one tier (#3702)
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 33s
CI & Build / TypeScript typecheck (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 1m29s
CI & Build / Python lint (push) Successful in 4s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 33s
CI & Build / TypeScript typecheck (push) Successful in 1m3s
CI & Build / Python tests (push) Successful in 1m7s
CI & Build / Build & push image (push) Successful in 1m29s
Both arms passed tier="conditional", on the reasoning that an always-on
rule is already in the session so re-surfacing it is pure noise. That
conflates two different things:
PRESENT IN CONTEXT the rule was delivered at session start
SALIENT AT THE MOMENT the rule is in front of the reader when the action
it governs is about to be taken
A rule handed over in a list at turn zero is present while a session writes
a config value three hundred turns later. It is not surfaced. So the filter
did not skip a redundant hint — it made a whole class of rules permanently
ineligible for the only mechanism that puts a rule in front of an agent AT
the moment, and the more important a rule is, the likelier it sat in that
class.
Underneath, the filter was doing the THRESHOLD's job. Whether a rule belongs
in a hint is a relevance question and a similarity bar is the control for
relevance. A categorical exclusion standing in for a relevance judgment
cannot be tuned, cannot be measured, and cannot be wrong in a way anybody
notices.
MEASURED, NOT SETTLED. The old comment's fear is real: a hint that fires on
every write and says obvious things teaches the reader to skip the block. It
had simply never been checked, and retrieval_logs already records the scores
to check it with. Rules clearing often and high means the fear was justified
and the BAR is the work; rules clearing rarely in a thin band means relevance
was always sufficient.
Only eligibility moved. The bar stays at 0.72 and k stays at 1, so the
resulting distribution has one cause — and k=1 bounds the blast radius: a
wider pool can change which rule surfaces and how often, never how long a
single hint gets.
The threshold rationale's first premise ("the eligible corpus is TINY —
tier=conditional only") is updated rather than deleted: a larger pool makes
clearing the bar mean MORE, so that argument weakened, and the bar was left
alone anyway rather than move two variables at once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011cPyzNnegXHr5iRMzzy5KJ
This commit is contained in:
@@ -774,10 +774,17 @@ async def semantic_search_rules(
|
||||
is the surfacing question, and it has its own machinery
|
||||
(get_applicable_rules) rather than a second, subtly different copy here.
|
||||
|
||||
`tier` narrows to one tier. The write-path hint passes "conditional",
|
||||
because an always-on rule is ALREADY in the session — surfacing it again as
|
||||
a suggestion is pure noise, and noise on a hint that fires on every write
|
||||
is how a hint gets ignored.
|
||||
`tier` narrows to one tier, and NONE is the ordinary case. The write-path
|
||||
and pre-tool hints deliberately pass nothing: an always-on rule is already
|
||||
in the session, but being in a list from turn zero is not the same as being
|
||||
in front of the reader when the action it governs is taken, and filtering
|
||||
on tier made a whole class of rules permanently ineligible for the one
|
||||
mechanism that surfaces a rule AT the moment. Relevance is the threshold's
|
||||
job; see the block above RULEHINT_LIMIT in services/plugin_context.py for
|
||||
the argument and for what the resulting scores are being read against.
|
||||
|
||||
Pass a tier when a caller genuinely wants one class — a listing, an audit,
|
||||
a UI that renders the tiers apart. Not to approximate relevance.
|
||||
|
||||
Collapses to best-chunk-per-rule like the note search, so a long rule split
|
||||
across chunks competes once rather than crowding the results with itself.
|
||||
|
||||
Reference in New Issue
Block a user