docs(retrieval): stop asking the operator to diagnose retrieval (#4102)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m30s
CI & Build / Build & push image (push) Successful in 33s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m30s
CI & Build / Build & push image (push) Successful in 33s
The last item in this step's done-when: no wording anywhere asks an operator to tune for correctness. Four Settings hints told them to do exactly that — "raise it if rules keep arriving unread", "lower it if a git push arrives with nothing", "lower this if genuine duplicates go unnoticed". Every one of those asks the operator to diagnose a ranker from symptoms, which is the job the model now does from the records: the telemetry says what each bar refused, and reading those records is what separates a real miss from a bar doing its job. The hints keep the explanation of WHAT each number is — that is worth reading — and drop the homework. `plugin_context.py` said the defaults "are meant to be tuned from retrieval_logs once data accrues", which was true and had no owner. It now names who does it and with what. `retrieval_telemetry`'s docstring gained the warning that belongs beside it: this readout has been measured pointing the wrong way, so the ids it returns are the point, not its percentiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
This commit is contained in:
@@ -56,9 +56,14 @@ _GOAL_CHARS = 200
|
||||
# Per-user settings (keys live in the generic settings table). The threshold is
|
||||
# deliberately STRICTER than the pull-search default (embeddings
|
||||
# DEFAULT_SIMILARITY_THRESHOLD = 0.45): an unsolicited per-turn inject must clear
|
||||
# a higher bar than a search the agent chose to run. Defaults start conservative
|
||||
# and are meant to be tuned from retrieval_logs (source='auto_inject') once data
|
||||
# accrues — they're exposed in the Settings UI, no restart needed.
|
||||
# a higher bar than a search the agent chose to run.
|
||||
#
|
||||
# The defaults below are STARTING POINTS, and correcting them is the model's
|
||||
# job, not the operator's (#4102): `retrieval_surfaces` says what is in force,
|
||||
# `retrieval_telemetry(near_miss_samples=N)` says what it refused, and
|
||||
# `tune_retrieval` moves it with the reason attached. The operator can set any
|
||||
# of them in Settings and their change is recorded the same way — but nobody
|
||||
# has to read a log to get correct behaviour out of this.
|
||||
AUTOINJECT_ENABLED_KEY = "kb_autoinject_enabled"
|
||||
# The key and the value both live in the registry now (#4102); these names
|
||||
# survive because the comments above them are where each number's measurement
|
||||
|
||||
Reference in New Issue
Block a user