Every ledger is cleared on a compact, and a retrieval floor becomes something the model maintains #163

Merged
bvandeusen merged 12 commits from dev into main 2026-09-17 12:01:42 -04:00
Owner

Milestone 416 steps 3 and 4. Twelve commits, CI 6968 green on ea108ac (1702 passed, TypeScript typecheck and integration green).

#4101 — every push surface gets a ledger; pull surfaces deliberately do not

The defect was a hand-maintained list of files to delete. /compact swept two ledger paths that somebody had written down, and every ledger added since was never added to the list — so after a compaction a session kept suppressing records it no longer held.

Fixed by making the sweep convention-derived (SCRIBE_LEDGER_DIRS) rather than a longer list, with a guard that reads that roster instead of restating it. The first cut swept only scribe-priorart and missed scribe-autoinject — the arm that fires most — which is exactly the failure mode the roster exists to stop.

The note arms also changed contract: a repeat is now rendered and marked seen rather than withheld. A withheld record is indistinguishable from one that never matched; a marked one tells the reader it was surfaced earlier and may no longer be in context. The place arm marks its repeats too — a half-marked menu is worse than either rule.

#4102 — the bar becomes a cost floor, and the model maintains it

Filed as "pick low floors, let k bind". The investigation found that premise half wrong: code arms still separate cleanly (write_path near-miss p50 0.6367 against accepted p50 0.7284, the same separation #2223 measured), while rule arms overlap almost entirely. One floor cannot serve both, and no better constant exists to ship — these are cosine distances in one embedding model's space measured against one corpus.

The operator's decision replaced the plan:

"the floor should be chosen and adjusted by the model using it… the user should be able to touch it but the model should be the thing handling it 9 times out of 10."

What landed:

  • services/retrieval_surfaces.py — one registry. Six push surfaces, each with {floor, budget} plus what it asks, over what corpus, and how often it fires, because a floor cannot be moved sensibly without all three. The surface name is its retrieval_logs.source, pinned by test.
  • retrieval_tuning_events + migration 0103 — the argument, which settings cannot hold. actor is separate from user_id because a person and the model act as the same user.
  • Three MCP toolsretrieval_surfaces, tune_retrieval, retrieval_tuning_history. tune_retrieval refuses a blank or perfunctory reason and carries the 69-decline worked example, because this readout has been measured pointing the wrong way.
  • REST + Settings UI — the five budgets that never had controls, and a "What has been tuned" panel. The generic /api/settings endpoint now routes registry-owned keys through the recorder, so a floor cannot move without leaving the trail.
  • Backup v16 — the tuning history travels with the settings it explains.

Not in this PR

The floors themselves are unchanged from what was already shipped. This adds the registry and the means to correct them; it does not retune anything.

🤖 Generated with Claude Code

https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy

Milestone 416 steps 3 and 4. Twelve commits, CI 6968 green on `ea108ac` (1702 passed, TypeScript typecheck and integration green). ## #4101 — every push surface gets a ledger; pull surfaces deliberately do not The defect was a hand-maintained list of files to delete. `/compact` swept two ledger paths that somebody had written down, and every ledger added since was never added to the list — so after a compaction a session kept suppressing records it no longer held. Fixed by making the sweep convention-derived (`SCRIBE_LEDGER_DIRS`) rather than a longer list, with a guard that reads that roster instead of restating it. The first cut swept only `scribe-priorart` and missed `scribe-autoinject` — the arm that fires most — which is exactly the failure mode the roster exists to stop. The note arms also changed contract: a repeat is now **rendered and marked `seen`** rather than withheld. A withheld record is indistinguishable from one that never matched; a marked one tells the reader it was surfaced earlier and may no longer be in context. The place arm marks its repeats too — a half-marked menu is worse than either rule. ## #4102 — the bar becomes a cost floor, and the model maintains it Filed as "pick low floors, let `k` bind". The investigation found that premise half wrong: code arms still separate cleanly (`write_path` near-miss p50 0.6367 against accepted p50 0.7284, the same separation #2223 measured), while rule arms overlap almost entirely. One floor cannot serve both, and no better constant exists to ship — these are cosine distances in one embedding model's space measured against one corpus. The operator's decision replaced the plan: > "the floor should be chosen and adjusted by the model using it… the user should be able to touch it but the model should be the thing handling it 9 times out of 10." What landed: - **`services/retrieval_surfaces.py`** — one registry. Six push surfaces, each with `{floor, budget}` plus what it asks, over what corpus, and how often it fires, because a floor cannot be moved sensibly without all three. The surface name is its `retrieval_logs.source`, pinned by test. - **`retrieval_tuning_events`** + migration 0103 — the argument, which `settings` cannot hold. `actor` is separate from `user_id` because a person and the model act as the same user. - **Three MCP tools** — `retrieval_surfaces`, `tune_retrieval`, `retrieval_tuning_history`. `tune_retrieval` refuses a blank or perfunctory reason and carries the 69-decline worked example, because this readout has been measured pointing the wrong way. - **REST + Settings UI** — the five budgets that never had controls, and a "What has been tuned" panel. The generic `/api/settings` endpoint now routes registry-owned keys through the recorder, so a floor cannot move without leaving the trail. - **Backup v16** — the tuning history travels with the settings it explains. ## Not in this PR The floors themselves are unchanged from what was already shipped. This adds the registry and the means to correct them; it does not retune anything. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
bvandeusen added 12 commits 2026-09-17 12:01:35 -04:00
fix(plugin): a compaction clears every session ledger, not the two on the list (#4101)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m4s
CI & Build / Build & push image (push) Skipped
c61f7301bc
`scribe_session_context.sh` cleared `.rules.ids` and `.opened.ids` by name and
left `.ids`, `.sync.ids` and `.derive.ids` standing, under a comment asserting
that was a decision. Reading the note arms says it was not: their exclusions go
straight into `semantic_search_notes`, so a surfaced note leaves the result set
rather than being rendered as a reference the way #3750 gave a repeated rule,
and unlike the rules ledger they never age. Hard, permanent, never cleared — a
note surfaced in a session's first minute is unreachable for the rest of it,
which is milestone 386's own defect alive on the arms that fire most often.

The list was the bug, so the fix is not a longer list. `scribe_clear_session_
ledgers` matches the naming convention instead — a per-session ledger is
`<sid>[.<kind>].ids` — which covers all five and covers the sixth on the day it
is written. `<sid>.unreached` is deliberately outside it: that records an
outage, not held context, and #2932 needs it to survive.

tests/test_session_ledger_clear.py runs the hook rather than grepping it for
`rm -f`, since grepping for the names is the pattern being removed. It pins
both directions — `compact`/`clear` take all five, `startup`/`resume` take
none — plus the convention the glob rests on, checked against the hooks
themselves so a ledger named outside it fails loudly instead of silently never
clearing.

Also drops a stale comment pointing at a rules-etag marker that milestone 394
retired.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
feat(retrieval): a repeat on the note arms is a reference, not silence (#4101)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 42s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / Python tests (push) Failing after 1m6s
CI & Build / Build & push image (push) Skipped
5c64ea0b4f
#3750 settled this for rules: a record the session was told about an hour ago
is not a record in front of the reader now, so the second time it is the best
answer it is rendered again with a tail saying so. The note and snippet arms
never got that fix, and theirs was worse — the ledger went into
`semantic_search_notes` as `exclude_ids`, so the repeat left the candidate set
entirely. Three things followed:

  - the second time a note was the best answer the session got SILENCE,
    indistinguishable from "nothing matched", on the arms that fire most
    (`auto_inject` alone ran 598 calls in five days);
  - a compaction made that permanent, since the ledger outlived the context it
    described — fixed one layer down in c61f730;
  - and `best_available` was measured against a candidate set the caller had
    already edited, so the bar could be blamed for a record the caller
    withheld (#3739, from the side its fix never reached).

The ledger is now a RENDERING fact. Every repeat is still ranked, still shown,
and carries a `seen` marker; the band is computed over all hits, because
letting the ledger move the cutoff would make "you were shown this" change what
counts as relevant. The marker is one word and deliberately not the rule arms'
phrasing — "before deciding it does not apply" is the voice of a record that
binds, and a dev-log borrowing it would claim authority it does not have.

Telemetry takes the rule arms' contract (#3752): `results` and `record_surfaced`
both take fresh only, the repeat is counted in `suppressed`, so this source's
surfaced set still matches its own log row (#3668). That makes a fact readable
that could not be stated here before — `result_count == 0` with
`suppressed_count > 0` is "everything that matched, the session has already
seen", which is a different claim about the bar from "nothing cleared it".

On the write path this also splits a variable that carried two claims. `seen`
was the ledger plus everything the call had already rendered, and both were
treated as reasons to withhold; `in_menu` keeps the same-call exclusion while
the ledger becomes a marker. That narrows the `best_available` compromise at
its old comment to the pulled-and-already-listed case, and retires the argument
that a suppression count here would be partial — nothing is hidden inside the
query any more.

Deliberately unchanged: the write-path SYNC class still shows once. Its claim
is about an edit in progress rather than a record's continuing relevance, and
repeating it every write to the same file would be nagging.

tests/test_ledger_references_not_silence.py pins both arms — the ledger never
reaching the search, the repeat rendered and distinguishable, the telemetry
split, the all-repeats call being readable, and the two exceptions (this call's
own menu, and the sync class).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(plugin): the sweep missed the arm that fires most — two ledger directories (#4101)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 42s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m46s
CI & Build / Build & push image (push) Skipped
f1e63d207f
CI caught two things in c61f730, and the second is the one that mattered.

1. `scribe_autoinject.sh` keeps its note ledger in `${TMPDIR}/scribe-autoinject`,
   not `scribe-priorart`. Every ledger NAMED in the hooks was in the one
   directory the sweep visited, so it read as complete — and the arm that fires
   most (598 calls in five days) was the only one still carrying the bug. The
   clear ran, found nothing to remove, and exited 0. `SCRIBE_LEDGER_DIRS` in
   `scribe_defs.sh` is now the roster, and the guard reads that string rather
   than a copy of it, so a test can no longer agree with itself forever.

2. The convention guard over-reached: it flagged `<sid>.snap` and
   `<sid>.blocked`, which live in `scribe-afterwrite` and `scribe-reportcheck`
   and can never be touched by the sweep. It is now two tests keyed on the
   directory, because the two assumptions fail differently — one lets a ledger
   sit where nothing sweeps, the other lets it sit in the right place under a
   name the sweep does not match.

`test_only_the_rule_ledger_is_cleared_and_the_note_ledgers_are_left` also went
red, correctly: the note arms' exemption was a real decision, recorded in a
test, and my commit message said nothing about it had been decided. That was
wrong and the test was right to stop me. The decision is reversed rather than
ignored, and the reasoning is that #4101 removed its premise: it rested on a
note repeat being WITHHELD, so clearing the ledger meant re-injecting whole
menu lines the session already had. Now a repeat is rendered with a `seen`
marker, so the ledger only decides whether that marker is true — and across a
compaction it is false, telling a freshly-summarised session it has already
seen a record that is nowhere in its context. The test keeps its name and
records the reversal with the reason, rather than being deleted.

Verified by running the hook directly: all six ledgers across both directories
gone on `compact`, `<sid>.unreached` left standing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(retrieval): the place arm marks its repeats too, and three tests meet the new contract (#4101)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 41s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m2s
CI & Build / Build & push image (push) Skipped
825491d859
CI on 5c64ea0 surfaced a half-measure I had shipped and three tests pinning the
semantics it replaces.

**The half-measure.** Only the semantic arm was marking repeats; the write
path's PLACE arm still dropped a nearby snippet that was on the ledger. One
menu, two rules, decided by which arm happened to find a record — and the
marker would then read as a complete account of what the session has met while
covering half the lines. `seen` now means this call's own menu and nothing
else, so both reuse arms mark and neither withholds.

**test_session_dedup_excludes_ids_from_the_reuse_arms** asserted the drop and
the ledger going into the query. Both are the defect; it is renamed and asserts
the marker instead.

**test_a_pulled_snippet_already_seen_is_evidence_not_menu** asserted #7 was
evidence and NOT menu, with the limit widened by one to pay for the drop. The
premise was the hard exclusion; #7 is now both, and the widening is no longer
needed for it.

**test_a_record_this_arm_withheld_itself_is_not_a_near_miss** is the
interesting one. Its property still holds and its setup no longer reaches it:
a ledger repeat comes back from the search and is rendered, so nothing is
withheld and `best_available` describes the bar. The one post-search filter
left is a PULLED record this same call already listed by place — kept in the
query because `resembles` needs its score, dropped from the menu because it is
already on it — so the test is rebuilt on that, and a sibling test pins the
direction that changed, so nulling the score whenever the ledger matches
cannot pass both.

Also stubs `superseded_ids` and `owner_names_for` in the new test file, which
were reaching for Postgres in a unit job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(tests): two expectations that the new contract corrected (#4101)
CI & Build / integration (push) Successful in 41s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 2m9s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / Build & push image (push) Successful in 27s
5d47342fbc
Both were mine, and both show the change behaving as designed.

`test_session_dedup_marks_the_reuse_arms_rather_than_silencing_them` asserted
#12 stays out of the search's `exclude_ids`. It does not, and should not: the
place arm has just listed it, so the semantic arm must not list it again. That
is the same-call rule, not the ledger. The claim I meant — the LEDGER never
reaches the query — is asserted on its own against a record the call has not
otherwise rendered.

`test_a_pulled_snippet_already_seen_is_evidence_not_menu` expected #7 and #8.
It gets #7 alone, because #7 (0.91) now anchors the margin band at 0.81 and #8
scores 0.80. Withholding #7 used to promote a materially weaker hit into a slot
it had not earned, with nothing in the output saying so — the band measures
distance from the best answer, and letting the ledger decide which answer that
is was the axis confusion the rule band was written to avoid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
refactor(retrieval): one registry for every surface's floor and budget (#4102)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 11s
CI & Build / integration (push) Successful in 50s
CI & Build / TypeScript typecheck (push) Successful in 57s
CI & Build / Python tests (push) Failing after 1m3s
CI & Build / Build & push image (push) Skipped
09b48457ff
Groundwork for the step's real change. The operator's decision is that the
floor is chosen and adjusted by the model using it, not shipped as a value
somebody has to defend:

  "we need a model consistent surface for the adjustment of these floor values.
   the user should be able to touch it but the model should be the thing
   handling it 9 times out of 10."

A tuning surface cannot be consistent across six arms that each spell their
configuration differently, so the arms stop owning their numbers.
`retrieval_surfaces.SURFACES` names each one, its floor key and default, its
budget key and default, and — because they are rendered by the tuning tool and
the Settings UI — what it asks, over what corpus, and how often it fires. A
floor cannot be moved responsibly by anyone who does not know those three.

Three things fall out:

- **`k` becomes a real budget everywhere.** Only auto-inject had a configurable
  one; `RULEHINT_LIMIT`, `PROMPTRULE_LIMIT` and `reply_preferences.LIMIT` were
  constants. `k` is what binds under a low floor, so it has to be settable per
  surface — and per surface is the point, since `pre_tool_rule` fires before
  every Bash call while `prompt_rule` fires once a turn.
- **`write_path` gets its own budget, inherited not reset.** It shared
  auto-inject's outright on the argument that "how many titles at once" means
  the same thing on both. It does not, for the same reason. Unset, it still
  reads auto-inject's key, so an install that tuned the shared knob does not
  silently drop to a new default.
- **The duplicated read-and-clamp goes.** That shape is canon #2860 across 295
  of 372 judged siblings. Survivable while the numbers were constants; not once
  they are meant to move.

The long measurement comments stay exactly where they are — #2223's noise-floor
probe, #3853's command-vs-code split, #3851's band measurement. The constants
they annotate now alias the registry, so there is one value and the reasoning
still sits beside it.

Tests build the write-path config from the registry (`helpers.writepath_cfg`)
instead of from hand-written dicts. That is not tidiness: the rule arms read
their numbers inside a fail-open `except`, so a dict missing one key does not
raise where a reader would see it — the arm silently becomes a no-op that reads
exactly like "fired and found nothing". Ten hand-written dicts each looked
complete on the day they were typed.

tests/test_retrieval_surfaces.py pins the identity everything rests on: a
surface's name IS its telemetry source. Nothing in the type system says so —
`record_retrieval(source="pre_tool_rule")` is a literal in another file — and
renaming one without the other yields an arm that can be tuned and not
measured, or measured and not tuned, with no symptom either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(tests): two readers moved, and the settings guard now checks the registry (#4102)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 9s
CI & Build / integration (push) Successful in 43s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Failing after 1m2s
CI & Build / Build & push image (push) Skipped
003bfd7a0a
CI on 09b4845 caught both, and the second is an improvement rather than a
repair.

`test_services_reply_preferences` patched `reply_preferences.get_setting`,
which the registry refactor removed — its floor and budget are resolved through
`retrieval_surfaces` now. Its key-independence test also asserted the arm asks
for exactly one key; it asks for two, because both of its numbers are its own
since this step, so the assertion names both and adds a check that the registry
and the module constant still agree about the floor key. They writing different
keys is the failure where the Settings form saves one string and the arm reads
another.

`test_settings_defaults_agree` parsed `plugin_context.py` for a bare
module-level float, and those constants now alias the registry. Rather than
teach the regex about aliases, the six retrieval floors are keyed on their
SURFACE NAME and read from the registry directly — which is strictly better for
this guard: a surface name is also its telemetry source, so a row names the same
arm the readout does, and a floor cannot be checked against a stale constant
that happened to keep its old value. `PLAN_MATCH_DEFAULT_THRESHOLD` is not a
push surface and keeps the older shape, with a note saying why.

Added while there: `test_every_tunable_surface_has_a_control`, derived from the
registry, so a seventh surface arrives as a failing test rather than as a number
only the model can reach (rules 25, 27).

Also lands the audit trail this step needs — `retrieval_tuning_events` (model +
migration 0103) and `services/retrieval_tuning.py`. The tool layer on top is
the next commit; the table is here because `reason` being REQUIRED is the whole
guardrail, and the schema is where that starts. A number moved silently leaves
nothing for the operator to review or disagree with, and the operator's decision
is that the model moves these "9 times out of 10".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
feat(retrieval): the model moves its own floors, and says why (#4102)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 8s
CI & Build / integration (push) Successful in 43s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / Python tests (push) Failing after 1m5s
CI & Build / Build & push image (push) Skipped
ca49a46c23
Milestone 416 step 4's write half. `retrieval_surfaces.py` made the six
push arms describe their `{floor, budget}` the same way; this adds the
three MCP tools that let the model READ that and change it, and the
backup sections that carry the reasons.

The operator's decision, which this implements:

    "the floor should be chosen and adjusted by the model using it… the
    user should be able to touch it but the model should be the thing
    handling it 9 times out of 10."

WHY A REASON IS REQUIRED, AND WHY THE TOOL ARGUES AGAINST PERCENTILES

The milestone originally listed self-tuning as a non-goal on one
measured case, and that case is now the tool's docstring rather than a
prohibition: `report_preference` logged 69 consecutive declines with
the refused record 0.0006 under the bar, and every percentile said
"lower it". The refused record was rule 77 "Extract intent from loose
phrasing" matched against a query about report layout — a false
positive. Lowering would have attached that rule to every completion
report ever written.

What separated the statistic from the correct action was OPENING the
record. So `tune_retrieval` refuses a blank or perfunctory reason,
tells the caller to read `retrieval_telemetry(near_miss_samples=5)`
and the record ids it names, and carries that 69-decline example — an
abstract warning loses to a number. The non-goal that survives is
*statistical* auto-tuning; nothing here reads a percentile and picks a
value.

BACKUP (v16), which is what CI caught

`retrieval_tuning_events` was neither backed up nor excluded, and
#2293's guard said so. It is backed up: `settings` already carried the
numbers, so dropping this would restore an install with six moved
dials and no argument for any of them — precisely the state the table
exists to prevent, and worse now that the model is the one moving
them. One `_retrieval_tuning_event_rows` builder called from both
exporters (snippet #2851); `user_id` travels because a restore has to
remap it, which is why the row builder is not the model's `to_dict()`.
`surface` is a registry name rather than a foreign key, so the history
survives a restore into an install whose ids all differ.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(mcp): the three tuning tools get a read/write classification (#4102)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 43s
CI & Build / TypeScript typecheck (push) Successful in 56s
CI & Build / Python tests (push) Successful in 1m31s
CI & Build / Build & push image (push) Successful in 22s
25bd6742e0
CI 6962 — `test_every_registered_tool_is_classified_exactly_once`. The
backup gap is closed (1692 passed); this is the next guard, and the same
shape of one: a tool registered without a classification is silently
denied to a read key, with nothing to notice (#3191).

`retrieval_surfaces` and `retrieval_tuning_history` read. Read access
matters more than usual for these two — a session that cannot see the
bar in force, or the reason it was last moved, is one that will move it
again blind.

`tune_retrieval` writes in both senses: the number the arm reads, and
the reason appended to the audit trail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
feat(retrieval): the operator can see what was tuned, and every write is recorded (#4102)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 13s
CI & Build / integration (push) Successful in 49s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m30s
CI & Build / Build & push image (push) Canceled after 31s
6240652dce
The other half of the bargain in milestone 416 step 4. The model moves
these dials; this is what makes that reviewable rather than merely
automatic.

THE HOLE THIS CLOSES

Every retrieval floor is an ordinary settings key, and `/api/settings`
accepts any key at all. A floor written through it landed correctly and
recorded nothing — a tuning history with holes in it, which is worse
than no history because it reads as complete.

So the generic endpoint now routes registry-owned keys through
`set_dial` instead of writing them as plain rows. ROUTED, not refused:
refusing would only work for callers that had been updated, while this
way the form, a script, and an old client all leave the trail, and
there is no version of "forgot to use the other endpoint". Clearing a
control is written as an explicit set back to the shipped default,
because the operator reverting something is the single most important
move this history can record.

`set_dial` now also refuses to record a no-op. The Settings form
re-sends every field on every save, so without that one press of Save
would write six rows saying the operator set six dials to the numbers
they were already on — and a history nobody can skim is one nobody
reads.

WHAT THE OPERATOR GETS

`/api/retrieval/surfaces`, `/surfaces/<name>` and `/tuning-history`,
with `actor` fixed server-side rather than taken from the payload: a
payload-supplied actor would let a model claim to be the operator, and
"did I do this, or did the session?" is the first question this list
is asked.

In Settings: the five missing BUDGETS (until now only auto-inject had
one, so the only control over a noisy surface was to raise its bar —
which discards that surface's best candidates along with its worst),
and a "What has been tuned" panel showing each change, who made it, and
the reason given. The operator's own changes are marked.

The MCP tool demands a reason; these endpoints do not. That asymmetry
is deliberate and stated in routes/retrieval.py: the requirement exists
to make the MODEL read the records before moving a number on someone
else's behalf, and the operator is that someone — a mandatory
justification box on every control would be friction charged to the one
participant who owes no explanation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
docs(retrieval): stop asking the operator to diagnose retrieval (#4102)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / integration (push) Successful in 48s
CI & Build / TypeScript typecheck (push) Successful in 52s
CI & Build / Python tests (push) Successful in 1m30s
CI & Build / Build & push image (push) Successful in 33s
c8bfa6947c
The last item in this step's done-when: no wording anywhere asks an
operator to tune for correctness.

Four Settings hints told them to do exactly that — "raise it if rules
keep arriving unread", "lower it if a git push arrives with nothing",
"lower this if genuine duplicates go unnoticed". Every one of those
asks the operator to diagnose a ranker from symptoms, which is the job
the model now does from the records: the telemetry says what each bar
refused, and reading those records is what separates a real miss from
a bar doing its job. The hints keep the explanation of WHAT each number
is — that is worth reading — and drop the homework.

`plugin_context.py` said the defaults "are meant to be tuned from
retrieval_logs once data accrues", which was true and had no owner.
It now names who does it and with what.

`retrieval_telemetry`'s docstring gained the warning that belongs
beside it: this readout has been measured pointing the wrong way, so
the ids it returns are the point, not its percentiles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
fix(docs): two comments cited the wrong snippet for the duplication (#4102)
CI & Build / Plugin hooks (push) Successful in 10s
CI & Build / Python lint (push) Successful in 3s
CI & Build / integration (push) Successful in 47s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / Python tests (push) Successful in 1m32s
CI & Build / Build & push image (push) Successful in 23s
ea108acac5
Both said the repeated read-and-clamp was "canon #2860 across 295 of 372
siblings". Wrong on both halves, and checked rather than assumed:
#2860 is "Scribe service function — the async_session unit", the
service-layer canon, and its instance count is that canon's dominance in
src/scribe/services — nothing to do with a threshold helper. The
divergence hook flagged the new module against it because the new module
is a service, which is the hook working correctly and me misreading it.

The duplication itself was real and is what the registry consolidated:
`plugin_context` read and clamped the pair twice over, three rule arms
did it again, and `reply_preferences._threshold` once more. The comments
now say that, and claim nothing about a ledger count.

`rule_usage.py:122` cites #2860 correctly as the service canon and is
untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01821k5B3Ysecp9fNYs92Kuy
bvandeusen merged commit d9e512e9b4 into main 2026-09-17 12:01:42 -04:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bvandeusen/FabledScribe#163