refactor(retrieval): the specs are the registry - SURFACES, the ranked POINTS rows and RANKED_SOURCES are read off the pipeline specs (milestone 456 step 5, #4907)
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Successful in 2m1s
CI & Build / Build & push image (push) Successful in 27s
CI & Build / Python lint (push) Successful in 3s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 54s
CI & Build / integration (push) Successful in 59s
CI & Build / Python tests (push) Successful in 2m1s
CI & Build / Build & push image (push) Successful in 27s
Each arm spec now carries its tuning pair (Surface, moved verbatim into retrieval_pipeline) and a Declared block - what the telemetry readout must know and cannot read off its rows. The two ranked stages that are not arms (preference_slot, rule_via_lesson) are RankedSource specs, and the note slots carry their own declaration. - retrieval_surfaces.SURFACES = the TUNED_ARMS tuning, same order - retrieval_registry.POINTS ranked rows = one per spec in RANKED; the lookups, asked, ambient and pull rows stay declared there - rule_usage.RANKED_SOURCES = RULE_RANKED_SOURCES + moment_rule Settings keys, defaults, prose and order are unchanged (checked field by field against HEAD). tests/test_retrieval_specs.py pins the derivation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -64,9 +64,10 @@ pointed in opposite directions, and only opening the record could tell.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from scribe.services.settings import bounded_float, get_setting
|
||||
from scribe.services.retrieval_pipeline import ( # noqa: F401 - Surface re-exported
|
||||
TUNED_ARMS, Surface,
|
||||
)
|
||||
|
||||
# A budget nobody should be able to set past. Not a tuning value — a guard on
|
||||
# the worst case, so a mistyped setting cannot turn a menu into a wall of text.
|
||||
@@ -75,153 +76,13 @@ from scribe.services.settings import bounded_float, get_setting
|
||||
MAX_BUDGET = 10
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Surface:
|
||||
"""One push arm's tunable pair, plus enough prose to tune it responsibly.
|
||||
|
||||
`asks` / `over` / `fires` are not documentation for this file — they are
|
||||
rendered by the tuning tool and the Settings UI. A floor cannot be moved
|
||||
sensibly by anyone, model or human, who does not know what the query is, what
|
||||
corpus it runs against, or how often it costs something. Those three facts
|
||||
are exactly what separates these arms from each other, and they were
|
||||
previously recoverable only by reading `plugin_context.py`.
|
||||
"""
|
||||
|
||||
name: str
|
||||
"""The telemetry `source` value, and the join key.
|
||||
|
||||
MUST equal the string this arm passes to `record_retrieval`. Everything
|
||||
useful about tuning depends on that identity: the tool that moves a floor
|
||||
and the table that says what the floor did have to be talking about the same
|
||||
arm. A test asserts it rather than a comment asking nicely.
|
||||
"""
|
||||
|
||||
floor_key: str
|
||||
floor_default: float
|
||||
budget_key: str
|
||||
budget_default: int
|
||||
asks: str
|
||||
over: str
|
||||
fires: str
|
||||
measured_model: str = "BAAI/bge-small-en-v1.5"
|
||||
measured_shape: int = 1
|
||||
"""What the SHIPPED defaults above were measured against (#4104).
|
||||
|
||||
A floor is a distance in one embedding model's geometry, over documents cut
|
||||
one particular way. Either can change, and when one does every number in
|
||||
this table describes something that no longer exists.
|
||||
|
||||
TWO FIELDS, NEVER ONE FUSED STRING (rule 149). A mismatch has to be able to
|
||||
say WHICH half moved: a new embedding model and a re-cut document shape
|
||||
invalidate the same numbers for different reasons and call for different
|
||||
responses. `"<model>@<n>"` could only report that something changed, which
|
||||
is the answer nobody can act on. Same reason `calibration_stamp()` returns
|
||||
a dict and the event table gives each half its own column.
|
||||
|
||||
Recorded per surface rather than once for the module because they need not
|
||||
move together: a surface retuned after a model change carries the new stamp
|
||||
while its untouched siblings still carry the old one, and telling those
|
||||
apart is the whole job.
|
||||
|
||||
LITERALS, deliberately, rather than an import of the live values — a stamp
|
||||
says what was true when the number was chosen, so one that tracked the
|
||||
current model would always agree with it and could never report staleness.
|
||||
"""
|
||||
|
||||
budget_falls_back_to: str = ""
|
||||
"""A budget key to inherit when this surface has none of its own set.
|
||||
|
||||
Only `write_path` uses it, and only because it USED to share auto-inject's
|
||||
`top_k` outright. Giving it a key without this would silently reset the
|
||||
budget of every install that had tuned the shared one — a behaviour change
|
||||
delivered as a default, which is the shape of regression nobody reports
|
||||
because nothing looks broken.
|
||||
"""
|
||||
|
||||
|
||||
# THE TABLE IS READ OFF THE SPECS (milestone 456 step 5). Each ranked arm in
|
||||
# `retrieval_pipeline` carries its own `Surface` — the floor and budget pair,
|
||||
# its settings keys and defaults, and the prose a tuner reads — so an arm added
|
||||
# there is tunable by construction, and nothing here can drift from it. The
|
||||
# keys and defaults are the ones this table held before the move, unchanged.
|
||||
SURFACES: dict[str, Surface] = {
|
||||
"auto_inject": Surface(
|
||||
name="auto_inject",
|
||||
floor_key="kb_autoinject_threshold",
|
||||
floor_default=0.55,
|
||||
budget_key="kb_autoinject_top_k",
|
||||
budget_default=3,
|
||||
asks="the operator's message, as they typed it",
|
||||
over="notes, snippets, processes and issues",
|
||||
fires="once per operator turn",
|
||||
),
|
||||
"write_path": Surface(
|
||||
name="write_path",
|
||||
floor_key="kb_writepath_threshold",
|
||||
floor_default=0.68,
|
||||
budget_key="kb_writepath_top_k",
|
||||
budget_default=3,
|
||||
budget_falls_back_to="kb_autoinject_top_k",
|
||||
asks="the code being written, rewritten as a concept query",
|
||||
over="snippets and recorded issues",
|
||||
fires="before every Write and Edit",
|
||||
),
|
||||
"write_path_rule": Surface(
|
||||
name="write_path_rule",
|
||||
floor_key="kb_rulehint_threshold",
|
||||
floor_default=0.72,
|
||||
budget_key="kb_rulehint_top_k",
|
||||
budget_default=5,
|
||||
asks="the code being written, against rule triggers",
|
||||
over="global rules plus the bound project's own",
|
||||
fires="before every Write and Edit",
|
||||
),
|
||||
"pre_tool_rule": Surface(
|
||||
name="pre_tool_rule",
|
||||
floor_key="kb_toolrule_threshold",
|
||||
floor_default=0.68,
|
||||
budget_key="kb_toolrule_top_k",
|
||||
budget_default=5,
|
||||
asks="the command about to run, against rule triggers",
|
||||
over="global rules plus the bound project's own",
|
||||
fires="before every Bash call — the busiest arm there is",
|
||||
),
|
||||
"prompt_rule": Surface(
|
||||
name="prompt_rule",
|
||||
floor_key="kb_promptrule_threshold",
|
||||
floor_default=0.72,
|
||||
budget_key="kb_promptrule_top_k",
|
||||
budget_default=3,
|
||||
asks="the operator's message, against rule triggers",
|
||||
over="global rules plus the bound project's own",
|
||||
fires="once per operator turn",
|
||||
),
|
||||
"report_preference": Surface(
|
||||
name="report_preference",
|
||||
floor_key="kb_reportpref_threshold",
|
||||
floor_default=0.72,
|
||||
budget_key="kb_reportpref_top_k",
|
||||
budget_default=3,
|
||||
# THE ONE FIXED QUERY, and the reason this arm behaves unlike the rest.
|
||||
# The others score something that varies per call; this one scores a
|
||||
# constant string, so its top score for a given corpus is also a
|
||||
# constant. A floor a hair above that constant is not a quiet arm, it is
|
||||
# a dead one, and no amount of traffic will ever reveal it — which is
|
||||
# precisely how this arm spent 69 calls declining the same record.
|
||||
asks="a fixed question about how to lay out a completion report",
|
||||
over="preferences",
|
||||
fires="when a task finishes",
|
||||
),
|
||||
# THE REPLY BACKSTOP (milestone 458, folded in from 456 step 8). Its floor
|
||||
# is a STOP bar, not a hint bar: at the end of a turn nothing can be shown
|
||||
# beside the reply, so a hit either holds the reply for one read or says
|
||||
# nothing. Hence a default at the checkpoint's level and a budget of one —
|
||||
# the call row's results are then exactly the rule that would hold.
|
||||
"reply_rule": Surface(
|
||||
name="reply_rule",
|
||||
floor_key="kb_replyrule_threshold",
|
||||
floor_default=0.80,
|
||||
budget_key="kb_replyrule_top_k",
|
||||
budget_default=1,
|
||||
asks="the reply that ends a turn, against rule triggers",
|
||||
over="global rules plus the bound project's own",
|
||||
fires="once per turn, when the reply is finished",
|
||||
),
|
||||
arm.tuning.name: arm.tuning for arm in TUNED_ARMS if arm.tuning is not None
|
||||
}
|
||||
|
||||
# Reserved slots are deliberately absent. `preference_slot`, `reuse_slot` and
|
||||
|
||||
Reference in New Issue
Block a user