docs(6205): retrieval_pipeline comments state the reason, not the instance that taught it
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m8s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 23s
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 12s
CI & Build / TypeScript typecheck (push) Successful in 53s
CI & Build / integration (push) Successful in 1m8s
CI & Build / Python tests (push) Successful in 1m55s
CI & Build / Build & push image (push) Successful in 23s
Record ids, milestone and step numbers, and the stories of copies drifting apart are removed; each invariant and slot is justified in present tense. The rule-band probe table stays as calibration. Section headers drop their milestone tags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,30 +1,26 @@
|
|||||||
"""One retrieval pipeline: a ranked surface is a spec, not a hand-copied arm.
|
"""One retrieval pipeline: a ranked surface is a spec, not a hand-copied arm.
|
||||||
|
|
||||||
Milestone 456. Every ranked rule surface used to write out the same sequence
|
Every ranked rule surface runs the same sequence — search, band, split fresh
|
||||||
by hand — search, band, split fresh from repeats, log the call before any early
|
from repeats, log the call before any early return, reserve a slot, render,
|
||||||
return, reserve a slot, render, record what was surfaced — once per arm, each
|
record what was surfaced. Written out by hand once per arm, each modelled on
|
||||||
modelled on the nearest sibling. The copies drifted, and the defects that
|
the nearest sibling, the copies drift, and a defect gets fixed in one copy
|
||||||
mattered were fixed one copy at a time (#3497, #3750, #3752): "this arm shipped
|
and inherited by the next.
|
||||||
with the same defect inherited from its sibling" was a comment in the code.
|
|
||||||
The invariants were held together by a parametrised test over three copies
|
|
||||||
rather than by there being one implementation.
|
|
||||||
|
|
||||||
Here the sequence is written ONCE (`run_rule_arm`), and an arm is a `RuleArm`:
|
Here the sequence is written ONCE (`run_rule_arm`), and an arm is a `RuleArm`:
|
||||||
which source it records under, and which of the stages it takes. The
|
which source it records under, and which of the stages it takes. The
|
||||||
differences between arms that used to be buried in their copies are now four
|
differences between arms are four booleans on one screen, where an accident
|
||||||
booleans on one screen — and the ones that are accidents rather than
|
is visible as one.
|
||||||
decisions are milestone 456 step 7's to settle, one at a time, on evidence.
|
|
||||||
|
|
||||||
THE INVARIANTS, each written once:
|
THE INVARIANTS, each written once:
|
||||||
|
|
||||||
- the call row is written before any early return (#3497), because a call
|
- the call row is written before any early return, because a call that
|
||||||
that found nothing is the only evidence a bar is too high;
|
found nothing is the only evidence a bar is too high;
|
||||||
- the call row's results and the surfacing rows are both the FRESH cut, so
|
- the call row's results and the surfacing rows are both the FRESH cut, so
|
||||||
the two tables describe the same delivery (#3668, #3752);
|
the two tables describe the same delivery;
|
||||||
- the session ledger decides how a repeat RENDERS, never whether it is
|
- the session ledger decides how a repeat RENDERS, never whether it is
|
||||||
retrieved (#3750, #4101);
|
retrieved;
|
||||||
- band before dedup (#3851), so "you were shown this" never changes what
|
- band before dedup, so "you were shown this" never changes what counts
|
||||||
counts as relevant;
|
as relevant;
|
||||||
- telemetry never costs the reader a line, and the arm as a whole fails
|
- telemetry never costs the reader a line, and the arm as a whole fails
|
||||||
open: a recall aid may never break the prompt, the command or the write.
|
open: a recall aid may never break the prompt, the command or the write.
|
||||||
|
|
||||||
@@ -43,14 +39,14 @@ from scribe.services.lessons import LESSON_NOTE_TYPE, claim_line
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
# ── Shared rendering and ranking helpers (moved from plugin_context) ──────
|
# ── Shared rendering and ranking helpers ─────────────────────────────────
|
||||||
|
|
||||||
# MEASURED, NOT REASONED — and the reasoning it replaced was wrong (#3851).
|
# MEASURED, NOT REASONED.
|
||||||
#
|
#
|
||||||
# The prediction was that rules would rank SHARPLY, because `rule_document()`
|
# The natural prediction is that rules rank SHARPLY, because `rule_document()`
|
||||||
# shapes them the way snippets are shaped — trigger in the embedded title and
|
# shapes them the way snippets are shaped — trigger in the embedded title and
|
||||||
# again above the body — and note 2485 measured snippets separating their top
|
# again above the body — and snippets separate their top hit by ~0.15 while
|
||||||
# hit by 0.153 while every other kind managed 0.010–0.023.
|
# every other kind manages 0.010–0.023.
|
||||||
#
|
#
|
||||||
# They do not. Three probes against real act queries, scored by the same
|
# They do not. Three probes against real act queries, scored by the same
|
||||||
# embedding the arms use:
|
# embedding the arms use:
|
||||||
@@ -60,21 +56,20 @@ logger = logging.getLogger(__name__)
|
|||||||
# a bare-owner-filter query top 0.656, gap to second 0.020
|
# a bare-owner-filter query top 0.656, gap to second 0.020
|
||||||
#
|
#
|
||||||
# That is dev-log territory, not snippet territory: rules arrive as a
|
# That is dev-log territory, not snippet territory: rules arrive as a
|
||||||
# tightly-packed block. Shaping alone did not buy separation, which is worth
|
# tightly-packed block. Shaping alone does not buy separation.
|
||||||
# recording because the opposite was the natural inference from 2485.
|
|
||||||
#
|
#
|
||||||
# So the band is narrow BECAUSE the corpus is flat. At 0.10 — the notes
|
# So the band is narrow BECAUSE the corpus is flat. At 0.10 — the notes
|
||||||
# menu's value — every one of the top eight on the push probe falls inside,
|
# menu's value — every one of the top eight on the push probe falls inside,
|
||||||
# including a CI-registry rule and another project's branch policy. At 0.05
|
# including a CI-registry rule and another project's branch policy. At 0.05
|
||||||
# it admits roughly three ranks, which is the span where the scores are still
|
# it admits roughly three ranks, which is the span where the scores are still
|
||||||
# saying something. 2485's Finding 3 is the standing caveat: no band value
|
# saying something. The standing caveat: no band value fixes a tie, and if
|
||||||
# fixes a tie, and if rules ever rank as flat as dev-logs did this control
|
# rules ever rank as flat as dev-logs this control stops working and the
|
||||||
# stops working and the answer is a reranker (#1038), not a smaller number.
|
# answer is a reranker, not a smaller number.
|
||||||
_RULEHINT_BAND = 0.05
|
_RULEHINT_BAND = 0.05
|
||||||
|
|
||||||
|
|
||||||
def _rule_band(hits: list) -> list:
|
def _rule_band(hits: list) -> list:
|
||||||
"""The top hit, plus every hit within `_RULEHINT_BAND` of it (#3851).
|
"""The top hit, plus every hit within `_RULEHINT_BAND` of it.
|
||||||
|
|
||||||
The instrument that lets an act surface a SET without inventing one: a
|
The instrument that lets an act surface a SET without inventing one: a
|
||||||
fixed k fills its slots whether or not anything deserves them, while this
|
fixed k fills its slots whether or not anything deserves them, while this
|
||||||
@@ -82,8 +77,8 @@ def _rule_band(hits: list) -> list:
|
|||||||
still shows one and four competing rules show four.
|
still shows one and four competing rules show four.
|
||||||
|
|
||||||
Sync and pure, and deliberately its own function rather than a comparison
|
Sync and pure, and deliberately its own function rather than a comparison
|
||||||
written twice — the two act arms are the pair #3497 records drifting apart
|
written twice: the two act arms drift apart when each is modelled on the
|
||||||
by being modelled on each other instead of sharing.
|
other instead of sharing.
|
||||||
|
|
||||||
Takes `(score, rule)` pairs already ordered best-first, as both
|
Takes `(score, rule)` pairs already ordered best-first, as both
|
||||||
`semantic_search_*` helpers return them.
|
`semantic_search_*` helpers return them.
|
||||||
@@ -100,9 +95,7 @@ def checkpoint_for(
|
|||||||
"""The one rule, if any, that should STOP this act rather than annotate it.
|
"""The one rule, if any, that should STOP this act rather than annotate it.
|
||||||
|
|
||||||
Sync and pure so it can be read against a fixed list of hits without a
|
Sync and pure so it can be read against a fixed list of hits without a
|
||||||
database — the two arms share it for the reason `_rule_band` is shared:
|
database — the two arms share it for the reason `_rule_band` is shared.
|
||||||
#3497 is the record of these two drifting apart by being modelled on each
|
|
||||||
other instead of sharing one function.
|
|
||||||
|
|
||||||
FOUR CONDITIONS, AND EACH IS A DIFFERENT KIND OF WRONG IT PREVENTS.
|
FOUR CONDITIONS, AND EACH IS A DIFFERENT KIND OF WRONG IT PREVENTS.
|
||||||
|
|
||||||
@@ -119,7 +112,7 @@ def checkpoint_for(
|
|||||||
names it.
|
names it.
|
||||||
|
|
||||||
3. THE SESSION HAS NOT OPENED IT. `held` is observable — a PostToolUse
|
3. THE SESSION HAS NOT OPENED IT. `held` is observable — a PostToolUse
|
||||||
hook watches for the `get_rule` call (#4100) — so this is a recorded
|
hook watches for the `get_rule` call — so this is a recorded
|
||||||
event and not a model's self-report about its own context. A session
|
event and not a model's self-report about its own context. A session
|
||||||
that read the rule has already had the thing the checkpoint exists to
|
that read the rule has already had the thing the checkpoint exists to
|
||||||
produce, and stopping it again would be punishing the behaviour being
|
produce, and stopping it again would be punishing the behaviour being
|
||||||
@@ -131,9 +124,8 @@ def checkpoint_for(
|
|||||||
a stop justified by a score nobody claimed.
|
a stop justified by a score nobody claimed.
|
||||||
|
|
||||||
Returns a dict rather than a rule or a tuple. Widening a tuple is an
|
Returns a dict rather than a rule or a tuple. Widening a tuple is an
|
||||||
interface change to every unpack site that the compiler does not report
|
interface change to every unpack site that the compiler does not report,
|
||||||
(#4207, learned the expensive way in this same milestone's first week), and
|
and this value crosses a JSON boundary into a shell script where a missing
|
||||||
this value crosses a JSON boundary into a shell script where a missing
|
|
||||||
field is a silently empty variable.
|
field is a silently empty variable.
|
||||||
"""
|
"""
|
||||||
if not kept or floor <= 0:
|
if not kept or floor <= 0:
|
||||||
@@ -154,7 +146,7 @@ def checkpoint_for(
|
|||||||
}
|
}
|
||||||
# RENDERED HERE, at the one call site, rather than by each arm. Two arms
|
# RENDERED HERE, at the one call site, rather than by each arm. Two arms
|
||||||
# that each remember to render it are two arms that can stop agreeing on
|
# that each remember to render it are two arms that can stop agreeing on
|
||||||
# what a stop says — which is #3497's history for this exact pair. The
|
# what a stop says. The
|
||||||
# text stays a separate function so it can be read and tested without a
|
# text stays a separate function so it can be read and tested without a
|
||||||
# rule object, but nothing outside this line decides whether to call it.
|
# rule object, but nothing outside this line decides whether to call it.
|
||||||
found["reason"] = checkpoint_reason(found)
|
found["reason"] = checkpoint_reason(found)
|
||||||
@@ -164,7 +156,7 @@ def checkpoint_for(
|
|||||||
def checkpoint_reason(checkpoint: dict) -> str:
|
def checkpoint_reason(checkpoint: dict) -> str:
|
||||||
"""The text the agent reads INSTEAD of running the act.
|
"""The text the agent reads INSTEAD of running the act.
|
||||||
|
|
||||||
Written as a practice rather than a prohibition (rule 165): it says what
|
Written as a practice rather than a prohibition: it says what
|
||||||
to do and why it is worth doing, not what is forbidden. The act is not
|
to do and why it is worth doing, not what is forbidden. The act is not
|
||||||
wrong — nothing here knows whether it is — and saying so plainly is what
|
wrong — nothing here knows whether it is — and saying so plainly is what
|
||||||
keeps the stop from reading as an accusation the system is in no position
|
keeps the stop from reading as an accusation the system is in no position
|
||||||
@@ -195,10 +187,10 @@ def checkpoint_reason(checkpoint: dict) -> str:
|
|||||||
def _rule_hint_line(
|
def _rule_hint_line(
|
||||||
rule, *, where: str, seen: bool, held: bool = False, compact: bool = False,
|
rule, *, where: str, seen: bool, held: bool = False, compact: bool = False,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""One rule hint line — both arms, both tails, both kinds (#3750, #3849).
|
"""One rule hint line — both arms, both tails, both kinds.
|
||||||
|
|
||||||
THREE INDEPENDENT AXES SINCE #3851. `compact` joins `kind` and `seen`, and
|
THREE INDEPENDENT AXES. `compact` joins `kind` and `seen`, and like them
|
||||||
like them it reads none of the others: it says how much ROOM this line
|
it reads none of the others: it says how much ROOM this line
|
||||||
gets, which is a fact about its rank among today's hits rather than about
|
gets, which is a fact about its rank among today's hits rather than about
|
||||||
the rule. A compact line is still a full claim that the rule may apply —
|
the rule. A compact line is still a full claim that the rule may apply —
|
||||||
it simply cites the rule instead of quoting its trigger. Crucially it
|
it simply cites the rule instead of quoting its trigger. Crucially it
|
||||||
@@ -206,13 +198,11 @@ def _rule_hint_line(
|
|||||||
independent: shortening a line must not decide what it says about whether
|
independent: shortening a line must not decide what it says about whether
|
||||||
the session is holding the rule.
|
the session is holding the rule.
|
||||||
|
|
||||||
WHY THE LATER LINES ARE QUIETER. Measured at #3851: a full line runs ~143
|
WHY THE LATER LINES ARE QUIETER. A full line runs ~143 tokens once the
|
||||||
tokens once the trigger is rendered, and #3855 tripled trigger lengths
|
trigger is rendered, so five full lines cost ~568 tokens before every Bash
|
||||||
across the corpus, so five full lines cost ~568 tokens before every Bash
|
call. Top-full-plus-references costs ~299 — about 2x a single full line,
|
||||||
call. Top-full-plus-references costs ~299 — about 2x the old single line,
|
for four more rules. The token budget forbids four voices at full volume,
|
||||||
for four more rules. The budget argument that once justified a single slot
|
not four voices.
|
||||||
was real; what it actually forbids is four voices at full volume, not four
|
|
||||||
voices.
|
|
||||||
|
|
||||||
The top hit keeps the full rendering because it is the one the ranker is
|
The top hit keeps the full rendering because it is the one the ranker is
|
||||||
most confident about, and a reader who acts on exactly one line should
|
most confident about, and a reader who acts on exactly one line should
|
||||||
@@ -222,32 +212,31 @@ def _rule_hint_line(
|
|||||||
and neither reads the other. A preference and a rule differ in force; a
|
and neither reads the other. A preference and a rule differ in force; a
|
||||||
repeat and a first surfacing differ in whether the session already holds
|
repeat and a first surfacing differ in whether the session already holds
|
||||||
the line. Those are unrelated facts, and keeping them unrelated in the
|
the line. Those are unrelated facts, and keeping them unrelated in the
|
||||||
code is what stopped the second kind from reopening the repeat question.
|
code is what lets a new kind join without reopening the repeat question.
|
||||||
|
|
||||||
ONE FUNCTION BECAUSE THE TAILS MUST NOT DRIFT. The two arms phrase their
|
ONE FUNCTION BECAUSE THE TAILS MUST NOT DRIFT. The two arms phrase their
|
||||||
heads differently ("may apply here" vs "may apply to this Bash call") and
|
heads differently ("may apply here" vs "may apply to this Bash call") and
|
||||||
that difference is deliberate. Everything after it must not differ, and
|
that difference is deliberate. Everything after it must not differ: an
|
||||||
#3497's history is that the pre-tool arm inherited a defect from its
|
arm modelled on its sibling rather than sharing with it inherits its
|
||||||
sibling by being modelled on it rather than sharing with it. Two copies of
|
defects, and two copies of a two-branch string is how one branch gets
|
||||||
a two-branch string is how one branch gets fixed and the other does not.
|
fixed and the other does not.
|
||||||
|
|
||||||
WHY A REPEAT GETS A LINE AT ALL. Both arms used to drop a hit whose id was
|
WHY A REPEAT GETS A LINE AT ALL. Dropping a hit whose id is already on the
|
||||||
already on the session's exclusion ledger and emit nothing. That is correct
|
session's exclusion ledger is correct only while the session still HOLDS
|
||||||
only while the session still HOLDS what it was told, and a compaction
|
what it was told, and a compaction breaks exactly that: the earlier
|
||||||
breaks exactly that: the earlier injection is summarized away while the id
|
injection is summarized away while the id stays on the ledger, so the rule
|
||||||
stays on the ledger, so the rule is absent from context AND unreachable for
|
would be absent from context AND unreachable for the rest of the session.
|
||||||
the rest of the session (#3749 closes the compaction half; this closes the
|
The same holds, more slowly, for a session that simply stops holding a
|
||||||
ordinary half, where a session simply stops holding a line it read an hour
|
line it read an hour ago.
|
||||||
ago).
|
|
||||||
|
|
||||||
Only ONE CLAUSE of the original line is false on a repeat — the claim that
|
Only ONE CLAUSE of a first-surfacing line is false on a repeat — the claim that
|
||||||
the rule is not in the session's loaded set. So only that clause changes.
|
the rule is not in the session's loaded set. So only that clause changes.
|
||||||
Title, trigger and pull pointer are identical either way, the statement is
|
Title, trigger and pull pointer are identical either way, the statement is
|
||||||
never injected either way, and a repeat therefore costs the same ~40 tokens
|
never injected either way, and a repeat therefore costs the same ~40 tokens
|
||||||
as a first surfacing and no more.
|
as a first surfacing and no more.
|
||||||
|
|
||||||
DELIBERATELY NOT ASKING THE SESSION WHETHER IT HOLDS THE RULE. A model
|
DELIBERATELY NOT ASKING THE SESSION WHETHER IT HOLDS THE RULE. A model
|
||||||
asked "do you still hold rule 156?" will say yes, and the claim is
|
asked "do you still hold rule N?" will say yes, and the claim is
|
||||||
unverifiable self-report about its own context. The answer is also not
|
unverifiable self-report about its own context. The answer is also not
|
||||||
needed: the line is cheap enough to always emit and carries its own remedy
|
needed: the line is cheap enough to always emit and carries its own remedy
|
||||||
in both branches. Removing the question removes the fragility rather than
|
in both branches. Removing the question removes the fragility rather than
|
||||||
@@ -265,8 +254,8 @@ def _rule_hint_line(
|
|||||||
# A reader skimming an injected block gets one word to place the register \u2014
|
# A reader skimming an injected block gets one word to place the register \u2014
|
||||||
# so it is the SECOND word that moves, and it is the word naming force.
|
# so it is the SECOND word that moves, and it is the word naming force.
|
||||||
# Everything structural after it is identical, so the three kinds read as
|
# Everything structural after it is identical, so the three kinds read as
|
||||||
# one set rather than three formats (milestone 385's step 5 writes the
|
# one set rather than three formats (the lesson voice is written against
|
||||||
# lesson voice against these two; they are one paragraph, not three).
|
# these two; they are one paragraph, not three).
|
||||||
noun = "Preference" if preference else "Standing rule"
|
noun = "Preference" if preference else "Standing rule"
|
||||||
# The only other place force is asserted. A rule's line tells the reader
|
# The only other place force is asserted. A rule's line tells the reader
|
||||||
# not to dismiss it unread, because dismissing a rule unread is how the
|
# not to dismiss it unread, because dismissing a rule unread is how the
|
||||||
@@ -277,21 +266,21 @@ def _rule_hint_line(
|
|||||||
"for how this has been done before" if preference
|
"for how this has been done before" if preference
|
||||||
else "before deciding it does not apply"
|
else "before deciding it does not apply"
|
||||||
)
|
)
|
||||||
# THREE STATES, BECAUSE TWO OF THEM WERE BEING TOLD THE SAME LIE (#4100).
|
# THREE STATES, BECAUSE TWO OF THEM MUST NOT BE TOLD THE SAME THING.
|
||||||
#
|
#
|
||||||
# `seen` means an arm NAMED this rule earlier. It does not mean the session
|
# `seen` means an arm NAMED this rule earlier. It does not mean the session
|
||||||
# read it — the line is a teaser, and a teaser skimmed past leaves nothing
|
# read it — the line is a teaser, and a teaser skimmed past leaves nothing
|
||||||
# behind, least of all after a compaction summarises the turn it arrived
|
# behind, least of all after a compaction summarises the turn it arrived
|
||||||
# in. "You saw it earlier this session" asserted something about the
|
# in. "You saw it earlier this session" would assert something about the
|
||||||
# reader's context that the server had no way to know.
|
# reader's context that the server has no way to know.
|
||||||
#
|
#
|
||||||
# `held` is the observable half: a PostToolUse hook watches for the
|
# `held` is the observable half: a PostToolUse hook watches for the
|
||||||
# `get_rule` call itself, so this is a recorded EVENT rather than a claim.
|
# `get_rule` call itself, so this is a recorded EVENT rather than a claim.
|
||||||
# That distinction is what keeps the non-goal above intact — the objection
|
# That distinction is what keeps the non-goal above intact — the objection
|
||||||
# was to asking a model about its own context, not to noticing what it did.
|
# was to asking a model about its own context, not to noticing what it did.
|
||||||
#
|
#
|
||||||
# The middle state is the honest one and the one that was missing: named,
|
# The middle state is the honest one: named, not opened. It gets the full
|
||||||
# not opened. It gets the full invitation, because a session that skipped
|
# invitation, because a session that skipped
|
||||||
# the teaser is in almost the same position as one that never saw it.
|
# the teaser is in almost the same position as one that never saw it.
|
||||||
if held:
|
if held:
|
||||||
tail = (
|
tail = (
|
||||||
@@ -310,17 +299,17 @@ def _rule_hint_line(
|
|||||||
)
|
)
|
||||||
if compact:
|
if compact:
|
||||||
# THE TRIGGER GOES; THE TAIL STAYS. Only one of the two is expensive \u2014
|
# THE TRIGGER GOES; THE TAIL STAYS. Only one of the two is expensive \u2014
|
||||||
# a trigger runs 300-400 characters after #3855, the tail about 100 \u2014
|
# a trigger runs 300-400 characters, the tail about 100 \u2014
|
||||||
# so dropping the trigger is nearly the whole saving and dropping the
|
# so dropping the trigger is nearly the whole saving and dropping the
|
||||||
# tail would be mostly sacrifice.
|
# tail would be mostly sacrifice.
|
||||||
#
|
#
|
||||||
# It would also destroy the one thing #3750 exists to say. The tail is
|
# It would also destroy the one thing a repeat line exists to say. The
|
||||||
# what tells a reader whether they were already told this and may no
|
# tail is what tells a reader whether they were already told this and
|
||||||
# longer be holding it, and that is the entire difference they can act
|
# may no longer be holding it, and that is the entire difference they
|
||||||
# on; a reference with no tail reads as a first surfacing whether it is
|
# can act on; a reference with no tail reads as a first surfacing
|
||||||
# one or not. This branch shipped without it for one commit and
|
# whether it is one or not.
|
||||||
# test_a_rule_the_session_already_holds_is_referenced_not_re_offered
|
# test_a_rule_the_session_already_holds_is_referenced_not_re_offered
|
||||||
# caught it, which is the guard working exactly as #3750 intended.
|
# pins this.
|
||||||
return (
|
return (
|
||||||
f"Also \u2014 {noun.lower()} \u201c{rule.title}\u201d. {tail}"
|
f"Also \u2014 {noun.lower()} \u201c{rule.title}\u201d. {tail}"
|
||||||
)
|
)
|
||||||
@@ -331,14 +320,14 @@ def _rule_hint_line(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# ── What a spec declares about itself (milestone 456 step 5) ─────────────
|
# ── What a spec declares about itself ────────────────────────────────────
|
||||||
#
|
#
|
||||||
# A ranked surface used to be described in three hand-kept lists besides its
|
# A ranked surface is described in three places besides its code: its tuning
|
||||||
# code: its tuning pair in `retrieval_surfaces.SURFACES`, its row in
|
# pair in `retrieval_surfaces.SURFACES`, its row in
|
||||||
# `retrieval_registry.POINTS`, and — for a rule surface — its membership in
|
# `retrieval_registry.POINTS`, and — for a rule surface — its membership in
|
||||||
# `rule_usage.RANKED_SOURCES`. Tests existed to make the four agree. Now the
|
# `rule_usage.RANKED_SOURCES`. The spec carries all of it, and the three lists
|
||||||
# spec carries all of it, and the three lists are read off the specs, so an
|
# are read off the specs, so an arm added here is tunable, measured and
|
||||||
# arm added here is tunable, measured and counted by construction.
|
# counted by construction rather than by four hand-kept lists agreeing.
|
||||||
#
|
#
|
||||||
# THE CLASSES LIVE HERE, NOT BESIDE THE LISTS, for the import graph: the
|
# THE CLASSES LIVE HERE, NOT BESIDE THE LISTS, for the import graph: the
|
||||||
# registry, the tuning table and the usage counter all read the specs, so the
|
# registry, the tuning table and the usage counter all read the specs, so the
|
||||||
@@ -353,8 +342,8 @@ class Surface:
|
|||||||
rendered by the tuning tool and the Settings UI. A floor cannot be moved
|
rendered by the tuning tool and the Settings UI. A floor cannot be moved
|
||||||
sensibly by anyone, model or human, who does not know what the query is, what
|
sensibly by anyone, model or human, who does not know what the query is, what
|
||||||
corpus it runs against, or how often it costs something. Those three facts
|
corpus it runs against, or how often it costs something. Those three facts
|
||||||
are exactly what separates these arms from each other, and they were
|
are exactly what separates these arms from each other, and without them
|
||||||
previously recoverable only by reading `plugin_context.py`.
|
here they are recoverable only by reading `plugin_context.py`.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
@@ -375,13 +364,13 @@ class Surface:
|
|||||||
fires: str
|
fires: str
|
||||||
measured_model: str = "BAAI/bge-small-en-v1.5"
|
measured_model: str = "BAAI/bge-small-en-v1.5"
|
||||||
measured_shape: int = 1
|
measured_shape: int = 1
|
||||||
"""What the SHIPPED defaults above were measured against (#4104).
|
"""What the SHIPPED defaults above were measured against.
|
||||||
|
|
||||||
A floor is a distance in one embedding model's geometry, over documents cut
|
A floor is a distance in one embedding model's geometry, over documents cut
|
||||||
one particular way. Either can change, and when one does every number in
|
one particular way. Either can change, and when one does every number in
|
||||||
this table describes something that no longer exists.
|
this table describes something that no longer exists.
|
||||||
|
|
||||||
TWO FIELDS, NEVER ONE FUSED STRING (rule 149). A mismatch has to be able to
|
TWO FIELDS, NEVER ONE FUSED STRING. A mismatch has to be able to
|
||||||
say WHICH half moved: a new embedding model and a re-cut document shape
|
say WHICH half moved: a new embedding model and a re-cut document shape
|
||||||
invalidate the same numbers for different reasons and call for different
|
invalidate the same numbers for different reasons and call for different
|
||||||
responses. `"<model>@<n>"` could only report that something changed, which
|
responses. `"<model>@<n>"` could only report that something changed, which
|
||||||
@@ -401,11 +390,11 @@ class Surface:
|
|||||||
budget_falls_back_to: str = ""
|
budget_falls_back_to: str = ""
|
||||||
"""A budget key to inherit when this surface has none of its own set.
|
"""A budget key to inherit when this surface has none of its own set.
|
||||||
|
|
||||||
Only `write_path` uses it, and only because it USED to share auto-inject's
|
Only `write_path` uses it: it inherits auto-inject's `top_k` until its own
|
||||||
`top_k` outright. Giving it a key without this would silently reset the
|
key is set. Without the fallback, an install that tuned the shared budget
|
||||||
budget of every install that had tuned the shared one — a behaviour change
|
would have it silently reset — a behaviour change delivered as a default,
|
||||||
delivered as a default, which is the shape of regression nobody reports
|
which is the shape of regression nobody reports because nothing looks
|
||||||
because nothing looks broken.
|
broken.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
@@ -419,7 +408,7 @@ class Declared:
|
|||||||
"""One line, for an agent reading a warning that names this source."""
|
"""One line, for an agent reading a warning that names this source."""
|
||||||
|
|
||||||
quiet_because: str = ""
|
quiet_because: str = ""
|
||||||
"""Set when silence over an active window is correct, saying why (#2475)."""
|
"""Set when silence over an active window is correct, saying why."""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -442,7 +431,7 @@ class RuleArm:
|
|||||||
"""The telemetry `source`. MUST equal the registry and SURFACES key."""
|
"""The telemetry `source`. MUST equal the registry and SURFACES key."""
|
||||||
|
|
||||||
band: bool
|
band: bool
|
||||||
"""Keep only the hits within `_RULEHINT_BAND` of the top one (#3851)."""
|
"""Keep only the hits within `_RULEHINT_BAND` of the top one."""
|
||||||
|
|
||||||
compact_tail: bool
|
compact_tail: bool
|
||||||
"""Lines after the first cite the rule instead of quoting its trigger."""
|
"""Lines after the first cite the rule instead of quoting its trigger."""
|
||||||
@@ -451,13 +440,13 @@ class RuleArm:
|
|||||||
"""The top hit may hold the act for one read (`checkpoint_for`)."""
|
"""The top hit may hold the act for one read (`checkpoint_for`)."""
|
||||||
|
|
||||||
preference_slot: bool
|
preference_slot: bool
|
||||||
"""Reserve one line for a preference that lost the ranking (#3894)."""
|
"""Reserve one line for a preference that lost the ranking."""
|
||||||
|
|
||||||
kind: str | None = None
|
kind: str | None = None
|
||||||
"""Ask the ranker for one record kind only ("preference"), or every kind."""
|
"""Ask the ranker for one record kind only ("preference"), or every kind."""
|
||||||
|
|
||||||
stop_only: bool = False
|
stop_only: bool = False
|
||||||
"""The door shows nothing but the stop (milestone 458's reply moment).
|
"""The door shows nothing but the stop (the reply moment).
|
||||||
|
|
||||||
At the end of a turn there is no context to annotate: a hit either holds
|
At the end of a turn there is no context to annotate: a hit either holds
|
||||||
the reply for one read or reaches nobody. So only the checkpoint's rule is
|
the reply for one read or reaches nobody. So only the checkpoint's rule is
|
||||||
@@ -471,9 +460,9 @@ class RuleArm:
|
|||||||
declared: Declared | None = None
|
declared: Declared | None = None
|
||||||
|
|
||||||
|
|
||||||
# Today's differences, reproduced exactly (milestone 456 step 2). Whether the
|
# The arms differ on purpose where these flags differ. Whether the prompt arm
|
||||||
# prompt arm should band, and whether the act arms should reserve a
|
# should band, and whether the act arms should reserve a preference, are open
|
||||||
# preference, are step 7's questions — not settled by this refactor.
|
# questions to settle on telemetry, not by symmetry.
|
||||||
PROMPT_RULE = RuleArm(
|
PROMPT_RULE = RuleArm(
|
||||||
"prompt_rule", band=False, compact_tail=False, checkpoint=False,
|
"prompt_rule", band=False, compact_tail=False, checkpoint=False,
|
||||||
preference_slot=True,
|
preference_slot=True,
|
||||||
@@ -525,7 +514,7 @@ WRITE_PATH_RULE = RuleArm(
|
|||||||
REPLY_RULE = RuleArm(
|
REPLY_RULE = RuleArm(
|
||||||
"reply_rule", band=False, compact_tail=False, checkpoint=True,
|
"reply_rule", band=False, compact_tail=False, checkpoint=True,
|
||||||
preference_slot=False, stop_only=True,
|
preference_slot=False, stop_only=True,
|
||||||
# THE REPLY BACKSTOP (milestone 458, folded in from 456 step 8). Its floor
|
# THE REPLY BACKSTOP. Its floor
|
||||||
# is a STOP bar, not a hint bar: at the end of a turn nothing can be shown
|
# is a STOP bar, not a hint bar: at the end of a turn nothing can be shown
|
||||||
# beside the reply, so a hit either holds the reply for one read or says
|
# beside the reply, so a hit either holds the reply for one read or says
|
||||||
# nothing. Hence a default at the checkpoint's level and a budget of one —
|
# nothing. Hence a default at the checkpoint's level and a budget of one —
|
||||||
@@ -582,7 +571,7 @@ class RuleMoment:
|
|||||||
query: str
|
query: str
|
||||||
project_id: int | None
|
project_id: int | None
|
||||||
"""The bound project, 0 or None when unbound. Logged as given, searched as
|
"""The bound project, 0 or None when unbound. Logged as given, searched as
|
||||||
`project_id or None` — global rules plus that project's own (milestone 414)."""
|
`project_id or None` — global rules plus that project's own."""
|
||||||
|
|
||||||
where: str = ""
|
where: str = ""
|
||||||
"""How a line names the moment: "here", "to this Bash call", …"""
|
"""How a line names the moment: "here", "to this Bash call", …"""
|
||||||
@@ -592,7 +581,7 @@ class RuleMoment:
|
|||||||
"""Rules the session has been shown — rendered as repeats, not hidden."""
|
"""Rules the session has been shown — rendered as repeats, not hidden."""
|
||||||
|
|
||||||
held: frozenset[int] = frozenset()
|
held: frozenset[int] = frozenset()
|
||||||
"""Rules the session has OPENED (#4100)."""
|
"""Rules the session has OPENED."""
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -624,7 +613,7 @@ def _telemetry_failed(source: str) -> None:
|
|||||||
The recorders are called DIRECTLY at each site, inside their own try,
|
The recorders are called DIRECTLY at each site, inside their own try,
|
||||||
rather than through a wrapper taking the recorder as an argument: the
|
rather than through a wrapper taking the recorder as an argument: the
|
||||||
registry test finds recorder call sites by their name, and a wrapper would
|
registry test finds recorder call sites by their name, and a wrapper would
|
||||||
hide every site in this module from it (#3191's narrowing).
|
hide every site in this module from it.
|
||||||
"""
|
"""
|
||||||
logger.debug("retrieval telemetry failed for %s", source, exc_info=True)
|
logger.debug("retrieval telemetry failed for %s", source, exc_info=True)
|
||||||
|
|
||||||
@@ -637,7 +626,7 @@ async def _ranked(
|
|||||||
|
|
||||||
Returns (hits, kept, fresh): what the ranker returned, what the band kept,
|
Returns (hits, kept, fresh): what the ranker returned, what the band kept,
|
||||||
and the kept hits this session has not been shown. The row is written
|
and the kept hits this session has not been shown. The row is written
|
||||||
HERE, before any caller can return early (#3497), with `results` the fresh
|
HERE, before any caller can return early, with `results` the fresh
|
||||||
cut and `suppressed` counting both the band and the ledger, so a long
|
cut and `suppressed` counting both the band and the ledger, so a long
|
||||||
session's all-repeat zero reads apart from a bar that turned everything
|
session's all-repeat zero reads apart from a bar that turned everything
|
||||||
away.
|
away.
|
||||||
@@ -681,7 +670,7 @@ async def _ranked(
|
|||||||
async def _reserve_slot_for_preference(
|
async def _reserve_slot_for_preference(
|
||||||
io: RuleIO, moment: RuleMoment, hits: list, *, floor: float,
|
io: RuleIO, moment: RuleMoment, hits: list, *, floor: float,
|
||||||
) -> tuple[list, int | None]:
|
) -> tuple[list, int | None]:
|
||||||
"""Guarantee a preference one slot, if one clears the bar (#3894).
|
"""Guarantee a preference one slot, if one clears the bar.
|
||||||
|
|
||||||
THE ASYMMETRY THIS EXISTS FOR. A rule and a preference are not equally
|
THE ASYMMETRY THIS EXISTS FOR. A rule and a preference are not equally
|
||||||
served by a shared score contest, because their losses are not equal:
|
served by a shared score contest, because their losses are not equal:
|
||||||
@@ -697,33 +686,32 @@ async def _reserve_slot_for_preference(
|
|||||||
over the one whose loss is total, and it does so INVISIBLY: the rule that
|
over the one whose loss is total, and it does so INVISIBLY: the rule that
|
||||||
won is a legitimate hit, the telemetry looks healthy, and the only symptom
|
won is a legitimate hit, the telemetry looks healthy, and the only symptom
|
||||||
is a preference that quietly never arrives. `reuse_slot` exists for the
|
is a preference that quietly never arrives. `reuse_slot` exists for the
|
||||||
same shape one corpus over (#2463), where snippets kept losing to project
|
same shape one corpus over, where snippets lose to project records that
|
||||||
records that merely resembled the query.
|
merely resemble the query.
|
||||||
|
|
||||||
THE SLOT BUYS POSITION, NOT A LOWER BAR — same as `reuse_slot`, which also
|
THE SLOT BUYS POSITION, NOT A LOWER BAR — same as `reuse_slot`, which also
|
||||||
reserves at `cfg["threshold"]`. A weak preference cannot buy the slot, so
|
reserves at `cfg["threshold"]`. A weak preference cannot buy the slot, so
|
||||||
silence stays the default and the reserved line is never worse than the
|
silence stays the default and the reserved line is never worse than the
|
||||||
ones it sits beside. If `preference_slot` later shows a stream of
|
ones it sits beside. If `preference_slot` later shows a stream of
|
||||||
near-misses, `best_available_id` (#3807) names which preference was
|
near-misses, `best_available_id` names which preference was
|
||||||
refused and a separate bar becomes an argument with evidence behind it
|
refused and a separate bar becomes an argument with evidence behind it
|
||||||
rather than a knob added on a guess.
|
rather than a knob added on a guess.
|
||||||
|
|
||||||
LEDGER REPEATS STILL COUNT AS REPRESENTED. A preference already on the
|
LEDGER REPEATS STILL COUNT AS REPRESENTED. A preference already on the
|
||||||
session's ledger occupies the slot rather than being skipped for a fresh
|
session's ledger occupies the slot rather than being skipped for a fresh
|
||||||
one: it is still rendered (#3750), just with the tail that says so, and a
|
one: it is still rendered, just with the tail that says so, and a
|
||||||
preference is the kind of record where being reminded is the point.
|
preference is the kind of record where being reminded is the point.
|
||||||
|
|
||||||
Returns the possibly-extended hit list, and the id the slot spent — the
|
Returns the possibly-extended hit list, and the id the slot spent — the
|
||||||
caller needs that to keep each source's surfaced set matching its own log
|
caller needs that to keep each source's surfaced set matching its own log
|
||||||
row (#3668), since the slot logs under its own name.
|
row, since the slot logs under its own name.
|
||||||
"""
|
"""
|
||||||
if any(rule.kind == "preference" for _s, rule in hits):
|
if any(rule.kind == "preference" for _s, rule in hits):
|
||||||
return hits, None
|
return hits, None
|
||||||
|
|
||||||
# ITS OWN SOURCE, and both sides of the trade logged. #2463's own finding
|
# ITS OWN SOURCE, and both sides of the trade logged. A slot whose query
|
||||||
# is the warning rather than the precedent here: the hit that slot pushed
|
# is not logged beside the hit it pushed out can never be judged against
|
||||||
# OUT was in retrieval_logs while the query that pushed it out was not, so
|
# what it displaced.
|
||||||
# the slot could never be judged against what it displaced.
|
|
||||||
found, _kept, _fresh = await _ranked(
|
found, _kept, _fresh = await _ranked(
|
||||||
io, moment, source=PREFERENCE_SLOT_SOURCE, floor=floor, limit=1,
|
io, moment, source=PREFERENCE_SLOT_SOURCE, floor=floor, limit=1,
|
||||||
kind="preference",
|
kind="preference",
|
||||||
@@ -748,11 +736,10 @@ async def _reserve_slot_for_preference(
|
|||||||
# ledger rather than taste. A displaced hit was RETURNED by the general
|
# ledger rather than taste. A displaced hit was RETURNED by the general
|
||||||
# search and is sitting in that call's `retrieval_logs` row, but would not
|
# search and is sitting in that call's `retrieval_logs` row, but would not
|
||||||
# have been shown — so `prompt_rule`'s surfaced set would stop matching
|
# have been shown — so `prompt_rule`'s surfaced set would stop matching
|
||||||
# its own log row, and #3668's identity would break for a reason nothing
|
# its own log row, and that identity would break for a reason nothing in
|
||||||
# in the data explains. That identity is the cheapest true statement
|
# the data explains. It is the cheapest true statement available about
|
||||||
# available about this pair of tables, and milestone #379 is what it costs
|
# this pair of tables; without it, two counters disagreeing reads as a
|
||||||
# to lose it: five steps planned against a gap that was two counters
|
# write path dropping rows.
|
||||||
# disagreeing, not a write path dropping rows.
|
|
||||||
#
|
#
|
||||||
# The price is one extra line, only when the general search already filled
|
# The price is one extra line, only when the general search already filled
|
||||||
# the limit AND a preference cleared the bar without placing. Cheap, and
|
# the limit AND a preference cleared the bar without placing. Cheap, and
|
||||||
@@ -788,7 +775,7 @@ async def run_rule_arm(
|
|||||||
shown, _slot_id = await _reserve_slot_for_preference(
|
shown, _slot_id = await _reserve_slot_for_preference(
|
||||||
io, moment, shown, floor=floor,
|
io, moment, shown, floor=floor,
|
||||||
)
|
)
|
||||||
# `shown`, not `fresh` (#3750): a call whose only hit is a repeat
|
# `shown`, not `fresh`: a call whose only hit is a repeat
|
||||||
# still has something to say, it just says it differently.
|
# still has something to say, it just says it differently.
|
||||||
if not shown:
|
if not shown:
|
||||||
return RuleResult()
|
return RuleResult()
|
||||||
@@ -798,16 +785,16 @@ async def run_rule_arm(
|
|||||||
rule, where=moment.where,
|
rule, where=moment.where,
|
||||||
seen=rule.id in moment.exclude,
|
seen=rule.id in moment.exclude,
|
||||||
held=rule.id in moment.held,
|
held=rule.id in moment.held,
|
||||||
# Rank decides volume (#3851): the ranker's best guess gets
|
# Rank decides volume: the ranker's best guess gets
|
||||||
# the trigger, the rest get cited.
|
# the trigger, the rest get cited.
|
||||||
compact=arm.compact_tail and idx > 0,
|
compact=arm.compact_tail and idx > 0,
|
||||||
)
|
)
|
||||||
for idx, (_score, rule) in enumerate(shown)
|
for idx, (_score, rule) in enumerate(shown)
|
||||||
]
|
]
|
||||||
# FRESH-ONLY (#3752): a repeat is a rendering decision, not a
|
# FRESH-ONLY: a repeat is a rendering decision, not a retrieval
|
||||||
# retrieval outcome, and counting it would inflate the denominator
|
# outcome, and counting it would inflate the denominator
|
||||||
# pull_through is read from. It is also exactly what this call's own
|
# pull_through is read from. It is also exactly what this call's own
|
||||||
# row recorded, so the two tables stay equal (#3668). RANKED, not
|
# row recorded, so the two tables stay equal. RANKED, not
|
||||||
# ambient: the source is in `rule_usage.RANKED_SOURCES`.
|
# ambient: the source is in `rule_usage.RANKED_SOURCES`.
|
||||||
rule_ids = [rule.id for _score, rule in fresh]
|
rule_ids = [rule.id for _score, rule in fresh]
|
||||||
source = arm.source
|
source = arm.source
|
||||||
@@ -838,20 +825,19 @@ async def run_rule_arm(
|
|||||||
return RuleResult()
|
return RuleResult()
|
||||||
|
|
||||||
|
|
||||||
# ── The notes corpus (milestone 456 step 4) ──────────────────────────────
|
# ── The notes corpus ─────────────────────────────────────────────────────
|
||||||
#
|
#
|
||||||
# The same stages over the other corpus: search, withhold what this response
|
# The same stages over the other corpus: search, withhold what this response
|
||||||
# already lists, split fresh from repeats, log the call before any early
|
# already lists, split fresh from repeats, log the call before any early
|
||||||
# return, band, reserved slots, surfacing rows. Two arms take them — the
|
# return, band, reserved slots, surfacing rows. Two arms take them — the
|
||||||
# prompt's menu (`auto_inject`) and the write path's match by meaning
|
# prompt's menu (`auto_inject`) and the write path's match by meaning
|
||||||
# (`write_path`) — and they differed only in which kinds they ask for, whether
|
# (`write_path`) — and they differ only in which kinds they ask for, whether
|
||||||
# a menu sits above them in the same response, and which slots they reserve.
|
# a menu sits above them in the same response, and which slots they reserve.
|
||||||
#
|
#
|
||||||
# ONE DIFFERENCE FROM THE RULE ARMS IS KEPT, ON PURPOSE, FOR STEP 7. A notes
|
# ONE DIFFERENCE FROM THE RULE ARMS IS KEPT, ON PURPOSE. A notes arm logs the
|
||||||
# arm logs the cut BEFORE its band; a rule arm logs after it. #2085 chose the
|
# cut BEFORE its band; a rule arm logs after it. For notes the log row is the
|
||||||
# first for notes (the log row is the candidate set a floor is tuned against,
|
# candidate set a floor is tuned against and the surfacing rows are what the
|
||||||
# the surfacing rows are what the reader saw) and #3851 the second for rules.
|
# reader saw; for rules the band is part of the cut being judged.
|
||||||
# Both are recorded decisions, so this refactor reproduces both.
|
|
||||||
#
|
#
|
||||||
# What stays OUTSIDE: every lookup. The records named by number, the snippets
|
# What stays OUTSIDE: every lookup. The records named by number, the snippets
|
||||||
# recorded at a path, the rulings, the design system and the shape ledger
|
# recorded at a path, the rulings, the design system and the shape ledger
|
||||||
@@ -860,7 +846,7 @@ async def run_rule_arm(
|
|||||||
|
|
||||||
# Margin gate: drop any hit more than this far below the top hit's score, so a
|
# Margin gate: drop any hit more than this far below the top hit's score, so a
|
||||||
# single strong match doesn't drag in a wall of barely-passing neighbours.
|
# single strong match doesn't drag in a wall of barely-passing neighbours.
|
||||||
# Twice the rule band, which is narrow because the rule corpus is flat (#3851).
|
# Twice the rule band, which is narrow because the rule corpus is flat.
|
||||||
_NOTE_BAND = 0.10
|
_NOTE_BAND = 0.10
|
||||||
|
|
||||||
|
|
||||||
@@ -877,9 +863,8 @@ def _record_kind(note) -> str:
|
|||||||
|
|
||||||
A TASK ALSO CARRIES ITS STATUS, because for that kind alone the line is
|
A TASK ALSO CARRIES ITS STATUS, because for that kind alone the line is
|
||||||
read as a claim about live work. A finished step and an open one rendered
|
read as a claim about live work. A finished step and an open one rendered
|
||||||
identically is not a cosmetic gap: a done step was cited as a milestone's
|
identically is not a cosmetic gap: a line carrying only an id, a kind and
|
||||||
open one on the strength of a line exactly like this, which carries an id,
|
a title invites a reader to cite a done step as open work.
|
||||||
a kind and a title and said nothing about where the work stood (#4154).
|
|
||||||
Only for tasks — a note or a snippet has no status to be wrong about.
|
Only for tasks — a note or a snippet has no status to be wrong about.
|
||||||
"""
|
"""
|
||||||
if note.is_task:
|
if note.is_task:
|
||||||
@@ -895,22 +880,22 @@ def _record_kind(note) -> str:
|
|||||||
return note.note_type or "note"
|
return note.note_type or "note"
|
||||||
|
|
||||||
|
|
||||||
# WHAT A MENU LINE CARRIES (#4364): the record's NAME, its kind and System,
|
# WHAT A MENU LINE CARRIES: the record's NAME, its kind and System,
|
||||||
# and the WHOLE passage that matched. Metadata plus the evidence, rather than a
|
# and the WHOLE passage that matched. Metadata plus the evidence, rather than a
|
||||||
# title asked to be both.
|
# title asked to be both.
|
||||||
#
|
#
|
||||||
# The name, not the title. A snippet's or lesson's title is `name — when it
|
# The name, not the title. A snippet's or lesson's title is `name — when it
|
||||||
# applies` by construction (`embeddings.trigger_title`), because that join is
|
# applies` by construction (`embeddings.trigger_title`), because that join is
|
||||||
# what makes it rank on its situation. That is an EMBEDDING shape, and rendered
|
# what makes it rank on its situation. That is an EMBEDDING shape, and rendered
|
||||||
# as a menu line it ran to 1,500+ characters — the trigger paragraph spent
|
# as a menu line it runs to 1,500+ characters — the trigger paragraph spent
|
||||||
# again on every line, and again on every repeat. The trigger still arrives
|
# again on every line, and again on every repeat. The trigger still arrives
|
||||||
# when it is what matched: it is in the chunk, and `_menu_passage` hands it
|
# when it is what matched: it is in the chunk, and `_menu_passage` hands it
|
||||||
# over when the title was the whole match.
|
# over when the title was the whole match.
|
||||||
#
|
#
|
||||||
# The whole passage, not 200 characters of it. The search already chose the
|
# The whole passage, not a head-and-tail cut of it. The search already chose
|
||||||
# chunk that matched; the old cut kept its head and tail, and the head is the
|
# the chunk that matched, and its head is the title every chunk is prefixed
|
||||||
# title every chunk is prefixed with — so the reader got the title twice and
|
# with — so a cut gives the reader the title twice and loses the middle,
|
||||||
# lost the middle, which is where the match was (lesson #4248). A chunk is at
|
# which is where the match was. A chunk is at
|
||||||
# most ~1.4 KB (`embeddings._CHUNK_CHAR_BUDGET`), and it is shown once: a
|
# most ~1.4 KB (`embeddings._CHUNK_CHAR_BUDGET`), and it is shown once: a
|
||||||
# repeat is a one-line pointer (`_menu_seen_line`), not a second copy.
|
# repeat is a one-line pointer (`_menu_seen_line`), not a second copy.
|
||||||
|
|
||||||
@@ -928,7 +913,7 @@ def _menu_name(title: str | None, note_type: str | None, data=None, body: str |
|
|||||||
from scribe.services.embeddings import untrigger_title
|
from scribe.services.embeddings import untrigger_title
|
||||||
from scribe.services.lessons import lesson_trigger
|
from scribe.services.lessons import lesson_trigger
|
||||||
trigger = lesson_trigger(SimpleNamespace(data=data, body=body or ""))
|
trigger = lesson_trigger(SimpleNamespace(data=data, body=body or ""))
|
||||||
# One line even when the stored name is a story (#4797).
|
# One line even when the stored name is a story.
|
||||||
return claim_line(untrigger_title(title, trigger).strip() or title)
|
return claim_line(untrigger_title(title, trigger).strip() or title)
|
||||||
return title
|
return title
|
||||||
|
|
||||||
@@ -972,16 +957,16 @@ def menu_entry(
|
|||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
"""One record on a notes menu: its line, and what sits under it.
|
"""One record on a notes menu: its line, and what sits under it.
|
||||||
|
|
||||||
The prompt menu's two blocks — records named by number and records ranked
|
Shared by the prompt menu's two blocks — records named by number and
|
||||||
by meaning — used to write this out twice, differing only in what they
|
records ranked by meaning — which differ only in what they pass: a ranked
|
||||||
pass: a ranked line carries its score, a named one does not, and each puts
|
line carries its score, a named one does not, and each puts its own text
|
||||||
its own text under the line (the matched passage, or the record's opening).
|
under the line (the matched passage, or the record's opening).
|
||||||
|
|
||||||
A repeat is a POINTER, not a copy (#4364). The record is in this session's
|
A repeat is a POINTER, not a copy. The record is in this session's
|
||||||
context already — the ledger is cleared at compaction, so "seen" stays true
|
context already — the ledger is cleared at compaction, so "seen" stays true
|
||||||
— and re-rendering it spent its whole line again for nothing.
|
— and re-rendering it would spend its whole line again for nothing.
|
||||||
|
|
||||||
A superseded record is DEMOTED, not removed (#278), so one can still reach
|
A superseded record is DEMOTED, not removed, so one can still reach
|
||||||
a menu, and when it does the reader has to be told: an agent handed stale
|
a menu, and when it does the reader has to be told: an agent handed stale
|
||||||
material with nothing marking it acts on it with full confidence.
|
material with nothing marking it acts on it with full confidence.
|
||||||
"""
|
"""
|
||||||
@@ -1047,7 +1032,7 @@ class NoteArm:
|
|||||||
note_type: tuple[str, ...] | None = None
|
note_type: tuple[str, ...] | None = None
|
||||||
task_kind: str | None = None
|
task_kind: str | None = None
|
||||||
include_global_kinds: bool = True
|
include_global_kinds: bool = True
|
||||||
"""Lessons are project-independent (#3730), so they join the candidate set
|
"""Lessons are project-independent, so they join the candidate set
|
||||||
from wherever they were learned."""
|
from wherever they were learned."""
|
||||||
|
|
||||||
scope: str = "browse"
|
scope: str = "browse"
|
||||||
@@ -1058,7 +1043,7 @@ class NoteArm:
|
|||||||
withholds: bool = False
|
withholds: bool = False
|
||||||
"""A menu sits above this arm in the same response, and what it lists is
|
"""A menu sits above this arm in the same response, and what it lists is
|
||||||
kept out of the search (`exclude_ids`) — same-call duplication, which is a
|
kept out of the search (`exclude_ids`) — same-call duplication, which is a
|
||||||
different claim from the session ledger (#4101)."""
|
different claim from the session ledger."""
|
||||||
|
|
||||||
tuning: Surface | None = None
|
tuning: Surface | None = None
|
||||||
declared: Declared | None = None
|
declared: Declared | None = None
|
||||||
@@ -1078,11 +1063,11 @@ AUTO_INJECT = NoteArm(
|
|||||||
),
|
),
|
||||||
declared=Declared("the notes menu offered at the prompt boundary"),
|
declared=Declared("the notes menu offered at the prompt boundary"),
|
||||||
)
|
)
|
||||||
# Snippets AND recorded experience (#2246): an issue saying "we tried this and
|
# Snippets AND recorded experience: an issue saying "we tried this and it
|
||||||
# it deadlocked" is prior art for the code about to be written. `task_kind`
|
# deadlocked" is prior art for the code about to be written. `task_kind`
|
||||||
# keeps the open to-do list out — a task resembles the code and answers
|
# keeps the open to-do list out — a task resembles the code and answers
|
||||||
# nothing. Lessons too (milestone 385 step 5): the arm is kind-FILTERED, so a
|
# nothing. Lessons too: the arm is kind-FILTERED, so a kind absent here is
|
||||||
# kind absent here is unreachable, not merely outranked (#3702). No reserved
|
# unreachable, not merely outranked. No reserved
|
||||||
# slot: this arm fires before every Write and Edit, and the field is already
|
# slot: this arm fires before every Write and Edit, and the field is already
|
||||||
# narrow enough that the 200:1 dilution a slot answers does not happen.
|
# narrow enough that the 200:1 dilution a slot answers does not happen.
|
||||||
WRITE_PATH = NoteArm(
|
WRITE_PATH = NoteArm(
|
||||||
@@ -1153,7 +1138,7 @@ class NoteMoment:
|
|||||||
"""Searched and logged as `project_id or None`."""
|
"""Searched and logged as `project_id or None`."""
|
||||||
|
|
||||||
seen: frozenset[int] = frozenset()
|
seen: frozenset[int] = frozenset()
|
||||||
"""The session ledger: rendered as a pointer, never withheld (#4101)."""
|
"""The session ledger: rendered as a pointer, never withheld."""
|
||||||
|
|
||||||
named: frozenset[int] = frozenset()
|
named: frozenset[int] = frozenset()
|
||||||
"""Records this response shows by LOOKUP (named by number). Shown in their
|
"""Records this response shows by LOOKUP (named by number). Shown in their
|
||||||
@@ -1192,8 +1177,7 @@ def _note_band(hits: list) -> list:
|
|||||||
|
|
||||||
Computed over ALL hits, repeats included: the band measures distance from
|
Computed over ALL hits, repeats included: the band measures distance from
|
||||||
the top SCORE, and letting the ledger move that cutoff would make "you
|
the top SCORE, and letting the ledger move that cutoff would make "you
|
||||||
were shown this" change what counts as relevant (#3851's axis
|
were shown this" change what counts as relevant.
|
||||||
independence).
|
|
||||||
"""
|
"""
|
||||||
if not hits:
|
if not hits:
|
||||||
return []
|
return []
|
||||||
@@ -1207,11 +1191,11 @@ async def _reserve_note_slot(
|
|||||||
) -> tuple[list, int | None]:
|
) -> tuple[list, int | None]:
|
||||||
"""Guarantee `slot.kinds` one line, if one clears the arm's own bar.
|
"""Guarantee `slot.kinds` one line, if one clears the arm's own bar.
|
||||||
|
|
||||||
WHY A SLOT AT ALL (#2246, milestone 385 step 5). Ranking by raw cosine is
|
WHY A SLOT AT ALL. Ranking by raw cosine is blind to what KIND of record
|
||||||
blind to what KIND of record answers what kind of ask, and the corpus
|
answers what kind of ask, and the corpus makes that fatal: Scribe's
|
||||||
makes that fatal: Scribe's project records are about software work, so a
|
project records are about software work, so a task about building a
|
||||||
task about building a helper outranks the snippet that IS one. Snippets
|
helper outranks the snippet that IS one. Snippets can be ~0.5% of a
|
||||||
were ~0.5% of the corpus when this was measured; no floor fixes 200:1. A
|
corpus, and no floor fixes 200:1. A
|
||||||
lesson crowded out is worse still — it exists only to be met at the moment
|
lesson crowded out is worse still — it exists only to be met at the moment
|
||||||
it applies, so the arm that surfaces it IS its delivery.
|
it applies, so the arm that surfaces it IS its delivery.
|
||||||
|
|
||||||
@@ -1219,11 +1203,11 @@ async def _reserve_note_slot(
|
|||||||
floor and is NOT held to the band (the top score is the very thing these
|
floor and is NOT held to the band (the top score is the very thing these
|
||||||
kinds lose to), so a weak record cannot buy the line.
|
kinds lose to), so a weak record cannot buy the line.
|
||||||
|
|
||||||
ITS OWN SOURCE, from the first deploy. A guarantee has to be falsifiable,
|
ITS OWN SOURCE. A guarantee has to be falsifiable, and the hit a slot
|
||||||
and the hit a slot pushed out sits in the arm's row while the query that
|
pushed out sits in the arm's row while the query that pushed it out would
|
||||||
pushed it out would otherwise be nowhere (#2463).
|
otherwise be nowhere.
|
||||||
|
|
||||||
THE LEDGER IS NOT AN EXCLUSION HERE EITHER (#4101), but this call's own
|
THE LEDGER IS NOT AN EXCLUSION HERE EITHER, but this call's own
|
||||||
menu is: a record already on it must not be shown twice, while one shown
|
menu is: a record already on it must not be shown twice, while one shown
|
||||||
in an EARLIER call is exactly what a slot may spend itself on — a snippet
|
in an EARLIER call is exactly what a slot may spend itself on — a snippet
|
||||||
relevant then and now is the reuse case, not a duplicate of it.
|
relevant then and now is the reuse case, not a duplicate of it.
|
||||||
@@ -1270,7 +1254,7 @@ async def _reserve_note_slot(
|
|||||||
return menu, None
|
return menu, None
|
||||||
|
|
||||||
slot_id = int(placed[0][1].id)
|
slot_id = int(placed[0][1].id)
|
||||||
# FRESH ONLY, matching the row above: a ledger repeat is rendered (#4101)
|
# FRESH ONLY, matching the row above: a ledger repeat is rendered
|
||||||
# but is not a new surfacing.
|
# but is not a new surfacing.
|
||||||
if slot.books_own and slot_id not in shown:
|
if slot.books_own and slot_id not in shown:
|
||||||
try:
|
try:
|
||||||
@@ -1283,7 +1267,7 @@ async def _reserve_note_slot(
|
|||||||
if not slot.evicts:
|
if not slot.evicts:
|
||||||
# IT EXTENDS, IT NEVER DISPLACES, siding with `preference_slot`: a
|
# IT EXTENDS, IT NEVER DISPLACES, siding with `preference_slot`: a
|
||||||
# displaced hit sits in the arm's row, and evicting it would make the
|
# displaced hit sits in the arm's row, and evicting it would make the
|
||||||
# two tables disagree about the same call (#3668). And a record that
|
# two tables disagree about the same call. And a record that
|
||||||
# does not bind should not throw a better-scoring one off the menu.
|
# does not bind should not throw a better-scoring one off the menu.
|
||||||
return menu + placed, slot_id
|
return menu + placed, slot_id
|
||||||
# Take the LAST line, never the first: the strongest overall hit is still
|
# Take the LAST line, never the first: the strongest overall hit is still
|
||||||
@@ -1316,15 +1300,15 @@ async def run_note_arm(
|
|||||||
kwargs["exclude_ids"] = set(moment.in_menu - scored)
|
kwargs["exclude_ids"] = set(moment.in_menu - scored)
|
||||||
answered = await io.search(moment.user_id, moment.query, **kwargs)
|
answered = await io.search(moment.user_id, moment.query, **kwargs)
|
||||||
hits = [(s, n) for s, n in answered if int(n.id) not in moment.in_menu]
|
hits = [(s, n) for s, n in answered if int(n.id) not in moment.in_menu]
|
||||||
# WHAT THIS ARM WITHHELD AFTER THE SEARCH ANSWERED (#3739). The score
|
# WHAT THIS ARM WITHHELD AFTER THE SEARCH ANSWERED. The score the
|
||||||
# the search reports is measured BEFORE that drop, so a record already
|
# search reports is measured BEFORE that drop, so a record already
|
||||||
# listed in this response could be logged as one the BAR turned away
|
# listed in this response could be logged as one the BAR turned away
|
||||||
# — live proof once read a "rejection" at 0.822 against a lowest
|
# — a "rejection" scoring well above the lowest acceptance. Whenever
|
||||||
# acceptance of 0.6857. Whenever this removed anything, the honest
|
# this removed anything, the honest best-available is null: "not
|
||||||
# best-available is null: "not measured on this call".
|
# measured on this call".
|
||||||
withheld = len(answered) - len(hits)
|
withheld = len(answered) - len(hits)
|
||||||
hits = hits[:budget]
|
hits = hits[:budget]
|
||||||
# `results=fresh` and `suppressed` together (#3752): a rendered repeat
|
# `results=fresh` and `suppressed` together: a rendered repeat
|
||||||
# is not a new surfacing, so it is COUNTED rather than reported, and an
|
# is not a new surfacing, so it is COUNTED rather than reported, and an
|
||||||
# all-repeat zero reads apart from a bar nothing cleared. A record
|
# all-repeat zero reads apart from a bar nothing cleared. A record
|
||||||
# NAMED in this response is counted the same way — shown, but by the
|
# NAMED in this response is counted the same way — shown, but by the
|
||||||
@@ -1374,8 +1358,8 @@ async def run_note_arm(
|
|||||||
result.menu = menu
|
result.menu = menu
|
||||||
|
|
||||||
# What SURVIVED the band — the menu the reader saw — where the call
|
# What SURVIVED the band — the menu the reader saw — where the call
|
||||||
# row holds the candidate set a floor is tuned against (#2085). FRESH
|
# row holds the candidate set a floor is tuned against. FRESH ONLY,
|
||||||
# ONLY, the row's own cut (#4101, #3668). A slot that books its own
|
# the row's own cut. A slot that books its own
|
||||||
# line is not this arm's surfacing: counting it twice would leave the
|
# line is not this arm's surfacing: counting it twice would leave the
|
||||||
# slot's two tables describing different numbers of one event.
|
# slot's two tables describing different numbers of one event.
|
||||||
own = {
|
own = {
|
||||||
@@ -1401,7 +1385,7 @@ async def run_note_arm(
|
|||||||
return NoteResult()
|
return NoteResult()
|
||||||
|
|
||||||
|
|
||||||
# ── A rule reached through its lessons (milestone 440, #4633) ────────────
|
# ── A rule reached through its lessons ───────────────────────────────────
|
||||||
#
|
#
|
||||||
# A rule's own document is written in the rule's words, which are general by
|
# A rule's own document is written in the rule's words, which are general by
|
||||||
# design; the situations that keep proving it are often closer to what a
|
# design; the situations that keep proving it are often closer to what a
|
||||||
@@ -1411,11 +1395,10 @@ async def run_note_arm(
|
|||||||
# It searches the NOTES corpus and answers with rules, which is why it sits
|
# It searches the NOTES corpus and answers with rules, which is why it sits
|
||||||
# between the two halves of this module.
|
# between the two halves of this module.
|
||||||
#
|
#
|
||||||
# ITS OWN SLOT, NOT A RULE SLOT, as a stated default. There was nothing to
|
# ITS OWN SLOT, NOT A RULE SLOT, as a stated default decided by asymmetry: a
|
||||||
# measure until links are confirmed (#4632), so the asymmetry decides it: a
|
|
||||||
# via-lesson line that took a rule slot could push out a rule the ranker
|
# via-lesson line that took a rule slot could push out a rule the ranker
|
||||||
# matched DIRECTLY, a stronger claim displaced by a weaker one, while an extra
|
# matched DIRECTLY, a stronger claim displaced by a weaker one, while an extra
|
||||||
# line costs one line. Logged as its own source so it can be judged (#4636).
|
# line costs one line. Logged as its own source so it can be judged.
|
||||||
|
|
||||||
VIA_LESSON_SOURCE = "rule_via_lesson"
|
VIA_LESSON_SOURCE = "rule_via_lesson"
|
||||||
VIA_LESSON = RankedSource(
|
VIA_LESSON = RankedSource(
|
||||||
@@ -1493,7 +1476,8 @@ async def run_via_lesson_arm(
|
|||||||
chosen.append((s, rule, n))
|
chosen.append((s, rule, n))
|
||||||
if len(chosen) >= VIA_LESSON_LIMIT:
|
if len(chosen) >= VIA_LESSON_LIMIT:
|
||||||
break
|
break
|
||||||
# Logged whenever a search ran, results or not — the #3497 guard. The
|
# Logged whenever a search ran, results or not: a call that found
|
||||||
|
# nothing is the only evidence a bar is too high. The
|
||||||
# score is the LESSON's, because the lesson is what was matched; the
|
# score is the LESSON's, because the lesson is what was matched; the
|
||||||
# best-available id is left off for the same reason, since the row's
|
# best-available id is left off for the same reason, since the row's
|
||||||
# results are rules and an id beside them would read as a rule id.
|
# results are rules and an id beside them would read as a rule id.
|
||||||
@@ -1536,7 +1520,7 @@ async def run_via_lesson_arm(
|
|||||||
return RuleResult()
|
return RuleResult()
|
||||||
|
|
||||||
|
|
||||||
# ── The specs, read as lists (milestone 456 step 5) ──────────────────────
|
# ── The specs, read as lists ─────────────────────────────────────────────
|
||||||
#
|
#
|
||||||
# What `retrieval_surfaces.SURFACES`, the ranked rows of
|
# What `retrieval_surfaces.SURFACES`, the ranked rows of
|
||||||
# `retrieval_registry.POINTS` and `rule_usage.RANKED_SOURCES` are read from.
|
# `retrieval_registry.POINTS` and `rule_usage.RANKED_SOURCES` are read from.
|
||||||
@@ -1558,20 +1542,20 @@ RULE_RANKED_SOURCES: tuple[str, ...] = (
|
|||||||
pull can confirm or refute it (`rule_usage.RANKED_SOURCES`)."""
|
pull can confirm or refute it (`rule_usage.RANKED_SOURCES`)."""
|
||||||
|
|
||||||
|
|
||||||
# ── The moment arm (milestone 458) ───────────────────────────────────────
|
# ── The moment arm ───────────────────────────────────────────────────────
|
||||||
#
|
#
|
||||||
# A LOOKUP beside the ranked arms, not one of them. A rule mounted on a moment
|
# A LOOKUP beside the ranked arms, not one of them. A rule mounted on a moment
|
||||||
# arrives because somebody said it belongs there, not because the work's words
|
# arrives because somebody said it belongs there, not because the work's words
|
||||||
# resemble it — which is the whole point: a rule about WHEN work is finished
|
# resemble it — which is the whole point: a rule about WHEN work is finished
|
||||||
# never scores against what is said while finishing it. So there is no score,
|
# never scores against what is said while finishing it. So there is no score,
|
||||||
# no floor and no band, and it writes no retrieval_logs row (the rulings arm's
|
# no floor and no band, and it writes no retrieval_logs row (as the rulings
|
||||||
# precedent, #4769): every score-shaped warning would misread a lookup. What it
|
# arm writes none): every score-shaped warning would misread a lookup. What it
|
||||||
# records is the surfacing, with the moment beside each row, so a mount's
|
# records is the surfacing, with the moment beside each row, so a mount's
|
||||||
# pull-through can be read per moment.
|
# pull-through can be read per moment.
|
||||||
#
|
#
|
||||||
# It shares everything else with the ranked arms: the line renderer, the
|
# It shares everything else with the ranked arms: the line renderer, the
|
||||||
# session ledger (a repeat renders as a citation, never hidden — #3750) and
|
# session ledger (a repeat renders as a citation, never hidden) and the
|
||||||
# the fresh-only surfacing rows (#3752).
|
# fresh-only surfacing rows.
|
||||||
|
|
||||||
MOMENT_RULE_SOURCE = "moment_rule"
|
MOMENT_RULE_SOURCE = "moment_rule"
|
||||||
|
|
||||||
@@ -1608,7 +1592,7 @@ async def run_moment_arm(
|
|||||||
action that reached it — and every line says both ("at work.deliver,
|
action that reached it — and every line says both ("at work.deliver,
|
||||||
reached by `git push`"), so a moment that fired on the wrong act is
|
reached by `git push`"), so a moment that fired on the wrong act is
|
||||||
visible in the line itself and can be corrected in the session with
|
visible in the line itself and can be corrected in the session with
|
||||||
`unmap_action` (step 2's ruling). Fresh rules come first and carry their
|
`unmap_action`. Fresh rules come first and carry their
|
||||||
trigger; a rule the session was already shown is cited, not repeated in
|
trigger; a rule the session was already shown is cited, not repeated in
|
||||||
full. Fails open to an empty result.
|
full. Fails open to an empty result.
|
||||||
"""
|
"""
|
||||||
|
|||||||
Reference in New Issue
Block a user