feat(moments): the reply moment holds a finished reply for one read (milestone 458 step 4b, #4922)
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Failing after 1m25s
CI & Build / Build & push image (push) Skipped
CI & Build / Python lint (push) Successful in 2s
CI & Build / Plugin hooks (push) Successful in 14s
CI & Build / TypeScript typecheck (push) Successful in 55s
CI & Build / integration (push) Successful in 1m1s
CI & Build / Python tests (push) Failing after 1m25s
CI & Build / Build & push image (push) Skipped
The reply is the one act no tool call marks, and it is where "let me know if it works" gets said. A new Stop hook (scribe_reply_check.sh) sends the finished reply to POST /api/plugin/reply-rules, which checks it twice: - mounted: every unopened RULE on reply.report, plus reply.ask when the reply asks a question. Deterministic. - semantic: the reply's head and tail against every rule's trigger, on a new ranked surface, reply_rule. It is the backstop for whatever the earlier arms missed. Its floor is its stop bar (default 0.80, budget 1), with its own Settings dials. The new stop_only stage records surfacing for the rule that holds and nothing else, because nothing else reached anyone. Following the operator's ruling from 456 step 8, a rule that holds blocks once, in the server's words. The hook blocks only on a reason it was given, so an unreachable instance never stops a session, and it never holds the rewrite. The ledger is the act checkpoint's own, so a rule holds a session once across both doors and the per-session cap counts both. The turn reader moved from the report check into scribe_defs.sh (scribe_turn_facts / scribe_turn_fact), so the two Stop hooks read a turn the same way. The output was checked identical on a real transcript. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -98,6 +98,9 @@ const kbToolRuleThreshold = ref("0.68");
|
||||
// prose rather than anything a tool produced (#3852).
|
||||
const kbPromptRuleThreshold = ref("0.72");
|
||||
const kbReportPrefThreshold = ref("0.72");
|
||||
// The reply arm (milestone 458): its bar is the bar at which a finished reply
|
||||
// is HELD for one read, so it sits with the checkpoint, not with the hints.
|
||||
const kbReplyRuleThreshold = ref("0.8");
|
||||
// The BUDGETS, one per arm (#4102). Until this step only auto-inject had one
|
||||
// and every other arm's ceiling was a module constant nobody could reach — so
|
||||
// the only control an operator had over a noisy surface was to raise its bar,
|
||||
@@ -108,6 +111,7 @@ const kbRuleHintTopK = ref("5");
|
||||
const kbToolRuleTopK = ref("5");
|
||||
const kbPromptRuleTopK = ref("3");
|
||||
const kbReportPrefTopK = ref("3");
|
||||
const kbReplyRuleTopK = ref("1");
|
||||
// What has been changed about retrieval, newest first — the review surface for
|
||||
// changes the model made on the operator's behalf (#4102).
|
||||
const tuningEvents = ref<TuningEvent[]>([]);
|
||||
@@ -313,6 +317,8 @@ async function saveKbInject() {
|
||||
// hold the first command of every session behind whatever ranked first.
|
||||
const cpT = asBar(kbCheckpointThreshold.value, 0.8);
|
||||
const rpT = asBar(kbReportPrefThreshold.value, 0.72);
|
||||
// A stop bar like the checkpoint's: a fallback of 0 would hold every reply.
|
||||
const ryT = asBar(kbReplyRuleThreshold.value, 0.8);
|
||||
// The budgets, clamped the way the server clamps them: a whole number in
|
||||
// [1, 10]. Never 0 — an arm turned off is turned off by its switch, and a
|
||||
// budget of zero would run the search, log the retrieval and render nothing,
|
||||
@@ -324,11 +330,13 @@ async function saveKbInject() {
|
||||
const trK = asK(kbToolRuleTopK.value, 5);
|
||||
const prK = asK(kbPromptRuleTopK.value, 3);
|
||||
const rpK = asK(kbReportPrefTopK.value, 3);
|
||||
const ryK = asK(kbReplyRuleTopK.value, 1);
|
||||
kbWritePathTopK.value = String(wpK);
|
||||
kbRuleHintTopK.value = String(rhK);
|
||||
kbToolRuleTopK.value = String(trK);
|
||||
kbPromptRuleTopK.value = String(prK);
|
||||
kbReportPrefTopK.value = String(rpK);
|
||||
kbReplyRuleTopK.value = String(ryK);
|
||||
kbInjectThreshold.value = String(t);
|
||||
kbInjectTopK.value = String(k);
|
||||
kbDupThresholdSnippet.value = String(dupSnip);
|
||||
@@ -346,6 +354,7 @@ async function saveKbInject() {
|
||||
kbToolRuleThreshold.value = String(trT);
|
||||
kbPromptRuleThreshold.value = String(prT);
|
||||
kbReportPrefThreshold.value = String(rpT);
|
||||
kbReplyRuleThreshold.value = String(ryT);
|
||||
savingKbInject.value = true;
|
||||
kbInjectSaved.value = false;
|
||||
try {
|
||||
@@ -379,6 +388,9 @@ async function saveKbInject() {
|
||||
// and a constant that lands under the bar is a dead arm, not a quiet
|
||||
// one (#3860).
|
||||
kb_reportpref_threshold: String(rpT),
|
||||
// The reply backstop's own bar: it HOLDS a reply, so like the
|
||||
// checkpoint it is never derived from a hint bar.
|
||||
kb_replyrule_threshold: String(ryT),
|
||||
// The budgets. Every one of these keys is recognised by the server as a
|
||||
// retrieval dial, so this save is recorded in the tuning history as a
|
||||
// change the OPERATOR made — which is the one entry the model must not
|
||||
@@ -388,6 +400,7 @@ async function saveKbInject() {
|
||||
kb_toolrule_top_k: String(trK),
|
||||
kb_promptrule_top_k: String(prK),
|
||||
kb_reportpref_top_k: String(rpK),
|
||||
kb_replyrule_top_k: String(ryK),
|
||||
kb_duplicate_threshold_snippet: String(dupSnip),
|
||||
kb_duplicate_threshold_note: String(dupNote),
|
||||
kb_duplicate_threshold_task: String(dupTask),
|
||||
@@ -859,6 +872,9 @@ onMounted(async () => {
|
||||
if (allSettings.kb_reportpref_threshold !== undefined) {
|
||||
kbReportPrefThreshold.value = allSettings.kb_reportpref_threshold;
|
||||
}
|
||||
if (allSettings.kb_replyrule_threshold !== undefined) {
|
||||
kbReplyRuleThreshold.value = allSettings.kb_replyrule_threshold;
|
||||
}
|
||||
if (allSettings.kb_writepath_threshold !== undefined) {
|
||||
kbWritePathThreshold.value = allSettings.kb_writepath_threshold;
|
||||
}
|
||||
@@ -882,6 +898,9 @@ onMounted(async () => {
|
||||
if (allSettings.kb_reportpref_top_k !== undefined) {
|
||||
kbReportPrefTopK.value = allSettings.kb_reportpref_top_k;
|
||||
}
|
||||
if (allSettings.kb_replyrule_top_k !== undefined) {
|
||||
kbReplyRuleTopK.value = allSettings.kb_replyrule_top_k;
|
||||
}
|
||||
await loadTuningHistory();
|
||||
await loadSurfaces();
|
||||
if (allSettings.kb_duplicate_threshold_snippet !== undefined) {
|
||||
@@ -1919,6 +1938,41 @@ async function deleteUser(userId: number) {
|
||||
/>
|
||||
<p class="field-hint">How many preferences a finished task may be shown (1–10).</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-replyrule-threshold">Reply check threshold (0–1)</label>
|
||||
<input
|
||||
id="kb-replyrule-threshold"
|
||||
v-model="kbReplyRuleThreshold"
|
||||
type="number"
|
||||
min="0"
|
||||
max="1"
|
||||
step="0.01"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">
|
||||
The bar at which a <em>finished reply</em> is held for one read: the
|
||||
reply is searched against every rule's trigger, and a rule this
|
||||
session has not opened that scores at or above this bar holds the
|
||||
reply once, naming the rule. It is the backstop for whatever the
|
||||
earlier checks missed, so it is a stop bar like the one above, not a
|
||||
hint bar — keep it high.
|
||||
</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="kb-replyrule-topk">Rules searched per reply</label>
|
||||
<input
|
||||
id="kb-replyrule-topk"
|
||||
v-model="kbReplyRuleTopK"
|
||||
type="number"
|
||||
min="1"
|
||||
max="10"
|
||||
step="1"
|
||||
class="fs-input input"
|
||||
style="max-width: 8rem"
|
||||
/>
|
||||
<p class="field-hint">How many of the best-scoring rules a reply is checked against (1–10). Only the best can hold the reply; the rest are counted, which is what tells you how close the others came.</p>
|
||||
</div>
|
||||
<!-- THE REVIEW SURFACE (#4102). These numbers are maintained by the
|
||||
model that uses them: it reads which records each bar refused and
|
||||
moves the dial with the argument attached. This panel is the other
|
||||
|
||||
Reference in New Issue
Block a user